fix: clean Apply Now URLs and disable TED demo scraper
- Strip tracking query params from find_tender URLs (?origin=SearchResults) - Disable TED EU scraper (requires browser automation, was using demo data) - Update 220 find_tender database records with clean URLs - Delete 4 TED demo records from database - Add URL_FIX_SUMMARY.md documentation All 615 tenders now have direct links to tender detail pages. Fixes Apply Now button UX issue.
This commit is contained in:
57
URL_FIX_SUMMARY.md
Normal file
57
URL_FIX_SUMMARY.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# TenderRadar Apply Now Button - Fix Complete
|
||||
|
||||
**Date:** 2026-02-15
|
||||
**Status:** ✅ FIXED
|
||||
|
||||
## Summary
|
||||
|
||||
Fixed "Apply Now" button URLs for 615 tenders across 5 data sources. All URLs now link directly to tender detail pages instead of search portals.
|
||||
|
||||
## Changes Made
|
||||
|
||||
### 1. Find a Tender (220 tenders)
|
||||
- **Fixed:** Removed tracking query parameters from URLs
|
||||
- **Before:** `https://www.find-tender.service.gov.uk/Notice/013656-2026?origin=SearchResults&p=1`
|
||||
- **After:** `https://www.find-tender.service.gov.uk/Notice/013656-2026`
|
||||
- **Files:** `scrapers/find-tender.js`, database updated
|
||||
|
||||
### 2. TED EU (4 demo records)
|
||||
- **Fixed:** Disabled scraper using hardcoded demo data
|
||||
- **Status:** Requires browser automation (Playwright/Puppeteer)
|
||||
- **Action:** Deleted 4 demo records, documented for future
|
||||
- **Files:** `scrapers/ted-eu.js` rewritten with documentation
|
||||
|
||||
### 3. Other Sources (Already Working)
|
||||
These were already correct - no changes needed:
|
||||
- contracts_finder (364 tenders)
|
||||
- etendersni (11 tenders)
|
||||
- pcs_scotland (10 tenders)
|
||||
- sell2wales (10 tenders)
|
||||
|
||||
## Database State After Fix
|
||||
|
||||
Total tenders: 615
|
||||
All URLs validated: ✅
|
||||
No broken URLs: ✅
|
||||
No tracking parameters (except legitimate notice IDs): ✅
|
||||
|
||||
## Files Modified
|
||||
|
||||
1. `scrapers/find-tender.js` - Strip query params
|
||||
2. `scrapers/ted-eu.js` - Disabled with documentation
|
||||
3. Database - 220 find_tender records cleaned, 4 TED records deleted
|
||||
|
||||
## Backups Created
|
||||
|
||||
- `scrapers/find-tender.js.bak`
|
||||
|
||||
## Testing Completed
|
||||
|
||||
✅ Verified all source URL patterns
|
||||
✅ Checked for invalid/broken URLs
|
||||
✅ Confirmed query parameters removed from find_tender
|
||||
✅ Validated notice IDs in other sources
|
||||
|
||||
## Result
|
||||
|
||||
Users clicking "Apply Now" will now land directly on tender detail pages for all 615 active tenders.
|
||||
Reference in New Issue
Block a user