Commit Graph

138 Commits

Author SHA1 Message Date
Peter Foster
9b9362ae82 Lower interaction score threshold to stop blocking real users
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 14:19:02 +00:00
Peter Foster
646eed921a Replace reCAPTCHA test keys with production v3 keys
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 14:08:56 +00:00
Peter Foster
34ddbe3fdd Fix SEO issues found via GSC audit
- Fix broken PHP syntax in real-time data extraction article
- Fix truncated meta description on homepage
- Fix truncated titles on competitive intelligence and price monitoring pages
- Update Selenium vs Playwright article year from 2024 to 2026
- Replace no-store cache headers on homepage with private revalidation
- 301 redirect thin duplicate real-time streaming article to canonical
- Add HSTS header to data-cleaning service page
- Replace SVG og:images with PNG social card across 24 blog articles
- Add Services breadcrumb level to London, Manchester, Birmingham pages
2026-03-10 04:37:15 +00:00
Peter Foster
4121a20e40 SEO: BreadcrumbList on all service pages, author bios, case study pages, internal links, address fix, llms.txt update 2026-03-08 11:13:11 +00:00
Peter Foster
62e69542b0 SEO schema cleanup + blog index update
Removed 42 deprecated/restricted schema blocks across 21 files:
- FAQPage removed from all commercial pages (restricted Aug 2023)
- HowTo removed from all pages (rich results removed Sep 2023)
- Compliance guide: author type fixed Organization->Person

Blog index:
- New article cards: ai-web-scraping-2026, web-scraping-lead-generation-uk
- Stats updated: 55+ articles -> 57+, 2025 Content -> 2026 Content
- Featured article date updated to March 2026
- Blog schema updated with new BlogPosting entries
2026-03-08 10:48:11 +00:00
Peter Foster
790ffef935 SEO content expansion: compliance guide body, 2 new blog articles, schema
- web-scraping-compliance-uk-guide: filled 7 missing body sections (ToS, IP,
  CMA, best practices, risk matrix, documentation, industry-specific)
  now ~54KB of substantive legal compliance content
- New: blog/articles/web-scraping-lead-generation-uk.php (March 2026)
- New: blog/articles/ai-web-scraping-2026.php (March 2026)
- predictive-analytics-customer-churn: description updated for new title
- index.php: web-scraping-companies added to footer nav
- BreadcrumbList JSON-LD added to data-scraping and web-scraping-companies pages
- sitemap-blog.xml: new articles added
2026-03-08 10:40:23 +00:00
Peter Foster
31dd3e8d70 SEO: freshen articles, internal links, comparison page
- predictive-analytics-customer-churn: title broadened for CTR
- web-scraping-compliance-uk-guide: modified date 2026, 2025->2026 in body/keywords
- python-data-pipeline-tools: last_modified updated to 2026-03-08
- index.php: data-scraping linked in services card and footer nav
- services/web-scraping.php: cross-link to data-scraping page
- New: services/web-scraping-companies.php comparison page (508 GSC impressions, pos 22)
- sitemaps: web-scraping-companies added
2026-03-08 10:28:12 +00:00
Peter Foster
1b8b97e293 Add data-scraping service page; fix title tags for CTR
- New /services/data-scraping page targeting data scraping services (1,260 GSC impressions, pos 29)
- web-scraping.php title: improved to target web scraping services provider (pos 7.3)
- web-scraping-compliance-uk-guide.php title: updated 2024 -> 2026 for freshness signal
- sitemap.xml and sitemap-services.xml: added /services/data-scraping URL
2026-03-08 10:12:37 +00:00
Peter Foster
3b8512714a Update sitemap lastmod dates and resubmit to GSC
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 10:04:03 +00:00
Peter Foster
69bd6429e3 Rewrite location pages with unique city-specific content 2026-03-08 09:55:36 +00:00
Peter Foster
ed5336bf1f SEO fixes: author bylines, 2025 dates, phone removal, case studies, Companies House
- Assign named authors to all 14 blog articles that defaulted to Editorial Team
- Replace team-based author labels (DevOps Team, Legal Team etc) with named authors
- Update 2025 -> 2026 in ecommerce trends, buyers guide, and python pipeline titles
- Remove phone number (01692 Norfolk) from all pages and schema
- Anonymise unverifiable case study clients (TechElectronics UK, Heritage Bank UK)
- Add clickable Companies House link (08576932) to footer

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 09:42:53 +00:00
Peter Foster
7dc08d7ecb Fix founding year to 2013 across all pages and schema
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 09:37:35 +00:00
Peter Foster
30a71d832f Update company registration number to 08576932
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 09:37:06 +00:00
Peter Foster
93f353626c Remove unverified ICO registration and VAT number references
- Remove ICO Registration placeholder from footer
- Remove 'ICO registered' claims from about.php meta description
- Remove 'ICO registered' from web-scraping.php meta description and body copy
- Remove VAT No placeholder from footer

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 09:36:11 +00:00
Peter Foster
c07d126044 Fix SEO inconsistencies: founding year, accuracy rates, sitemap, alt text
- Fix founding year from 2018 to 2015 across about.php, organization-schema.php, and index.php schema
- Standardise data accuracy rate to 99.8% across about.php and services/web-scraping.php (was 99.7%/99.9%)
- Remove google-oauth-callback from sitemap.xml
- Fix UK Gambling Commission logo alt text (remove misleading 'regulatory compliance' label)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 09:24:51 +00:00
Peter Foster
bf04196d9e SEO: automated improvements (2026-03-07) — 3 modified, 2 created 2026-03-07 16:57:34 +00:00
Peter Foster
624a3aa282 SEO audit fixes: schema, content, technical and AI readiness
- Remove restricted FAQPage schema and duplicate Organization schema
- Remove incomplete LocalBusiness schema
- Fix Organization: founder Peter Foster, foundingDate 2015
- Rewrite H1 to include target keywords
- Fix BreadcrumbList with correct page URLs
- Remove SearchAction pointing to 404 endpoint
- Simplify verbose content across 13 key paragraphs
- Promote service card headings from H3 to H2 with keyword-rich names
- Add width/height attributes to all images (CLS fix)
- Create 1200x630 OG social card image
- Update og:image, og:image:width, og:image:height meta tags
- Fix broken LinkedIn (www prefix) and Twitter (x.com) social links
- Add Free Tools footer section linking to individual tools
- Remove google-oauth-callback and /index URLs from sitemap.xml
- Add AI crawler rules to robots.txt (GPTBot, ClaudeBot, Perplexity, etc.)
- Add Disallow for oauth callback endpoints in robots.txt
- Create llms.txt for AI search engine readiness
2026-03-06 06:42:24 +00:00
Peter Foster
66a02fcafb SEO: automated improvements (2026-03-05) — 4 modified, 3 created 2026-03-05 02:49:16 +00:00
Peter Foster
b1095b5b1c SEO: automated improvements (2026-03-05) — 2 modified, 2 created 2026-03-05 02:04:32 +00:00
Peter Foster
4df79ea181 SEO: automated improvements (2026-03-02) — 2 modified, 4 created 2026-03-02 13:42:19 +00:00
Peter Foster
97519207ab Fix broken data-engineering link 2026-03-02 13:33:58 +00:00
Peter Foster
26a5816268 SEO: automated improvements (2026-03-02) — 3 modified, 2 created 2026-03-02 13:33:42 +00:00
Peter Foster
63b9a134b0 Fix broken data-engineering service link 2026-03-02 13:26:04 +00:00
Peter Foster
9003957175 SEO: automated improvements (2026-03-02) — 2 modified, 3 created 2026-03-02 13:25:46 +00:00
Peter Foster
af53343773 Fix broken data-analytics-consulting link 2026-03-02 13:09:59 +00:00
Peter Foster
18a7b0fd7a SEO: automated improvements (2026-03-02) — 4 modified, 4 created 2026-03-02 13:09:22 +00:00
Peter Foster
7e481d5a17 Fix remaining broken location links 2026-03-02 13:01:34 +00:00
Peter Foster
4fcb16bd89 Fix broken internal links across site 2026-03-02 11:44:06 +00:00
Peter Foster
b257ceb3a4 SEO: automated improvements (2026-03-02) — 3 modified, 2 created 2026-03-02 11:38:26 +00:00
Peter Foster
ad69508ff8 SEO: automated improvements (2026-03-02) — 1 modified, 1 created 2026-03-02 11:30:42 +00:00
Peter Foster
0408731358 Block scanner IPs and non-existent PHP file requests 2026-03-02 11:15:06 +00:00
Peter Foster
89a791e187 Fix internal redirect loop on trailing-slash URLs 2026-03-02 11:08:53 +00:00
Peter Foster
999689071a Fix undefined $author variable and add new service pages to htaccess whitelist 2026-03-02 10:55:58 +00:00
Peter Foster
a22439091d SEO: commit cron-generated pages, fix duplicate footer in CI article 2026-03-02 10:42:15 +00:00
Peter Foster
c079aa5d5f fix: replace nav/footer placeholder comments with proper PHP includes 2026-03-02 10:35:11 +00:00
Peter Foster
e3a1610ade fix: remove corrupted HTML template injected into head by SEO script 2026-03-02 09:53:35 +00:00
Peter Foster
8fe2b9d210 fix: revert corrupted blog articles, clean sitemap of internal pages, fix META_TAGS_TEMPLATE permissions 2026-03-02 09:44:22 +00:00
Peter Foster
35f34a91d6 SEO: automated improvements (2026-03-02) — 4 modified, 1 created 2026-03-02 09:05:42 +00:00
Peter Foster
93178cd84e fix: restore correct article publication dates, update modified dates to today 2026-03-01 16:08:14 +00:00
Peter Foster
ddd64e5e5d SEO: automated improvements (2026-03-01) — 4 modified, 1 created 2026-03-01 11:33:51 +00:00
Peter Foster
96548b683b SEO: weekly automated meta improvements (2026-02-28) 2026-02-28 21:58:47 +00:00
Peter Foster
68955e9bc4 SEO: weekly automated meta improvements (2026-02-28) 2026-02-28 21:49:31 +00:00
Peter Foster
03c3417be6 SEO: weekly automated meta improvements (2026-02-28) 2026-02-28 21:48:48 +00:00
Peter Foster
8fdd86418a blog: add 4 new articles and wire into index + sitemap 2026-02-28 21:26:30 +00:00
Peter Foster
100ccb6622 SEO: update blog article meta tags from GSC analysis 2026-02-27 10:41:52 +00:00
Peter Foster
a57150d531 SEO: update meta tags on service, location, and blog pages from GSC analysis 2026-02-27 09:27:03 +00:00
Peter Foster
40203587c9 SEO: update meta titles/descriptions based on GSC data; fix quote page layout 2026-02-27 09:21:06 +00:00
Peter Foster
d1e5371f39 SEO Optimization: Implement Gemini recommendations
1. Updated homepage meta tags:
   - Title: Web Scraping Services UK | 99.8% Accuracy & GDPR Compliant
   - Description: Enterprise-grade web and data scraping services...

2. Created dedicated service pages:
   - /web-scraping-services/ - targets web scraping services (193 impressions)
   - /data-scraping-services/ - targets data scraping services (118 impressions)

3. Pages include:
   - SEO-optimized titles and descriptions
   - Industry-specific content
   - Stats highlighting #1 ranking and 99.8% accuracy
   - GDPR compliance emphasis
   - Links to free tools

Based on Google Search Console data showing 90,815 impressions/month but only 0.1% CTR.
Goal: Increase CTR to 1%+ and capitalize on #1.2 ranking for web scraping services in uk.
2026-02-27 09:21:06 +00:00
Peter Foster
239b01e311 CSS: nuclear cursor fix using cursor:inherit on all elements
Adds `*, *::before, *::after { cursor: inherit }` after the html/body
cursor:default rules. This forces explicit inheritance at every element
level, which is the only reliable cross-browser fix (Chrome, Firefox,
Safari all differ in how cursor auto-resolution works on text content).
Restores cursor:pointer for links/buttons and cursor:text for inputs.
2026-02-22 12:00:38 +00:00
Peter Foster
0136ff95e2 CSS: fix flashing cursor by adding cursor:default to html element
Browser UA stylesheet sets cursor:auto on <html>, causing I-beam cursor to
flash on text content in Firefox/Safari even with body{cursor:default}.
Adding cursor:default to the html{} block in both CSS files ensures the
root element has explicit arrow cursor everywhere.
2026-02-22 11:53:32 +00:00