The biggest fix: change-detection and the fast-path cache lookup were
scoped only by competitor_id, not by which specific trip page was
scraped. Since trip-intent matching lets a PM scrape a different URL per
run, get_last_hash_for_url() compared each new scrape against whatever
page happened to be hashed most recently for that competitor — almost
always a different page — so change_detected never settled back to
False and the true no-scrape fast path was unreachable in practice.
Scoped scrape_runs/products lookups by (competitor_id, url) instead, and
threaded a cleaned confirmed_url consistently through needs_refresh(),
detect_change(), and analyse_from_cache(). Verified end-to-end with a
regression test reproducing the exact reported scenario.
New: a per-seat, tier-capped trip watchlist (1 trip/seat at Analyse).
The nightly cron now re-scrapes only the specific trips a PM chose to
track instead of every competitor's homepage (a page already flagged
low-signal by is_generic_homepage()) — bounded, predictable volume, and
every resulting alert is about a trip someone actually cares about.
Reuses the existing confirmed_url/per-url caching machinery with zero
changes needed to jobs.py's job loop. Also fixes a real gap:
auth_service.py resolved the caller's active seat only to validate the
JWT and then discarded it — every route before this only ever needed
tenant_id. get_seat_from_request() now returns it.
Also fixed the Firecrawl /map timeout (15s → 30s; a large real site's
sitemap fetch took ~21s and was silently read as "no match found"), a
job left stuck at "Stopping…" forever after a worker process was killed
mid-run (status endpoint now self-heals via the underlying RQ job
state), and several results-table issues: hardcoded "G Adventures" in
the extraction prompt and a literal "G" in the comments column label
(replaced with the real tenant name), non-clickable trip links, a
cache-source badge that was silently clipped on most columns, a missing
view into the client's own scraped trip data, and bolding for
Strengths/Weaknesses/Differences/Pricing in the comments text.
370+ new/updated backend tests; full suite green.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
JasonFraser
a25909a459