The highest-impact fix: run_research_job() only ever read entry['id'] from
confirmed competitors, silently discarding the Firecrawl-matched or
manually-swapped URL — every scrape fell back to the competitor's bare
root domain regardless of what was confirmed. This explains most of the
"unpredictable/wrong data" behavior seen against the old app.py. Threaded
confirmed_url through jobs.py -> analysis_service.py so the actually
confirmed page is what gets scraped.
Other fixes bundled in this pass:
- Proxy layer: Webshare rotating-gateway country code lowercased (was
silently failing auth), Bright Data port corrected to 33335 (current
cert), dedicated-IP candidate rotation added before falling back to the
shared gateway/Bright Data escalation.
- Trip-intent matching: score_and_rank_urls() now hard-gates on
destination keyword and excludes bare root-domain candidates, instead
of letting duration/category signals alone push a wrong-country or
homepage URL over the confidence threshold.
- force_refresh cache-bypass toggle added (opt-in, collapsed "Advanced
options"), plus a "Skip this competitor" affordance for unmatched URLs.
- De-hardcoded "G Adventures" (the hackathon reference client) out of the
extraction prompt — now uses the real tenant's company name.
- RunHistory/Apps-Script "latest results" queries filtered to
triggered_by = "manual" so per-competitor content-hash bookkeeping rows
no longer drown out real batch runs.
- Dashboard fixes: stuck-state bug on revisiting the Analyse page,
AccountPage/BattlecardsPage restacked to single-column layout, Alerts
moved to a header bell dropdown, invisible Export/Run Analysis icons
(fill -> stroke), tour popup width mismatch, horizontal-scroll bug on
cards (missing min-width: 0 in flex layout).
- New PocketBase migrations for scrape_runs (content_hash fields,
tab_type, created_by_email) and products (departures, start_days).
335 backend tests passing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Rearchitects the Sheet integration per updated CLAUDE.md §15: a
stateless Flask backend has no way to reach into whichever Sheet a PM
has open, so the Sheet pulls its latest results on demand rather than
Flask ever pushing to it. This is also the simpler path for
non-technical users — no Sheet ID to find/paste, no service-account
sharing, just one button in the sidebar.
Apps Script (appscript/, new):
- appsscript.json — Editor Add-on manifest, spreadsheets.currentonly
scope (touches only whichever Sheet is open, nothing else in Drive)
- Code.gs — onOpen/onInstall/onHomepage, opens the sidebar
- Validation.gs — validateLicence() (6hr cache, POST /validate-email),
plus markOnboardingStepOnce_() shared helper
- Sync.gs — pullLatestResults()/getWorkbookTabs()/detectColumns(),
with a real header-alias map for every STANDARD_FIELDS column
(majorityPrice matches "Rack Rate"/"Standard Price"/etc, not just
one exact label) so silent write failures don't happen from a PM
naming a column slightly differently than expected
- Sidebar.html — tab selector + Pull latest results + last-pull status
Backend:
- GET /research/history/latest (email-resolved) replaces the removed
POST /sheet/push and GET /sheet/tabs
- GET /account/template now reads TEMPLATE_SHEET_URL instead of a
hardcoded placeholder
- PATCH /account/onboarding now accepts either a JWT session (dashboard)
or an email in the body (Apps Script has no JWT, only the PM's Google
email) — this was a real gap: Sync.gs's onboarding-flag calls would
have 401'd against the JWT-only version
- onboarding_service.DEFAULT_CHECKLIST corrected to the current schema
(template_imported/sidebar_installed/sync_from_sheet, not the stale
push_to_sheet/download_template)
- Fixed the same stale field names in CLAUDE.md §32's onboarding
runbook example payload, so it doesn't teach a future tenant-creation
script the wrong schema
Frontend:
- PushToSheet.vue removed, replaced by SyncToSheet.vue — a clipboard
copy button + instructions to use the sidebar, no tab selector (tab
selection lives in the sidebar, where the workbook is actually open)
- AccountPage.vue's template section rebuilt as three items: open
template, two-step import instructions + sidebar install link, and
a deliberately no-op verification input (never validated/fetched/
stored — the confirmation is purely the PM telling themselves it's
done, per explicit instruction not to create a dependency on their
workbook)
- useOnboardingTour.js step 5 updated to match
220 backend tests passing. Frontend production build verified clean.
Apps Script (.gs/.html) cannot be unit-tested in this environment —
no Google Apps Script runtime available locally; verified only by
careful cross-file consistency review (function names/signatures
matching across Code.gs/Validation.gs/Sync.gs/Sidebar.html).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Implements CLAUDE.md Build Order Phase 1 end to end:
- PocketBase migration covering all 15 MVP-scope collections
- Repository layer (11 repos) with real PocketBase-backed + in-memory
mock implementations for every collection
- Service layer (licence, trip-finder, battlecard, embedding, brief,
alert, analysis, onboarding, billing, auth) — Phase 2 scraping/
extraction dependencies are lazy-imported so this layer is fully
testable ahead of the app.py refactor
- Flask API (api.py) covering every MVP route from the spec, with
X-API-Key auth, Flask-Limiter rate limits, and structured JSON errors
- RQ job queue (jobs.py) with per-competitor failure isolation and
duration-based Gotify alerting
- 142 tests, 95% coverage, 100% on licence validation / seat
management / Stripe webhook dispatch per CLAUDE.md's testing floor
/sheet/push and /sheet/tabs intentionally return structured 501s until
the Apps Script Web App deployment exists (Phase 4).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
JasonFraser