feat(analytics): capture valid and invalid visit outcomes #13

Merged

eleith merged 1 commits from feat/visit-outcomes into main 2026-11-05 23:09:06 +00:00

Conversation 0 Commits 1 Files Changed 17

+231 -71

eleith-gitea-agent Bot commented 2026-11-05 23:05:29 +00:00

Copy link

Copy source

Scope

Complete request-time visit coverage for the agreed analytics cards. This is capture and storage only: no statistics queries, rollups, dashboard, URL-count implementation, API, dependency, or new workflow/job.

Built on the landed restart-only simplification in #12. Its ordinary inserts, startup-only collection switch, bounded drop warnings, and removed control table remain intact.

Outcome contract

Store one required outcome field:

• valid: an active configured link produces an allowed redirect.
• invalid: no matching active link, including disabled/not-yet-active/expired paths, a fallback redirect or fallback 404, or a blocked self-loop.

The value is decided at request time before ingestion. Later YAML edits do not reclassify earlier visits. HTTP status alone does not determine validity: an unmatched path that falls back with 302 is still invalid.

path = '/' identifies base visits independently. A configured active root is both base and valid; an unmatched/inactive root is both base and invalid. There is no redundant base flag or exclusive third outcome.

Only successfully parsed GETs on known redirect domains create these events. HEAD, unsupported methods, malformed targets, admin/unknown hosts, invalid trusted client addresses, and disabled collection remain untracked. Zero/positive collection changes still require stopping/draining both services before editing the setting and restarting; positive retention-period changes remain live.

Implementation

• Extend the shared raw/classified/stored visit type with VisitOutcome = 'valid' | 'invalid'.
• Add 0005_visit_outcome after landed 0004_remove_visit_collection. The new column is TEXT NOT NULL DEFAULT 'valid': all previously captured application events were successful active-link GETs, so this preserves their original meaning without consulting current YAML, scanning/updating history, or fabricating old invalid/base visits.
• Keep the ordinary Kysely insert unchanged: .values(visit) already stores the new field. The existing native worker still classifies once and performs one insert; there is no per-visit workflow or HTTP database wait.
• Make routing decisions explicit through small active-link, fallback, response and outcome helpers. Preserve status codes, headers, exact single-decode lookup, query forwarding, validity boundaries and self-loop behavior.
• Rename the capture helper to captureVisit because known-domain 404 visits are now included. Continue excluding query strings, cookies, referrers and destination URLs from the recorded payload.
• Count the outcome's 5/7 bytes within the existing 32 KiB budget. Keep dropVisitIfOverLimit, the 1,000-outstanding cap, whole-event drops, distinct bounded warnings, and FIFO/drain behavior.
• Worker test/smoke changes only supply the required event field; service business behavior is unchanged. No landed migration or product README was rewritten.

Tests and verification

• Final readability, correctness/security, test-quality, and When/Directory reference-parity reviews are clean. No source corrections were needed.
• Full pnpm verify passed: 459 unit tests, 7 web integration tests, and 5 service-process cases, plus schema/build/type/format/ESLint checks. Svelte reports zero errors/warnings.
• Extended existing path/validity/method tests with outcome assertions; focused root/config-change cases; dispatch privacy and negative-capture boundaries; real SQLite outcome roundtrip and legacy default labeling. No standalone migration test/concurrency harness or broad new matrix.
• The existing compiled-writer HTTP story now checks invalid/root persistence and HEAD exclusion while retaining lock isolation, shutdown drain and classification assertions; no duplicate E2E story was added.
• Fresh offline pruned web and worker artifacts tested outside the repository, with dependencies resolving inside each artifact and no source/loader fallback. Actual compiled web + Piscina + workflow service verified eight HTTP outcome events, both root overlaps, invalid 302/404 fallbacks, exclusions, trusted IP/raw UA/Firefox/Linux labels, legacy default valid, and unchanged earlier outcomes after YAML reload. Both processes exit naturally with code 0 and drained, secret-safe output. Control table stays absent.
• Scoped web/worker/config/DB lint, Changesets status and Git whitespace checks pass. Committed blobs match all 17 reviewed/passing source hashes.

Remote CI must pass on this head; the checks above are local. Agent-run Docker/Compose validation was not performed. No owner configuration/data access, merge, release, tag or deployment.

Commit: 29c7864 — feat(analytics): record request-time visit outcomes.

## Scope Complete request-time visit coverage for the agreed analytics cards. This is **capture and storage only**: no statistics queries, rollups, dashboard, URL-count implementation, API, dependency, or new workflow/job. Built on the landed restart-only simplification in #12. Its ordinary inserts, startup-only collection switch, bounded drop warnings, and removed control table remain intact. ## Outcome contract Store one required `outcome` field: - **`valid`**: an active configured link produces an allowed redirect. - **`invalid`**: no matching active link, including disabled/not-yet-active/expired paths, a fallback redirect or fallback 404, or a blocked self-loop. The value is decided **at request time before ingestion**. Later YAML edits do not reclassify earlier visits. HTTP status alone does not determine validity: an unmatched path that falls back with 302 is still invalid. `path = '/'` identifies base visits independently. A configured active root is both base and valid; an unmatched/inactive root is both base and invalid. There is no redundant base flag or exclusive third outcome. Only successfully parsed **GETs on known redirect domains** create these events. HEAD, unsupported methods, malformed targets, admin/unknown hosts, invalid trusted client addresses, and disabled collection remain untracked. Zero/positive collection changes still require stopping/draining both services before editing the setting and restarting; positive retention-period changes remain live. ## Implementation - Extend the shared raw/classified/stored visit type with `VisitOutcome = 'valid' | 'invalid'`. - Add **`0005_visit_outcome`** after landed `0004_remove_visit_collection`. The new column is `TEXT NOT NULL DEFAULT 'valid'`: all previously captured application events were successful active-link GETs, so this preserves their original meaning without consulting current YAML, scanning/updating history, or fabricating old invalid/base visits. - Keep the ordinary Kysely insert unchanged: `.values(visit)` already stores the new field. The existing native worker still classifies once and performs one insert; there is no per-visit workflow or HTTP database wait. - Make routing decisions explicit through small active-link, fallback, response and outcome helpers. Preserve status codes, headers, exact single-decode lookup, query forwarding, validity boundaries and self-loop behavior. - Rename the capture helper to `captureVisit` because known-domain 404 visits are now included. Continue excluding query strings, cookies, referrers and destination URLs from the recorded payload. - Count the outcome's 5/7 bytes within the existing 32 KiB budget. Keep `dropVisitIfOverLimit`, the 1,000-outstanding cap, whole-event drops, distinct bounded warnings, and FIFO/drain behavior. - Worker test/smoke changes only supply the required event field; service business behavior is unchanged. No landed migration or product README was rewritten. ## Tests and verification - Final readability, correctness/security, test-quality, and When/Directory reference-parity reviews are clean. No source corrections were needed. - **Full `pnpm verify` passed: 459 unit tests, 7 web integration tests, and 5 service-process cases**, plus schema/build/type/format/ESLint checks. Svelte reports zero errors/warnings. - Extended existing path/validity/method tests with outcome assertions; focused root/config-change cases; dispatch privacy and negative-capture boundaries; real SQLite outcome roundtrip and legacy default labeling. No standalone migration test/concurrency harness or broad new matrix. - The existing compiled-writer HTTP story now checks invalid/root persistence and HEAD exclusion while retaining lock isolation, shutdown drain and classification assertions; no duplicate E2E story was added. - Fresh offline pruned **web and worker artifacts** tested outside the repository, with dependencies resolving inside each artifact and no source/loader fallback. Actual compiled web + Piscina + workflow service verified eight HTTP outcome events, both root overlaps, invalid 302/404 fallbacks, exclusions, trusted IP/raw UA/Firefox/Linux labels, legacy default `valid`, and unchanged earlier outcomes after YAML reload. Both processes exit naturally with code 0 and drained, secret-safe output. Control table stays absent. - Scoped web/worker/config/DB lint, Changesets status and Git whitespace checks pass. Committed blobs match all 17 reviewed/passing source hashes. **Remote CI must pass on this head; the checks above are local.** Agent-run Docker/Compose validation was not performed. No owner configuration/data access, merge, release, tag or deployment. Commit: `29c7864` — `feat(analytics): record request-time visit outcomes`.

eleith-gitea-agent Bot added 1 commit 2026-11-05 23:05:29 +00:00

feat(analytics): record request-time visit outcomes 29c78641c8

eleith merged commit 29c78641c8 into main 2026-11-05 23:09:06 +00:00

eleith deleted branch feat/visit-outcomes 2026-11-05 23:09:06 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Assignees

Clear assignees

eleith-gitea-agent Bot eleith (eleith)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: eleith/goto#13