feat(worker): introduce the OpenWorkflow service foundation #11

Merged

eleith merged 2 commits from feat/worker-service into main 2026-11-05 15:22:10 +00:00

Conversation 0 Commits 2 Files Changed 34

+1209 -9

eleith-gitea-agent Bot commented 2026-11-05 14:56:51 +00:00

Copy link

Copy source

Purpose and ordering

Introduce the standalone worker-service foundation from main, independently of #10. Merge this foundation first; then rework #10 so retention runs here through OpenWorkflow schedules rather than in web's ingestion thread.

This PR does not implement retention, analytics workflows, or cron definitions. No dummy workflow is registered. Web's Piscina path remains responsible only for the existing fast visit ingestion/classification.

Service

• Add apps/worker using When's src/services/{startup,lifecycle,health,logger,engine-logger,config-reload} approach with a thin entrypoint and colocated tests.
• Initialize OpenWorkflow 0.11.0 with its native SQLite backend and empty workflow registry.
• Use Directory's tslog 5.2.0: pretty development output, JSON production output, stack capture off. Engine messages/arguments are sanitized; no Pino or pino-pretty dependency.
• Own the engine, backend, health listener and configuration monitor locally. Stop the engine before closing its backend; attempt all cleanup and report failures afterward. Preserve early signals, bounded shutdown and natural process exit.
• Load the selected YAML path with captured environment. Invalid edits retain last-good configuration; changing worker storage requests a controlled restart only after the monitor's stop handle is owned.
• Expose private GET /healthz on port 9000 by default. No host port, job submission API, metrics endpoint, or web startup dependency is introduced. Health is liveness/startup status, not analytics readiness.

Storage and development

database:
  path: ./data/goto.sqlite
  worker_path: ./data/worker.sqlite

Both paths resolve against the configuration deployment root. The service consumes only worker_path; it has no @goto/db dependency and does not open/migrate visit storage. Worker storage contains operational OpenWorkflow state, not copies of visit events.

Reuse When's development conditions/source exports and small Node resolver hooks with .js source specifiers. Production remains ordinary compiled JS without loaders. Docker definitions use exact Node 26.5.0/pnpm 11.9.0, non-root execution and hardened production mounts. Both CI/CD systems gate publishing on checks, prohibit PR publishing, and provide main/latest plus matching stable-tag aliases. The worker joins the existing fixed app version group.

No shared jobs package or context singleton is added without another real consumer. No email/calendar/business workflows, application database access, visit batching, or retention logic is copied into this foundation.

Verification

• Final readability, correctness, test-quality, and When/Directory parity reviews are clean. When reference: f2be42a; Directory logging reference: cd8f8e; exact OpenWorkflow 0.11.0 source inspected.
• Full pnpm verify: schema drift, builds/types, formatting/ESLint, 429 unit tests and 7 web integration tests passed, plus service-process smoke cases. Svelte reports zero errors/warnings; existing redirect/auth/HMR stories pass.
• Service smoke cases exercise compiled startup, actual development-source resolution, worker-storage restart, and occupied-port failure after engine startup. Forced termination fails the test; output assertions wait for stream closure and HTTP/process waits are bounded.
• Offline pruned production artifact tested outside the repository: runtime dependencies resolve inside the artifact, tslog is present and Pino absent, workflow queue remains empty, no visits database is created, and SIGTERM exits naturally with code 0.
• Worker/config scoped lint, Changesets status, and Git whitespace checks passed.

Agent-run Docker/Compose verification was not performed; container checks remain owner-only. Only throwaway state was created by checks. The owner's application data and existing PR #10 were not modified. Remote CI results remain separate from these local checks.

## Purpose and ordering Introduce the standalone worker-service foundation **from main, independently of #10**. Merge this foundation first; then rework #10 so retention runs here through OpenWorkflow schedules rather than in web's ingestion thread. This PR does not implement retention, analytics workflows, or cron definitions. No dummy workflow is registered. Web's Piscina path remains responsible only for the existing fast visit ingestion/classification. ## Service - Add `apps/worker` using When's `src/services/{startup,lifecycle,health,logger,engine-logger,config-reload}` approach with a thin entrypoint and colocated tests. - Initialize OpenWorkflow **0.11.0** with its native SQLite backend and empty workflow registry. - Use **Directory's tslog 5.2.0**: pretty development output, JSON production output, stack capture off. Engine messages/arguments are sanitized; no Pino or pino-pretty dependency. - Own the engine, backend, health listener and configuration monitor locally. Stop the engine before closing its backend; attempt all cleanup and report failures afterward. Preserve early signals, bounded shutdown and natural process exit. - Load the selected YAML path with captured environment. Invalid edits retain last-good configuration; changing worker storage requests a controlled restart only after the monitor's stop handle is owned. - Expose private `GET /healthz` on port 9000 by default. No host port, job submission API, metrics endpoint, or web startup dependency is introduced. Health is liveness/startup status, not analytics readiness. ## Storage and development ```yaml database: path: ./data/goto.sqlite worker_path: ./data/worker.sqlite ``` Both paths resolve against the configuration deployment root. The service consumes only `worker_path`; it has no `@goto/db` dependency and does not open/migrate visit storage. Worker storage contains operational OpenWorkflow state, not copies of visit events. Reuse When's development conditions/source exports and small Node resolver hooks with `.js` source specifiers. Production remains ordinary compiled JS without loaders. Docker definitions use exact Node 26.5.0/pnpm 11.9.0, non-root execution and hardened production mounts. Both CI/CD systems gate publishing on checks, prohibit PR publishing, and provide main/latest plus matching stable-tag aliases. The worker joins the existing fixed app version group. No shared jobs package or context singleton is added without another real consumer. No email/calendar/business workflows, application database access, visit batching, or retention logic is copied into this foundation. ## Verification - Final readability, correctness, test-quality, and When/Directory parity reviews are clean. When reference: `f2be42a`; Directory logging reference: `cd8f8e`; exact OpenWorkflow 0.11.0 source inspected. - Full `pnpm verify`: schema drift, builds/types, formatting/ESLint, **429 unit tests and 7 web integration tests passed**, plus service-process smoke cases. Svelte reports zero errors/warnings; existing redirect/auth/HMR stories pass. - Service smoke cases exercise compiled startup, actual development-source resolution, worker-storage restart, and occupied-port failure after engine startup. Forced termination fails the test; output assertions wait for stream closure and HTTP/process waits are bounded. - Offline pruned production artifact tested outside the repository: runtime dependencies resolve inside the artifact, tslog is present and Pino absent, workflow queue remains empty, no visits database is created, and SIGTERM exits naturally with code 0. - Worker/config scoped lint, Changesets status, and Git whitespace checks passed. Agent-run Docker/Compose verification was not performed; container checks remain owner-only. Only throwaway state was created by checks. The owner's application data and existing PR #10 were not modified. Remote CI results remain separate from these local checks.

eleith-gitea-agent Bot added 2 commits 2026-11-05 14:56:51 +00:00

feat(worker): add the OpenWorkflow service foundation 80f36e3733

build(worker): add containers and matching CI/CD 81fc971ee1

eleith merged commit 81fc971ee1 into main 2026-11-05 15:22:10 +00:00

eleith deleted branch feat/worker-service 2026-11-05 15:22:10 +00:00

eleith-gitea-agent Bot referenced this pull request 2026-11-05 19:27:49 +00:00

feat(analytics): enforce finite retention in the worker service #10

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Assignees

Clear assignees

eleith-gitea-agent Bot eleith (eleith)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: eleith/goto#11