eleith/hledger-web-reports
1
1 Fork 0
Code Issues Pull Requests Packages Releases Activity

Move report tools to Settings and record journal validation #38

Merged
eleith merged 1 commits from feat/settings-validation into main 2026-11-04 23:56:14 +00:00
Conversation 0 Commits 1 Files Changed 45
+1426 -302
45 changed files with 1426 additions and 302 deletions
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.changeset/record-validation-and-settings.md
+6
View File
@@ -0,0 +1,6 @@
---
'@hledger-reports/web': patch
'@hledger-reports/worker': patch
---
Move report preparation and the Live queries entry into Settings. Record each completed strict journal check before preparation and show its last saved outcome and time independently of financial snapshots. Incomplete checks retain earlier evidence, and browsing never checks journals or queues work. Existing prepared-report format and financial views are unchanged.
README.md
+8 -5
View File
@@ -4,7 +4,7 @@ A local, read-only hledger reporter. hledger owns accounting; OpenWorkflow + SQL
## Overview, transactions and live queries
**Overview (`/`) is the prepared financial summary:** monthly income-vs-expenses bars, depth-2 expense-category bars and a native market-valued net-worth trend, with exact supporting tables, for the latest six calendar months of the journal: the month of its latest recorded transaction and the previous five. Each page reads the latest complete current-format saved report. Without one, it says “No prepared reports available.” and links to manual preparation. **Reload overview** only reads saved data; it never starts calculations or checks journal changes. Pending or failed preparation leaves previous data available.
**Overview (`/`) is the prepared financial summary:** monthly income-vs-expenses bars, depth-2 expense-category bars and a native market-valued net-worth trend, with exact supporting tables, for the latest six calendar months of the journal: the month of its latest recorded transaction and the previous five. Each page reads the latest complete current-format saved report. Without one, it says “No prepared reports available.” and links to manual preparation. **Reload overview** only reads saved data; it never starts calculations or checks journal changes. Pending preparation keeps saved data available; a failed job does not necessarily mean no new set was committed.
The window is journal-anchored, not clock-anchored or a freshness guarantee. Exact native quantities and commodities are preserved. **Income/expense amounts are not available cash, cash flow or net worth**; the separate net-worth card uses native dated asset and liability values. See the [library contract](packages/hledger/README.md#overview-data) for report semantics and empty-journal behavior.
@@ -14,9 +14,11 @@ The hledger library reads the journal and organizes neutral, table/chart-ready `
**Cash and Investment registers list their explicitly configured accounts.** Select one for prepared activity with ordinary pagination: 50 rows for Cash, or 50 whole transactions for Investments. Dates, exact changes and running balances come from hledger and do not reset on page boundaries. The web reads the native account directory in `overview.json`, opens the selected account's complete JSON file, and projects only the requested page. Friendly labels do not change native identity or report scope. Browsing never executes native work; Overview and all native account activity still publish together as one complete set.
**Prepare reports (`/prepare`) explicitly requests work.** Submit the ordinary form to prepare Overview and all account registers together. Reload that page to check job status, then navigate to the saved reports. Worker startup and journal edits do not prepare anything automatically. After editing your journal, prepare again when you want new results. There is no browser polling or Overview/Transactions API.
**Settings (`/settings`) owns inline report preparation and status.** Submit its ordinary **Prepare reports** form to prepare Overview and all account registers together. Reload Settings to check this deployment's queue status, then navigate to saved reports. `/prepare` has been removed, with no redirect or POST alias. Worker startup and journal edits do not prepare anything automatically. After editing your journal, prepare again when you want new results. There is no browser polling or Overview/Transactions API.
**`/queries` remains a separate live report:** each Run executes native balance at recorded cost against the journal current at execution. Exact results persist in SQLite as operational workflow output, but the page displays only its latest response; reload clears it. Neither page offers user-facing run history.
Overview and Settings show the **last recorded strict validation outcome and time**, separately from when financial reports were prepared. Reading either never checks the journal; unavailable validation does not hide saved finances. See [validation and preparation](docs/architecture.md#last-recorded-validation) for their independent lifecycles.
**`/queries`, linked from Settings, remains the existing separate live report:** each Run executes native balance at recorded cost against the journal current at execution. Exact results persist in SQLite as operational workflow output, but the page displays only its latest response; reload clears it. Neither page offers user-facing run history.
Sign-in is required. Local development stays loopback-bound; an existing trusted HTTPS proxy may front the compiled app under the documented Host, origin and client-address contract. Do not expose its upstream directly. See [shared access and authentication](docs/authentication.md). Curated report catalogs and search remain deferred.
@@ -34,7 +36,7 @@ docker compose -f docker-compose.dev.yml up --build --wait
Runtime data is bind-mounted from **`apps/web/data/`**, so it is directly inspectable and ignored by Git/build tools. To stop without removing the data folder, run `docker compose -f docker-compose.dev.yml down` from the same directory.
Open **http://127.0.0.1:5173/**, sign in, then use **Prepare reports** to create the first saved set. Web readiness does not wait for financial calculations, and saved data remains readable if the worker is unavailable. Automated tests use the separate fictional `lots.journal` fixture, never your live journal. See [container development and CI](docs/development.md) for setup, configuration migration, compiled proxy setup and browser tests.
Open **http://127.0.0.1:5173/**, sign in, then use **Prepare reports** in **Settings** to create the first saved set. Web readiness does not wait for financial calculations, and saved data remains readable if the worker is unavailable. Automated tests use the separate fictional `lots.journal` fixture, never your live journal. See [container development and CI](docs/development.md) for setup, configuration migration, compiled proxy setup and browser tests.
The worker runs requested jobs; it does not watch or hash journals, detect staleness, or schedule report preparation. Saved timestamps tell you when reports were prepared, not whether they match current inputs. Explicit query Runs still read current input. Closing the page aborts its HTTP request, not durable work. Without `CONFIG_PATH`, protected requests fail closed; public health and schema endpoints remain available. No personal configuration is discovered.
@@ -76,10 +78,11 @@ Account identities and labels are literal strings, with exact matching and no en
Configuration and `data/` remain siblings. Renaming the config in place does not move data or change the selected journal. Both processes must select the same file:
- `data/jobs/`: private SQLite workflow queue, query parameters/results and preparation history.
- `data/snapshots/.validation/`: immutable completed-check records, separate from financial sets. Records are retained without automatic cleanup or a history UI; private journal/request metadata is never sent to the browser.
- `data/snapshots/current/overview.json`: the selected journal identity, `OverviewData` and the account directory. Each native account has a stable eight-character lowercase-letter/digit ID, full name, access state and row count.
- `data/snapshots/current/transactions/<account-id>.json`: one complete account register per file, with its identity, exact native amounts and exported transaction/posting metadata. Source and assertion positions remain private. This is processed report output, not original journal text or a complete investment cost-basis export. Account names remain JSON data, never paths. Pagination is a server-side array slice, not persisted page files.
Files are prepared together in a hidden staging set and published as a complete directory; `current` is the inspectable entry point. Readers accept only complete format-4 publications for the selected journal. Previous table-only snapshots require explicit preparation again. Unsupported older files are ignored and left untouched; prepare reports manually to create a current set. Readers do not migrate, delete or automatically regenerate files. Former source-observation files are also left untouched and are not read or written.
Financial files are prepared together in a hidden staging set and published as a complete directory; `current` is the inspectable entry point. Readers accept only complete format-5 publications for the selected journal; this change does not bump the financial format or require re-preparing supported sets. Unsupported older files are ignored and left untouched; explicitly prepare reports in Settings to create a current set. Readers do not migrate, delete or automatically regenerate files. Former source-observation files are also left untouched and are not read or written. A failed preparation can still leave a new complete report available; reload saved reports to observe the result. See [publication and recovery](packages/snapshots/README.md#publication-and-reads) for commit behavior.
In Compose, these paths are visible under **`apps/web/data/`** in the checkout. The previous named volume is left untouched; [copy it safely before the first bind-mounted start](docs/development.md#inspecting-data-and-preserving-the-previous-volume) to retain its queue and publications. Protect and back up the entire data directory; do not log financial data or native diagnostics. [Prepared-data lifetime](packages/snapshots/README.md#prepared-reports) covers growth, last-good recovery and untouched unsupported files.
apps/web/README.md
+40 -26
View File
@@ -7,24 +7,37 @@ See the [architecture](../../docs/architecture.md) and
## Pages
| Page | Behavior |
| -------------------------------------------------- | ---------------------------------------------------------------------------------- |
| `/` | Read saved Overview: income/expenses, categories, net worth and journal coverage |
| `/transactions` | Read the saved directory of native declared/used accounts |
| `/transactions/account?account=<id>&page=<number>` | Read one complete account file and return a 50-row slice |
| `/prepare` | Show preparation status at page load; ordinary POST form explicitly requests a job |
| `/queries` | Explicit native recorded-cost balance Run; reload clears the displayed result |
| Page | Behavior |
| ----------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| `/` | Read saved six-month Overview and independent last recorded validation outcome/time |
| `/registers/cash`, `/registers/investments` | Read configured members from the saved account directory |
| `/registers/cash/<id>`, `/registers/investments/<id>` | Read saved account activity with ordinary pagination |
| `/settings` | Inline preparation/status and saved validation; ordinary POST requests work; links to Live queries |
| `/queries` | Existing explicit recorded-cost balance tool; reload clears the displayed result |
Overview and Transactions do not query job status, hash inputs, infer freshness or
submit work. Without saved data, they say it has not been prepared and link to
Prepare reports. Existing reports remain visible if the worker is stopped or a new
preparation fails. After editing the journal, explicitly prepare again.
Overview and registers do not query job status, hash inputs, infer freshness or
submit work. Without saved financial data, they say it has not been prepared and
link to Settings. Saved files remain readable if the worker is stopped. After
editing the journal, explicitly prepare again. The six-month Overview and register
presentation are unchanged; there are no new period controls or account sidebar.
The Prepare reports form requests Overview and all account registers together.
Concurrent requests join active preparation; a later request recalculates even if
inputs are unchanged. The form works without JavaScript. There is no polling,
fetch controller, automatic invalidation or Overview/Transactions API. Reload the
preparation page to check status, then navigate to saved reports.
Settings is the canonical preparation/status page; `/prepare` has been removed with
no redirect or POST alias. Its inline Prepare reports form requests Overview and
all account registers together. Concurrent requests join active preparation; a later
request recalculates even if inputs are unchanged. The ordinary empty form works
without JavaScript and redirects with 303 to `/settings`. Reload Settings to check
this deployment's queue status, then navigate to saved reports. There is no polling,
fetch controller, automatic invalidation or Overview/Transactions API. Settings links
to the existing `/queries` screen instead of a standalone Live queries utility-nav
entry; this adds no query history or new navigation feature.
Overview and Settings show the last recorded strict validation outcome/time separately
from the saved financial preparation time. Unavailable validation never hides readable
finances; only outcome/time is public, not journal/request identity or diagnostics.
A failed job does not guarantee reports are unchanged: reload saved reports to see
the available result. See [validation ownership](../../docs/architecture.md#last-recorded-validation)
and [publication/recovery](../../packages/snapshots/README.md#publication-and-reads)
for the lifecycle details.
## Account labels and register membership
@@ -96,18 +109,18 @@ data/snapshots/current/
Overview contains the selected journal identity and account directory. One whole
account file contains all native rows. The worker publishes a complete immutable
set; a request pins that set before reading. Private journal identities and set
paths never enter page data. Format 4 also retains full exported transaction/posting
metadata and private source/assertion positions. The existing Transactions table
still receives only its displayed fields: date, description, related accounts,
split flag and exact quantity/commodity pairs. Projection happens after the
50-row slice, mapping stored `accountDate` to public `date`; stored records are
never passed directly into PageData.
paths never enter page data. Financial format **5** retains full exported
transaction/posting metadata and private source/assertion positions. Register loaders
project only the selected page's public table/detail fields; stored records are never
passed directly into PageData.
Only format 4 is read. Previous table-only snapshots require explicit preparation
again. Unsupported saved files are left untouched; use Prepare reports to create
Financial format remains **5**, with no re-preparation requirement for supported sets.
Unsupported saved files are left untouched; use Prepare reports in Settings to create
a current complete set.
There is no older-format fallback, partial-card compatibility or automatic migration.
Reports for a different configured journal are withheld.
Reports for a different configured journal are withheld. Immutable validation records
live separately in `data/snapshots/.validation/`, retained without cleanup or a history
UI. Homepage browsing reads these files, never the queue or native journal.
See [prepared storage](../../packages/snapshots/README.md#prepared-reports) and the
[old-volume copy guide](../../docs/development.md#inspecting-data-and-preserving-the-previous-volume).
@@ -168,7 +181,7 @@ mkdir -m 700 -p data &&
docker compose -f docker-compose.dev.yml up --build --wait
```
Open **http://127.0.0.1:5173/** and use Prepare reports. From the repository root:
Open **http://127.0.0.1:5173/** and use Prepare reports in Settings. From the repository root:
```sh
pnpm build:packages
@@ -184,6 +197,7 @@ src/lib/
charts/ Chart canvas and configurations named by chart
overview/ Financial summary panels
registers/ Cash and investment presentation
preparation/ Shared last recorded validation presentation
queries/ Live-query result presentation
server/ Server-only request and saved-report helpers
styles/ Document reset/theme, layout mechanics and native controls
apps/web/e2e/manual-preparation.e2e.ts
+1 -1
View File
@@ -19,5 +19,5 @@ test('explicitly prepares reports and browses saved Overview without JavaScript'
await page.reload();
await expect(income.getByRole('table')).toContainText('1000.00');
expect(await readPreparationTimestamp(page)).toBe(preparedAt);
expect(posts).toEqual(['/prepare']);
expect(posts).toEqual(['/settings']);
});
apps/web/e2e/overview-fixture.ts
+8 -8
View File
@@ -3,21 +3,19 @@ import { expect, type Page } from '@playwright/test';
const preparationStamp = (page: Page) =>
page
.getByRole('region', { name: 'Income vs expenses', exact: true })
.locator('p')
.filter({ hasText: /^Prepared at / })
.locator('time');
.getByLabel('Financial reports prepared at', { exact: true });
/** Explicit user action, including for external fictional-fixture smoke tests. */
export async function prepareReports(page: Page): Promise<string> {
await page.goto('/');
const stamp = preparationStamp(page);
const previous = (await stamp.count()) ? await stamp.getAttribute('datetime') : null;
await page.goto('/prepare');
await expect(page.getByRole('heading', { name: 'Prepare reports', exact: true })).toBeVisible();
await page.goto('/settings');
await expect(page.getByRole('heading', { name: 'Settings', exact: true })).toBeVisible();
await Promise.all([
page.waitForResponse(
(response) =>
new URL(response.url()).pathname === '/prepare' &&
new URL(response.url()).pathname === '/settings' &&
response.request().method() === 'POST' &&
response.request().isNavigationRequest() &&
response.request().resourceType() === 'document'
@@ -30,7 +28,7 @@ export async function prepareReports(page: Page): Promise<string> {
.poll(
async () => {
await page.reload();
const time = status.locator('time');
const time = status.getByLabel('Financial reports prepared at', { exact: true });
const current = (await time.count()) ? await time.getAttribute('datetime') : null;
return (
current !== null &&
@@ -41,7 +39,9 @@ export async function prepareReports(page: Page): Promise<string> {
{ timeout: 25_000, intervals: [100, 250, 500] }
)
.toBe(true);
const preparedAt = (await status.locator('time').getAttribute('datetime'))!;
const preparedAt = (await status
.getByLabel('Financial reports prepared at', { exact: true })
.getAttribute('datetime'))!;
await page.goto('/');
return preparedAt;
}
apps/web/src/lib/overview/IncomeExpenses.svelte
+3 -1
View File
@@ -21,7 +21,9 @@
<p>Latest recorded transaction: <time datetime={overview.asOf}>{overview.asOf}</time>.</p>
{/if}
<p class="muted">
Prepared at <time datetime={overview.preparedAt}>{overview.preparedAt}</time>.
Prepared at <time aria-label="Financial reports prepared at" datetime={overview.preparedAt}
>{overview.preparedAt}</time
>.
</p>
{#if overview.asOf === null}
<p>No dated transactions.</p>
apps/web/src/lib/preparation/ValidationStatus.svelte
+26
View File
@@ -0,0 +1,26 @@
<script lang="ts">
import type { JournalValidationResult } from '@hledger-reports/snapshots';
let {
validation,
readFailed
}: { validation: JournalValidationResult | null; readFailed: boolean } = $props();
</script>
<section aria-label="Saved journal validation">
{#if readFailed}
<p>Saved journal validation couldn't be read.</p>
{:else if validation}
<p>
Last recorded strict validation: {validation.outcome === 'passed'
? 'Passed'
: 'Could not validate'}. Completed at
<time aria-label="Journal validation completed at" datetime={validation.completedAt}
>{validation.completedAt}</time
>.
</p>
<p>Recorded separately from saved reports.</p>
{:else}
<p>No recorded journal check available.</p>
{/if}
</section>
apps/web/src/lib/preparation/ValidationStatus.test.ts
+38
View File
@@ -0,0 +1,38 @@
import { render } from 'svelte/server';
import { expect, it } from 'vitest';
import ValidationStatus from './ValidationStatus.svelte';
function text(body: string) {
return body.replace(/<[^>]*>/g, ' ').replace(/\s+/g, ' ');
}
it('presents the last completed pass with its own time, without claiming journal freshness', () => {
const completedAt = '2026-05-01T12:01:00Z';
const { body } = render(ValidationStatus, {
props: { validation: { outcome: 'passed', completedAt }, readFailed: false }
});
expect(text(body)).toContain('Last recorded strict validation: Passed.');
expect(text(body)).toContain(`Completed at ${completedAt}`);
expect(body).toContain(`datetime="${completedAt}"`);
expect(text(body)).toContain('Recorded separately from saved reports.');
expect(text(body)).not.toContain('reports prepared');
});
it('describes a completed unsuccessful check without calling the accounting invalid', () => {
const { body } = render(ValidationStatus, {
props: {
validation: { outcome: 'could-not-validate', completedAt: '2026-05-02T12:01:00Z' },
readFailed: false
}
});
expect(text(body)).toContain('Last recorded strict validation: Could not validate.');
expect(text(body)).not.toMatch(/Passed|Invalid/);
});
it('distinguishes missing evidence from a read failure and never shows a pass for either', () => {
const missing = render(ValidationStatus, { props: { validation: null, readFailed: false } });
const failed = render(ValidationStatus, { props: { validation: null, readFailed: true } });
expect(text(missing.body)).toContain('No recorded journal check available.');
expect(text(failed.body)).toContain("Saved journal validation couldn't be read.");
expect(text(missing.body) + text(failed.body)).not.toMatch(/Passed|Completed at/);
});
apps/web/src/lib/server/auth/gate.test.ts
+1 -1
View File
@@ -65,7 +65,7 @@ it('requires a session for protected and unclassified application routes, includ
it('rejects anonymous endpoints and actions instead of resolving them', async () => {
for (const input of [
setup('/(protected)/(endpoints)/api/queries'),
setup('/(protected)/prepare', 'POST')
setup('/(protected)/settings', 'POST')
]) {
const response = await requireSignIn(input);
expect(response.status).toBe(401);
apps/web/src/lib/server/reports.test.ts
+113
View File
@@ -0,0 +1,113 @@
import { beforeEach, expect, it, vi } from 'vitest';
import {
readReportPublication,
readValidation,
type ReportPublication,
type ValidationRecord
} from '@hledger-reports/snapshots';
import { readSavedReports, readSavedValidation } from './reports.js';
vi.mock('@hledger-reports/snapshots', () => ({
readReportPublication: vi.fn(),
readValidation: vi.fn()
}));
vi.mock('@hledger-reports/jobs', () => {
throw new Error('Saved reads must not load jobs.');
});
vi.mock('@hledger-reports/hledger', () => {
throw new Error('Saved reads must not execute native reports.');
});
const directory = '/PRIVATE/snapshots';
const journal = '/PRIVATE/journal';
const record: ValidationRecord = {
format: 1,
journal,
request: { id: 'PRIVATE-request', createdAt: '2026-05-01T12:00:00Z' },
result: { outcome: 'passed', completedAt: '2026-05-01T12:01:00Z' }
};
const publication: ReportPublication = {
journal,
directory: '/PRIVATE/set',
accounts: [],
overview: {
asOf: null,
preparedAt: '2026-04-01T12:00:00Z',
journalCoverage: { transactionCount: 0, firstDate: null, lastDate: null },
incomeExpenses: { months: [] },
expenseCategories: { depth: 2, rows: [] },
netWorth: { currency: null, points: [] }
}
};
beforeEach(() => {
vi.resetAllMocks();
vi.mocked(readValidation).mockResolvedValue(record);
vi.mocked(readReportPublication).mockResolvedValue(publication);
});
it('projects only the saved outcome and completion time for the configured journal', async () => {
// Even extra private fields must not cross the server projection boundary.
vi.mocked(readValidation).mockResolvedValue({
...record,
result: { ...record.result, privateDiagnostic: 'PRIVATE error' }
} as ValidationRecord);
const saved = await readSavedValidation(directory, journal);
expect(saved).toEqual({ validation: record.result, readFailed: false });
expect(saved.validation).not.toBe(record.result);
expect(readValidation).toHaveBeenCalledExactlyOnceWith(directory);
expect(JSON.stringify(saved)).not.toMatch(/PRIVATE|request|format|journal|privateDiagnostic/);
expect(readReportPublication).not.toHaveBeenCalled();
});
it('preserves a recorded could-not-validate outcome rather than inventing a pass', async () => {
const result = { ...record.result, outcome: 'could-not-validate' as const };
vi.mocked(readValidation).mockResolvedValue({ ...record, result });
expect(await readSavedValidation(directory, journal)).toEqual({
validation: result,
readFailed: false
});
});
it('withholds missing or unrelated evidence, comparing private journal identity exactly', async () => {
vi.mocked(readValidation).mockResolvedValueOnce(null);
expect(await readSavedValidation(directory, journal)).toEqual({
validation: null,
readFailed: false
});
expect(await readSavedValidation(directory, journal.toUpperCase())).toEqual({
validation: null,
readFailed: false
});
});
it('keeps financial reports available when the independent validation read fails', async () => {
vi.mocked(readValidation).mockRejectedValue(new Error('PRIVATE corrupt evidence'));
const [reports, validation] = await Promise.all([
readSavedReports(directory, journal),
readSavedValidation(directory, journal)
]);
expect(reports).toEqual({ publication, readFailed: false });
expect(validation).toEqual({ validation: null, readFailed: true });
});
it('keeps validation available when saved financial reports fail, without leaking errors', async () => {
vi.mocked(readReportPublication).mockRejectedValue(new Error('PRIVATE corrupt reports'));
expect(await readSavedReports(directory, journal)).toEqual({
publication: null,
readFailed: true
});
expect(await readSavedValidation(directory, journal)).toEqual({
validation: record.result,
readFailed: false
});
});
it('withholds a financial publication from another journal without changing its private read API', async () => {
expect(await readSavedReports(directory, '/PRIVATE/other')).toEqual({
publication: null,
readFailed: false
});
expect(readReportPublication).toHaveBeenCalledExactlyOnceWith(directory);
expect(readValidation).not.toHaveBeenCalled();
});
apps/web/src/lib/server/reports.ts
+24 -1
View File
@@ -1,4 +1,27 @@
import { readReportPublication, type ReportPublication } from '@hledger-reports/snapshots';
import {
readReportPublication,
readValidation,
type JournalValidationResult,
type ReportPublication
} from '@hledger-reports/snapshots';
// Validation is independent of financial publication and queue status.
export async function readSavedValidation(
snapshotDirectory: string,
journal: string
): Promise<{ validation: JournalValidationResult | null; readFailed: boolean }> {
try {
const record = await readValidation(snapshotDirectory);
if (!record || record.journal !== journal) return { validation: null, readFailed: false };
const validation = {
outcome: record.result.outcome,
completedAt: record.result.completedAt
};
return { validation, readFailed: false };
} catch {
return { validation: null, readFailed: true };
}
}
// Private publication context must never be included in page data.
export async function readSavedReports(
apps/web/src/routes/(protected)/+layout.svelte
+2 -4
View File
@@ -37,11 +37,9 @@
</nav>
<nav class="utility" aria-label="Utilities">
<a
href={resolve('/(protected)/prepare')}
data-sveltekit-reload
aria-current={page.url.pathname === '/prepare' ? 'page' : undefined}>Prepare reports</a
href={resolve('/(protected)/settings')}
aria-current={page.url.pathname === '/settings' ? 'page' : undefined}>Settings</a
>
<a href={resolve('/(protected)/queries')}>Live queries</a>
</nav>
<div class="session">
<span>{data.session?.user.name}</span>
apps/web/src/routes/(protected)/+page.server.ts
+20 -7
View File
@@ -1,15 +1,28 @@
import { readSavedReports } from '#lib/server/reports.js';
import { readSavedReports, readSavedValidation } from '#lib/server/reports.js';
import type { PageServerLoad } from './$types';
export const load = (async (event) => {
event.setHeaders({ 'cache-control': 'no-store' });
const { application } = event.locals;
if (!application) {
return { configured: false, overview: null, readFailed: false };
return {
configured: false,
overview: null,
readFailed: false,
validation: null,
validationReadFailed: false
};
}
const { publication, readFailed } = await readSavedReports(
application.paths.snapshotDirectory,
application.config.journal
);
return { configured: true, overview: publication?.overview ?? null, readFailed };
const [{ publication, readFailed }, { validation, readFailed: validationReadFailed }] =
await Promise.all([
readSavedReports(application.paths.snapshotDirectory, application.config.journal),
readSavedValidation(application.paths.snapshotDirectory, application.config.journal)
]);
return {
configured: true,
overview: publication?.overview ?? null,
readFailed,
validation,
validationReadFailed
};
}) satisfies PageServerLoad;
apps/web/src/routes/(protected)/+page.svelte
+4
View File
@@ -1,4 +1,6 @@
<script lang="ts">
import { resolve } from '$app/paths';
import ValidationStatus from '#lib/preparation/ValidationStatus.svelte';
import IncomeExpenses from '#lib/overview/IncomeExpenses.svelte';
import ExpenseCategories from '#lib/overview/ExpenseCategories.svelte';
import JournalCoverage from '#lib/overview/JournalCoverage.svelte';
@@ -31,6 +33,8 @@
</p>
</section>
{:else}
<ValidationStatus validation={data.validation} readFailed={data.validationReadFailed} />
<p><a href={resolve('/(protected)/settings')}>Settings</a> — prepare saved reports.</p>
{#if data.readFailed}
<p role="alert">Saved reports couldn't be read.</p>
{/if}
apps/web/src/routes/(protected)/holdings/+page.svelte
+10 -2
View File
@@ -19,9 +19,17 @@
{#if !data.configured}
<p>No configuration selected.</p>
{:else if data.readFailed}
<p role="alert">Saved reports couldn't be read.</p>
<p role="alert">
Saved reports couldn't be read. Prepare reports again in <a
href={resolve('/(protected)/settings')}>Settings</a
>.
</p>
{:else if !data.asOf}
<p>No prepared default date is available.</p>
<p>
No prepared default date is available. Prepare reports in <a
href={resolve('/(protected)/settings')}>Settings</a
>.
</p>
{/if}
<form method="POST" action={resolve('/(protected)/holdings')}>
<fieldset class="panel" disabled={!data.configured}>
apps/web/src/routes/(protected)/page.server.test.ts
+57 -6
View File
@@ -1,11 +1,14 @@
import { testConfig } from '#lib/server/application.fixture.js';
import { beforeEach, expect, it, vi } from 'vitest';
import { readReportPublication } from '@hledger-reports/snapshots';
import { readReportPublication, readValidation } from '@hledger-reports/snapshots';
import type { OverviewData } from '@hledger-reports/hledger';
import type { ApplicationContext } from '#lib/server/application.js';
import { load } from './+page.server.js';
vi.mock('@hledger-reports/snapshots', () => ({ readReportPublication: vi.fn() }));
vi.mock('@hledger-reports/snapshots', () => ({
readReportPublication: vi.fn(),
readValidation: vi.fn()
}));
vi.mock('@hledger-reports/jobs', () => {
throw new Error('Browsing must not load jobs.');
});
@@ -85,19 +88,29 @@ beforeEach(() => {
auth: async () => null
}));
vi.mocked(readReportPublication).mockResolvedValue(publication);
vi.mocked(readValidation).mockResolvedValue(null);
});
it('returns an unconfigured shell without reads when application is unavailable', async () => {
expect(await load(event({ application: null }))).toEqual({
configured: false,
overview: null,
readFailed: false
readFailed: false,
validation: null,
validationReadFailed: false
});
expect(readReportPublication).not.toHaveBeenCalled();
expect(readValidation).not.toHaveBeenCalled();
});
it('passively returns the configured saved overview without private publication context', async () => {
const request = event();
const data = await load(request);
expect(data).toEqual({ configured: true, overview, readFailed: false });
expect(data).toEqual({
configured: true,
overview,
readFailed: false,
validation: null,
validationReadFailed: false
});
expect(data.overview).toBe(overview);
expect(readReportPublication).toHaveBeenCalledExactlyOnceWith('/PRIVATE/context/snapshots');
expect(request.setHeaders).toHaveBeenCalledWith({ 'cache-control': 'no-store' });
@@ -109,10 +122,48 @@ it.each(['missing', 'other-journal'])(
vi.mocked(readReportPublication).mockResolvedValue(
condition === 'missing' ? null : { ...publication, journal: '/PRIVATE/other' }
);
expect(await load(event())).toEqual({ configured: true, overview: null, readFailed: false });
expect(await load(event())).toEqual({
configured: true,
overview: null,
readFailed: false,
validation: null,
validationReadFailed: false
});
}
);
it('sanitizes read failures without automatic repair', async () => {
vi.mocked(readReportPublication).mockRejectedValue(new Error('PRIVATE 123 USD'));
expect(await load(event())).toEqual({ configured: true, overview: null, readFailed: true });
expect(await load(event())).toEqual({
configured: true,
overview: null,
readFailed: true,
validation: null,
validationReadFailed: false
});
});
it('loads saved validation independently without private identity or financial preparation claims', async () => {
const validation = { outcome: 'passed' as const, completedAt: '2026-10-22T12:00:00Z' };
vi.mocked(readValidation).mockResolvedValue({
format: 1,
journal,
request: { id: 'PRIVATE request', createdAt: '2026-10-22T11:00:00Z' },
result: validation
});
expect(await load(event())).toEqual({
configured: true,
overview,
readFailed: false,
validation,
validationReadFailed: false
});
expect(readValidation).toHaveBeenCalledExactlyOnceWith('/PRIVATE/context/snapshots');
vi.mocked(readValidation).mockRejectedValue(new Error('PRIVATE corrupt validation'));
expect(await load(event())).toEqual({
configured: true,
overview,
readFailed: false,
validation: null,
validationReadFailed: true
});
});
apps/web/src/routes/(protected)/registers/cash/+page.svelte
+10 -2
View File
@@ -25,7 +25,11 @@
</section>
{:else}
{#if data.readFailed}
<p role="alert">Saved reports couldn't be read.</p>
<p role="alert">
Saved reports couldn't be read. Prepare reports again in <a
href={resolve('/(protected)/settings')}>Settings</a
>.
</p>
{/if}
{#if data.accounts !== null && !data.readFailed}
<section class="panel" aria-labelledby="accounts-heading">
@@ -62,7 +66,11 @@
{/if}
</section>
{:else if !data.readFailed}
<p>No prepared reports available.</p>
<p>
No prepared reports available. Prepare reports in <a href={resolve('/(protected)/settings')}
>Settings</a
>.
</p>
{/if}
{/if}
</main>
apps/web/src/routes/(protected)/registers/cash/[accountId]/+page.svelte
+11 -2
View File
@@ -1,4 +1,5 @@
<script lang="ts">
import { resolve } from '$app/paths';
import CashRegister from '#lib/registers/cash/CashRegister.svelte';
import { accountLabel } from '#lib/accounts.js';
import type { PageData } from './$types';
@@ -30,7 +31,11 @@
</p>
</section>
{:else if data.readFailed}
<p role="alert">Saved reports couldn't be read.</p>
<p role="alert">
Saved reports couldn't be read. Prepare reports again in <a
href={resolve('/(protected)/settings')}>Settings</a
>.
</p>
{:else}
{#if data.preparedAt}<p>
Prepared at <time datetime={data.preparedAt}>{data.preparedAt}</time>.
@@ -43,7 +48,11 @@
{:else if data.access === 'ready'}
<CashRegister {data} labels={data.accountLabels} />
{:else}
<p>No prepared reports available.</p>
<p>
No prepared reports available. Prepare reports in <a href={resolve('/(protected)/settings')}
>Settings</a
>.
</p>
{/if}
{/if}
</main>
apps/web/src/routes/(protected)/registers/investments/+page.svelte
+10 -2
View File
@@ -20,7 +20,11 @@
<h2 class="section-title" id="configuration-heading">No configuration selected.</h2>
</section>
{:else if data.readFailed}
<p role="alert">Saved reports couldn't be read.</p>
<p role="alert">
Saved reports couldn't be read. Prepare reports again in <a
href={resolve('/(protected)/settings')}>Settings</a
>.
</p>
{:else if data.accounts !== null}
<section class="panel" aria-labelledby="accounts-heading">
<h2 class="section-title" id="accounts-heading">Accounts</h2>
@@ -56,7 +60,11 @@
{/if}
</section>
{:else}
<p>No prepared reports available.</p>
<p>
No prepared reports available. Prepare reports in <a href={resolve('/(protected)/settings')}
>Settings</a
>.
</p>
{/if}
</main>
apps/web/src/routes/(protected)/registers/investments/[accountId]/+page.svelte
+10 -2
View File
@@ -34,7 +34,11 @@
<h2 class="section-title" id="configuration-heading">No configuration selected.</h2>
</section>
{:else if data.readFailed}
<p role="alert">Saved reports couldn't be read.</p>
<p role="alert">
Saved reports couldn't be read. Prepare reports again in <a
href={resolve('/(protected)/settings')}>Settings</a
>.
</p>
{:else}
{#if data.preparedAt}
<p>
@@ -49,7 +53,11 @@
{:else if data.access === 'ready' && data.register !== null}
<InvestmentRegister data={data.register} labels={data.accountLabels} />
{:else}
<p>No prepared reports available.</p>
<p>
No prepared reports available. Prepare reports in <a href={resolve('/(protected)/settings')}
>Settings</a
>.
</p>
{/if}
{/if}
</main>
apps/web/src/routes/(protected)/prepare/+page.server.ts → apps/web/src/routes/(protected)/settings/+page.server.ts
+13 -8
View File
@@ -5,7 +5,8 @@ import {
} from '#lib/server/preparation.js';
import { fail, redirect } from '@sveltejs/kit';
import { InvalidBody, readRequestText } from '#lib/server/http.js';
import { readSavedReports } from '#lib/server/reports.js';
import { readSavedReports, readSavedValidation } from '#lib/server/reports.js';
import type { JournalValidationResult } from '@hledger-reports/snapshots';
import type { Actions, PageServerLoad } from './$types';
export const load = (async (event) => {
@@ -15,6 +16,8 @@ export const load = (async (event) => {
const configured = application !== null;
let preparation: ReportsPreparationState | 'unavailable' = 'idle';
let preparedAt: string | null = null;
let validation: JournalValidationResult | null = null;
let validationReadFailed = false;
if (application) {
preparation = 'unavailable';
try {
@@ -22,13 +25,15 @@ export const load = (async (event) => {
} catch {
// Queue diagnostics are private; status failure does not hide saved reports.
}
const { publication } = await readSavedReports(
application.paths.snapshotDirectory,
application.config.journal
);
preparedAt = publication?.overview.preparedAt ?? null;
const [reports, savedValidation] = await Promise.all([
readSavedReports(application.paths.snapshotDirectory, application.config.journal),
readSavedValidation(application.paths.snapshotDirectory, application.config.journal)
]);
preparedAt = reports.publication?.overview.preparedAt ?? null;
validation = savedValidation.validation;
validationReadFailed = savedValidation.readFailed;
}
return { configured, preparation, preparedAt };
return { configured, preparation, preparedAt, validation, validationReadFailed };
}) satisfies PageServerLoad;
export const actions = {
@@ -61,6 +66,6 @@ export const actions = {
} catch {
return fail(503, { message: 'Reports could not be queued. Try again.' });
}
redirect(303, '/prepare');
redirect(303, '/settings');
}
} satisfies Actions;
apps/web/src/routes/(protected)/prepare/+page.svelte → apps/web/src/routes/(protected)/settings/+page.svelte
+16 -8
View File
@@ -1,5 +1,6 @@
<script lang="ts">
import { resolve } from '$app/paths';
import ValidationStatus from '#lib/preparation/ValidationStatus.svelte';
import type { PageProps } from './$types';
let { data, form }: Pick<PageProps, 'data' | 'form'> = $props();
@@ -7,12 +8,12 @@
</script>
<svelte:head>
<title>Prepare reports — hledger reports</title>
<title>Settings — hledger reports</title>
<meta name="description" content="Manually prepare saved overview and account reports." />
</svelte:head>
<main class="page" id="main" tabindex="-1">
<h1 class="page-title">Prepare reports</h1>
<h1 class="page-title">Settings</h1>
{#if !data.configured}
<h2 class="section-title">No configuration selected.</h2>
<p>
@@ -20,7 +21,8 @@
is loaded by default.
</p>
{:else}
<div class="preparation-status" aria-label="Preparation status">
<h2 class="section-title">Report preparation</h2>
<section class="preparation-status" aria-label="Preparation status">
<p>Status when this page loaded.</p>
{#if data.preparation === 'pending'}
<p>Report preparation was queued. Waiting for the worker.</p>
@@ -28,8 +30,8 @@
<p>Worker was preparing reports.</p>
{:else if data.preparation === 'failed'}
<p role="alert">
Report preparation failed. Any previously saved reports are unchanged. Check the journal
and worker, then prepare reports again to retry.
Report preparation failed. Reload saved reports to see the available result. Check the
journal and worker, then prepare reports again to retry.
</p>
{:else if data.preparation === 'unavailable'}
<p>Report preparation status is unavailable.</p>
@@ -38,17 +40,23 @@
{/if}
{#if data.preparedAt}
<p>
Last saved reports prepared at <time datetime={data.preparedAt}>{data.preparedAt}</time>.
Last saved reports prepared at <time
aria-label="Financial reports prepared at"
datetime={data.preparedAt}>{data.preparedAt}</time
>.
</p>
{/if}
</div>
<form method="POST" action={resolve('/(protected)/prepare')}>
<ValidationStatus validation={data.validation} readFailed={data.validationReadFailed} />
</section>
<p><a href={resolve('/(protected)/settings')} data-sveltekit-reload>Reload status</a></p>
<form method="POST" action={resolve('/(protected)/settings')}>
<button class="ui-button ui-button--primary" type="submit" disabled={preparing}
>Prepare reports</button
>
</form>
{/if}
{#if form?.message}<p role="alert">{form.message}</p>{/if}
<p><a href={resolve('/(protected)/queries')}>Live queries</a> runs reports on demand.</p>
</main>
<style>
apps/web/src/routes/(protected)/prepare/page.server.test.ts → apps/web/src/routes/(protected)/settings/page.server.test.ts
+48 -7
View File
@@ -2,7 +2,7 @@ import { testConfig } from '#lib/server/application.fixture.js';
import { beforeEach, expect, it, vi } from 'vitest';
import { readReportsPreparation, requestReportPreparation } from '#lib/server/preparation.js';
import type { JobsClient } from '@hledger-reports/jobs';
import { readReportPublication } from '@hledger-reports/snapshots';
import { readReportPublication, readValidation } from '@hledger-reports/snapshots';
import type { ApplicationContext } from '#lib/server/application.js';
import { actions, load } from './+page.server.js';
@@ -10,7 +10,10 @@ vi.mock('#lib/server/preparation.js', () => ({
readReportsPreparation: vi.fn(),
requestReportPreparation: vi.fn()
}));
vi.mock('@hledger-reports/snapshots', () => ({ readReportPublication: vi.fn() }));
vi.mock('@hledger-reports/snapshots', () => ({
readReportPublication: vi.fn(),
readValidation: vi.fn()
}));
vi.mock('@hledger-reports/hledger', () => {
throw new Error('Preparation routes must not execute native reports.');
});
@@ -35,7 +38,7 @@ const application: ApplicationContext = {
jobsClient: client as unknown as JobsClient
};
function event(options: { method?: 'GET' | 'POST'; origin?: string; body?: string } = {}) {
const url = new URL('http://localhost:5173/prepare');
const url = new URL('http://localhost:5173/settings');
const method = options.method ?? 'GET';
return {
url,
@@ -64,6 +67,7 @@ beforeEach(() => {
vi.mocked(readReportsPreparation).mockResolvedValue('idle');
vi.mocked(requestReportPreparation).mockResolvedValue(undefined);
vi.mocked(readReportPublication).mockResolvedValue(null);
vi.mocked(readValidation).mockResolvedValue(null);
});
it('passively observes preparation status without requesting work', async () => {
vi.mocked(readReportsPreparation).mockResolvedValue('running');
@@ -71,7 +75,9 @@ it('passively observes preparation status without requesting work', async () =>
expect(await load(request)).toEqual({
configured: true,
preparation: 'running',
preparedAt: null
preparedAt: null,
validation: null,
validationReadFailed: false
});
expect(request.setHeaders).toHaveBeenCalledWith({ 'cache-control': 'no-store' });
expect(readReportsPreparation).toHaveBeenCalledExactlyOnceWith(client);
@@ -93,26 +99,40 @@ it('serializes only saved time, never financial data, journal or publication dir
});
vi.mocked(readReportsPreparation).mockRejectedValue(new Error('PRIVATE queue'));
const result = await load(event());
expect(result).toEqual({ configured: true, preparation: 'unavailable', preparedAt });
expect(result).toEqual({
configured: true,
preparation: 'unavailable',
preparedAt,
validation: null,
validationReadFailed: false
});
expect(JSON.stringify(result)).not.toMatch(/PRIVATE|overview|accounts|directory|journal/);
});
it('unavailable application makes no reads or writes', async () => {
readLocals.mockReturnValue({ application: null, session: null, auth: async () => null });
expect(await load(event())).toEqual({ configured: false, preparation: 'idle', preparedAt: null });
expect(await load(event())).toEqual({
configured: false,
preparation: 'idle',
preparedAt: null,
validation: null,
validationReadFailed: false
});
expect((await post(event({ method: 'POST' })))?.status).toBe(503);
expect(readReportsPreparation).not.toHaveBeenCalled();
expect(readReportPublication).not.toHaveBeenCalled();
expect(readValidation).not.toHaveBeenCalled();
expect(requestReportPreparation).not.toHaveBeenCalled();
});
it('delegates each explicit request to the start/join coordinator and redirects without polling', async () => {
for (let index = 0; index < 2; index++) {
const request = event({ method: 'POST' });
await expect(post(request)).rejects.toMatchObject({ status: 303, location: '/prepare' });
await expect(post(request)).rejects.toMatchObject({ status: 303, location: '/settings' });
expect(request.setHeaders).toHaveBeenCalledWith({ 'cache-control': 'no-store' });
}
expect(vi.mocked(requestReportPreparation).mock.calls).toEqual([[client], [client]]);
expect(readReportsPreparation).not.toHaveBeenCalled();
expect(readReportPublication).not.toHaveBeenCalled();
expect(readValidation).not.toHaveBeenCalled();
expect(client.close).not.toHaveBeenCalled();
});
it('returns a safe retryable error on enqueue failure', async () => {
@@ -133,5 +153,26 @@ it.each([
expect(requestReportPreparation).not.toHaveBeenCalled();
expect(readReportsPreparation).not.toHaveBeenCalled();
expect(readReportPublication).not.toHaveBeenCalled();
expect(readValidation).not.toHaveBeenCalled();
}
);
it('shows saved validation even when financial reading fails and does not derive queue status from it', async () => {
const validation = { outcome: 'passed' as const, completedAt: '2026-05-01T12:30:00Z' };
vi.mocked(readReportPublication).mockRejectedValue(new Error('PRIVATE financial failure'));
vi.mocked(readReportsPreparation).mockResolvedValue('failed');
vi.mocked(readValidation).mockResolvedValue({
format: 1,
journal: application.config.journal,
request: { id: 'PRIVATE request', createdAt: '2026-05-01T12:00:00Z' },
result: validation
});
expect(await load(event())).toEqual({
configured: true,
preparation: 'failed',
preparedAt: null,
validation,
validationReadFailed: false
});
expect(requestReportPreparation).not.toHaveBeenCalled();
});
apps/web/src/routes/(protected)/settings/page.test.ts
+35
View File
@@ -0,0 +1,35 @@
import { render } from 'svelte/server';
import { expect, it, vi } from 'vitest';
import Settings from './+page.svelte';
vi.mock('$app/paths', () => ({
resolve: (route: string) => route.replace('/(protected)', '')
}));
it('keeps failed execution, saved finances and recorded validation distinct', () => {
const preparedAt = '2026-05-01T12:01:00Z';
const completedAt = '2026-05-01T12:00:00Z';
const { body } = render(Settings, {
props: {
data: {
configured: true,
preparation: 'failed',
preparedAt,
validation: { outcome: 'passed', completedAt },
validationReadFailed: false,
session: null,
accountLabels: [],
registers: { cash: false, investment: false }
},
form: null
}
});
const text = body.replace(/<[^>]*>/g, ' ').replace(/\s+/g, ' ');
expect(text).toContain('Report preparation failed.');
expect(text).toContain('Reload saved reports');
expect(text).toContain(`Last saved reports prepared at ${preparedAt}`);
expect(text).toContain('Last recorded strict validation: Passed.');
expect(text).toContain(`Completed at ${completedAt}`);
// An inspection-alias failure can follow a successful financial commit.
expect(text).not.toMatch(/reports[^.]*unchanged/i);
});
apps/worker/README.md
+25 -9
View File
@@ -55,25 +55,41 @@ builds. The web requests these jobs; this process performs them.
## Explicit report preparation
Web displays saved reports or no reports. A separate `/prepare` form POST explicitly calls the web-owned `requestReportPreparation(client)`; reading reports or preparation status never creates work. The worker executes requested jobs only. Boot, idle time, journal edits and restart do not request financial jobs. There are no source hashes, filesystem watches, discovery/reconciliation loops, staleness checks or automatic follow-ups. Scheduled query retention below is independent operational maintenance.
Web displays saved reports or no reports. The inline `/settings` preparation form POST explicitly calls the web-owned `requestReportPreparation(client)`; reading reports or preparation status never creates work. The worker executes requested jobs only. Boot, idle time, journal edits and restart do not request financial jobs. There are no source hashes, filesystem watches, discovery/reconciliation loops, staleness checks or automatic follow-ups. Scheduled query retention below is independent operational maintenance. `/prepare` is removed without a redirect or POST alias; Settings also links to the existing `/queries` tool.
`src/workflows/index.ts` explicitly registers all five implementations. `src/workflows/prepare-reports/run.ts` owns the concrete handler for `prepareReportsSpec`: persisted SDK name `hledger.prepare-dashboard`, version **1**, **void input**, full **OverviewData output**, **maximumAttempts 1**, and no schedule. Explicit requests join active work; after completion or failure, another explicit request starts a new run even with unchanged inputs. A failed run remains failed until an explicit retry; editing the journal does not clear it.
The handler reads the previous saved publication, allowing missing, corrupt or unsupported data to be replaced by an explicit job. Only a usable current-format publication with a matching journal path allows reuse of prior account IDs. It creates a stage and calls `prepareReports(journal, { writeRows: stage.writeRows })`. The library uses one cached hledger journal view for Overview and every full-history account register, delivering exact rows in awaited, ordered batches of at most 50. These are not persisted pages: the store writes **one whole JSON file per account** without pooling every register in Node. The library owns account selection, dates, quantities and native balances. Edits during a build do not trigger another job: the publication represents the view loaded for that execution.
The handler awaits standalone `checkJournal(journal)`, saves its completed outcome,
and proceeds to financial staging only after a recorded pass. An incomplete check
preserves earlier evidence; check or evidence-write failure stops financial work.
The extra preflight adds a third native process; the final financial batch remains
independently strict. It is not a replay-cached SDK step, and a recorded pass is
not a completed financial set or proof of unchanged inputs.
After successful native production and awaited staged I/O, `stage.publish(reports.overview, reports.accounts, journal)` commits the complete Overview, catalog and account files. The handler returns **its own calculated OverviewData**, not a readback of whichever publication is currently selected, and always awaits stage discard. Last-good data remains available through preparation failures. There are no SDK financial checkpoints; old queued runs with legacy steps can finish under this handler without changing those steps.
Snapshots owns immutable evidence and its selection rules; the web exposes only the
saved outcome/time. See [validation ownership](../../docs/architecture.md#last-recorded-validation)
for failure semantics and [recorded evidence](../../packages/snapshots/README.md#recorded-validation)
for storage/privacy/retention. Financial format **5**, six-month views and workflow
identity/version/output/retry policy are unchanged.
Only publication format **4** is accepted. Whole-account files retain the library-owned original transaction and every posting, comments, ordered tags, exact nested amounts, account context and private source/assertion positions. These records reflect hledger's account-register export, not a verbatim source archive or complete investment cost/basis record; the worker does not recover annotations stripped by hledger. Source paths remain private stored metadata, never browser payloads or job output. The existing **64 MiB per-file bound** is unchanged.
After recording a pass, the handler reads the previous saved publication, allowing missing, corrupt or unsupported data to be replaced by an explicit job. Only a usable current-format publication with a matching journal path allows reuse of prior account IDs. It creates a stage and calls `prepareReports(journal, { writeRows: stage.writeRows })`. The library uses one cached hledger journal view for Overview and every full-history account register, delivering exact rows in awaited, ordered batches of at most 50. These are not persisted pages: the store writes **one whole JSON file per account** without pooling every register in Node. The library owns account selection, dates, quantities and native balances. Edits during a build do not trigger another job: the publication represents the view loaded for that execution.
The store pins an immutable private `.sets/` directory for each read; `current/` is an atomically updated human-inspection alias. Unsupported formats (including format 3's table-only rows, formats 2/1 and raw Overview files) provide no current report data or reusable account IDs. Their files remain untouched: the user must explicitly **Prepare reports again**, with no automatic upgrade, snapshot migration, compatibility reader or garbage collection. This report-file policy is separate from the queued SDK job policies described here. OpenWorkflow persists only the handler's own **OverviewData** financial result in its private SQLite database, not duplicate account transaction arrays or source positions.
After successful native production and awaited staged I/O, `stage.publish(reports.overview, reports.accounts, journal)` commits the complete Overview, catalog and account files. The handler returns **its own calculated OverviewData**, not a readback of whichever publication is currently selected, and always awaits stage discard. Failure before the financial directory commit retains the previous complete set.
An inspection-only `current` alias failure after commit may fail the job while the new
valid complete set remains selectable. No completed set is rolled back or deleted;
Settings tells users to reload saved reports, not that failure guarantees unchanged data. There are no SDK financial checkpoints; old queued runs with legacy steps can finish under this handler without changing those steps.
Only publication format **5** is accepted; this change does not bump it. Whole-account files retain the library-owned original transaction and every posting, comments, ordered tags, exact nested amounts, account context and private source/assertion positions. These records reflect hledger's account-register export, not a verbatim source archive or complete investment cost/basis record; the worker does not recover annotations stripped by hledger. Source paths remain private stored metadata, never browser payloads or job output. There is no application-imposed saved-file size cap.
The store pins an immutable private `.sets/` directory for each read; `current/` is an atomically updated human-inspection alias. Unsupported formats (including format 4, format 3's table-only rows, formats 2/1 and raw Overview files) provide no current report data or reusable account IDs. Their files remain untouched: the user must explicitly **Prepare reports again**, with no automatic upgrade, snapshot migration, compatibility reader or garbage collection. This report-file policy is separate from the queued SDK job policies described here. OpenWorkflow persists only the handler's own **OverviewData** financial result in its private SQLite database, not duplicate account transaction arrays or source positions.
See the [snapshot store](../../packages/snapshots/README.md) for last-good recovery and storage growth, and [web requester](../web/README.md) for request coordination and [shared contracts](../../packages/jobs/README.md) for persisted identities.
## Register preparation limits
## Register preparation resources
The hledger library has one private `process.ts` boundary for all actual calls; unused standalone Overview/register execution APIs are removed. Version checking and explicit inspection use that boundary outside financial jobs. The combined application producer retains its **180-second total deadline**, **1 GiB total native output**, **1 MiB metadata/financial prefix**, and **4 MiB per native tuple**. Batches are written sequentially with backpressure into whole-account files. Native JSON generation can still use substantial hledger memory; browser pagination does not reduce native preparation cost or whole-account reads. Health remains separate from preparation.
The hledger library has one private `process.ts` boundary for all actual calls; unused standalone Overview/register execution APIs are removed. Version checking and explicit inspection use that boundary outside financial jobs. The native producer has no application-imposed execution deadline, output/row/tuple cap or saved-file size limit. Batches are written sequentially with backpressure into whole-account files. Native JSON generation can still use substantial hledger memory; browser pagination does not reduce native preparation cost or whole-account reads. Health remains separate from preparation.
These are execution limits, not a universal journal-capacity guarantee. A larger input may fail and leave the previous complete dataset available. Native CSV is not a fallback: it rounds some nonzero quantities to zero. Row callbacks own their I/O; native cancellation kills/reaps the process, then waits for the current write to settle before cleanup. The worker always discards its own unfinished stage, but completed directories are not removed, since they may already be selected. As with the existing publications, successful/orphaned completed directories remain until a separately agreed retention policy.
There is no universal journal-capacity guarantee: larger input may exhaust native or Node resources. A failure before financial commit leaves the previous complete dataset available. Native CSV is not a fallback: it rounds some nonzero quantities to zero. Row callbacks own their I/O; native cancellation kills/reaps the process, then waits for the current write to settle before cleanup. The worker always discards its own unfinished stage, but completed directories are not removed, since they may already be selected. As with the existing publications, successful/orphaned completed directories remain until a separately agreed retention policy.
## Current-journal queries
@@ -116,6 +132,6 @@ pnpm config:validate --config fixtures/config/hledger-reports.yaml
`--config-only` validates the full resolved web configuration, including authentication secrets, using the current environment and prints only success metadata or a safe failure code. Explicit `--config` takes precedence over `CONFIG_PATH`; neither mode discovers personal configuration.
Default preflight uses worker-only configuration resolution (no web secret requirement), verifies hledger **1.99.5** and runs read-only `check --strict` on the configured journal. It does not create query jobs or prepare snapshots. It is separate from worker startup and query execution.
Default preflight uses worker-only configuration resolution (no web secret requirement), verifies hledger **1.99.5** and runs read-only `check --strict` on the configured journal. It does not create query jobs, prepare snapshots or write validation evidence. It remains read-only, separate from worker startup, preparation preflight recording and query execution.
Tests exercise real hledger/SQLite workflows with fictional fixtures: whole-set preparation and recovery, live edits, current includes, read-only source handling, safe failures and correction, durable query output without query capture/result files, legacy rejection without file removal, and startup/shutdown ordering. They require the pinned native engine on PATH. Fixture data is under this app's [fixtures](fixtures/README.md).
apps/worker/src/workflows/prepare-reports/run.integration.test.ts
+26 -2
View File
@@ -18,7 +18,8 @@ import {
import {
readAccountTransactions,
readOverview,
readReportPublication
readReportPublication,
readValidation
} from '@hledger-reports/snapshots';
import { startWorker } from '../../index.js';
@@ -87,6 +88,15 @@ it('explicitly prepares a saved overview through the real SDK without a browser
status: 'completed'
});
const data = (await readOverview(f.snapshots))!;
const validation = (await readValidation(f.snapshots))!;
expect(validation).toMatchObject({
journal: f.journal,
request: { id: run.id, createdAt: run.createdAt.toISOString() },
result: { outcome: 'passed', completedAt: expect.any(String) }
});
expect(Date.parse(validation.result.completedAt)).toBeLessThanOrEqual(
Date.parse(data.preparedAt)
);
expect(data.asOf).toBe('2026-04-16');
expect(data.journalCoverage).toEqual({
transactionCount: 9,
@@ -198,6 +208,7 @@ it('explicitly prepares a saved overview through the real SDK without a browser
await service.stop();
service = undefined;
expect(await readOverview(f.snapshots)).toEqual(data);
expect(await readValidation(f.snapshots)).toEqual(validation);
expect(await readFile(f.journal, 'utf8')).toBe('include included.journal\n');
expect(await readFile(f.included, 'utf8')).toBe(source);
expect(await readFile(f.legacy, 'utf8')).toBe('legacy preview unchanged');
@@ -228,8 +239,13 @@ it('retains last-good data without retry flooding, permits an explicit SDK retry
expect(await readdir(join(f.snapshots, '.sets'))).toEqual(savedSets);
expect(failed.output).toBeNull();
expect(failed.error?.message).toBe(
'Prepared transactions could not be read. Check the journal.'
'hledger strict journal validation failed. Run hledger check --strict directly for diagnostics.'
);
expect(await readValidation(f.snapshots)).toMatchObject({
journal: f.journal,
request: { id: failed.id },
result: { outcome: 'could-not-validate', completedAt: expect.any(String) }
});
expect(JSON.stringify(failed.error)).not.toContain('PRIVATE');
expect(JSON.stringify(failed.error)).not.toContain(f.directory);
expect(await readOverview(f.snapshots)).toEqual(first);
@@ -245,6 +261,8 @@ it('retains last-good data without retry flooding, permits an explicit SDK retry
await enqueuePreparation(client);
await settled(client, 3, 'failed');
expect(await readOverview(f.snapshots)).toEqual(first);
const failedValidation = await readValidation(f.snapshots);
expect(failedValidation).toMatchObject({ result: { outcome: 'could-not-validate' } });
await writeFile(f.included, changedSalary());
expect(
(
@@ -258,6 +276,7 @@ it('retains last-good data without retry flooding, permits an explicit SDK retry
).toEqual([{ commodity: 'USD', quantity: '100.00' }]);
// Correction and a live query neither request preparation nor clear a failure.
await delay(1500);
expect(await readValidation(f.snapshots)).toEqual(failedValidation);
expect(await readLatestPreparationState(client)).toBe('failed');
expect(await preparations(client)).toHaveLength(3);
await enqueuePreparation(client);
@@ -267,6 +286,10 @@ it('retains last-good data without retry flooding, permits an explicit SDK retry
{ commodity: 'USD', quantity: '1500.00' }
]);
expect(retried.output).toEqual(updated);
expect(await readValidation(f.snapshots)).toMatchObject({
request: { id: retried.id },
result: { outcome: 'passed' }
});
await writeFile(f.included, '');
await enqueuePreparation(client);
const emptyRun = await settled(client, 5);
@@ -455,6 +478,7 @@ it('creates no financial work on boot, idle, edits or restart; every completed r
expect(await preparations(client)).toEqual([]);
expect(await readLatestPreparationState(client)).toBe('idle');
expect(await readOverview(f.snapshots)).toBeNull();
expect(await readValidation(f.snapshots)).toBeNull();
await writeFile(f.included, changedSalary());
await delay(1500);
expect(await preparations(client)).toEqual([]);
apps/worker/src/workflows/prepare-reports/run.test.ts
+77 -3
View File
@@ -1,16 +1,27 @@
import { beforeEach, expect, it, vi } from 'vitest';
import { prepareReports, type OverviewData } from '@hledger-reports/hledger';
import {
checkJournal,
JournalCheckError,
prepareReports,
type OverviewData
} from '@hledger-reports/hledger';
import {
createReportStage,
readReportPublication,
writeValidation,
type ReportPublication
} from '@hledger-reports/snapshots';
import { createPrepareReports } from './run.js';
vi.mock('@hledger-reports/hledger', () => ({ prepareReports: vi.fn() }));
vi.mock('@hledger-reports/hledger', async (original) => ({
...(await original<typeof import('@hledger-reports/hledger')>()),
checkJournal: vi.fn(),
prepareReports: vi.fn()
}));
vi.mock('@hledger-reports/snapshots', () => ({
createReportStage: vi.fn(),
readReportPublication: vi.fn()
readReportPublication: vi.fn(),
writeValidation: vi.fn()
}));
const journal = '/owned/main.journal';
const directory = '/owned/snapshots';
@@ -40,6 +51,8 @@ const checkpoint = vi.fn();
const context = { run, step: { run: checkpoint } } as unknown as Parameters<typeof prepare>[0];
beforeEach(() => {
vi.resetAllMocks();
vi.mocked(checkJournal).mockResolvedValue(undefined);
vi.mocked(writeValidation).mockResolvedValue(undefined);
vi.mocked(readReportPublication).mockResolvedValue(previous);
vi.mocked(createReportStage).mockResolvedValue(stage);
vi.mocked(prepareReports).mockImplementation(async (_journal, options) => {
@@ -48,6 +61,54 @@ beforeEach(() => {
});
});
it('records a completed nonzero check and stops before touching financial reports', async () => {
const error = new JournalCheckError('nonzero-exit');
vi.mocked(checkJournal).mockRejectedValue(error);
await expect(prepare(context)).rejects.toBe(error);
expect(writeValidation).toHaveBeenCalledExactlyOnceWith(directory, run, journal, {
outcome: 'could-not-validate',
completedAt: expect.any(String)
});
expect(readReportPublication).not.toHaveBeenCalled();
expect(createReportStage).not.toHaveBeenCalled();
expect(prepareReports).not.toHaveBeenCalled();
});
it.each([new JournalCheckError('not-completed'), new Error('unexpected check failure')])(
'preserves previous evidence when checking did not complete: %s',
async (error) => {
vi.mocked(checkJournal).mockRejectedValue(error);
await expect(prepare(context)).rejects.toBe(error);
expect(writeValidation).not.toHaveBeenCalled();
expect(createReportStage).not.toHaveBeenCalled();
expect(prepareReports).not.toHaveBeenCalled();
}
);
it('does not start financial work when evidence cannot be saved', async () => {
const error = new Error('evidence unavailable');
vi.mocked(writeValidation).mockRejectedValue(error);
await expect(prepare(context)).rejects.toBe(error);
expect(checkJournal).toHaveBeenCalledOnce();
expect(readReportPublication).not.toHaveBeenCalled();
expect(createReportStage).not.toHaveBeenCalled();
expect(prepareReports).not.toHaveBeenCalled();
});
it('awaits durable evidence before financial staging', async () => {
let release!: () => void;
vi.mocked(writeValidation).mockReturnValue(new Promise<void>((resolve) => (release = resolve)));
const result = prepare(context);
try {
await vi.waitFor(() => expect(writeValidation).toHaveBeenCalledOnce());
expect(createReportStage).not.toHaveBeenCalled();
expect(prepareReports).not.toHaveBeenCalled();
} finally {
release();
}
expect(await result).toBe(overview);
});
it.each([true, false])(
'reuses prior IDs only for the selected journal (same journal: %s)',
async (same) => {
@@ -56,6 +117,14 @@ it.each([true, false])(
journal: same ? journal : '/other/book.journal'
});
expect(await prepare(context)).toBe(overview);
expect(checkJournal).toHaveBeenCalledExactlyOnceWith(journal);
expect(writeValidation).toHaveBeenCalledExactlyOnceWith(directory, run, journal, {
outcome: 'passed',
completedAt: expect.any(String)
});
expect(vi.mocked(writeValidation).mock.invocationCallOrder[0]).toBeLessThan(
vi.mocked(createReportStage).mock.invocationCallOrder[0]!
);
expect(createReportStage).toHaveBeenCalledExactlyOnceWith(
directory,
run,
@@ -95,6 +164,11 @@ it.each(['producer', 'write', 'publish'] as const)(
if (kind === 'write') stage.writeRows.mockRejectedValue(error);
if (kind === 'publish') stage.publish.mockRejectedValue(error);
await expect(prepare(context)).rejects.toBe(error);
// A later financial failure must not rewrite the completed preflight.
expect(writeValidation).toHaveBeenCalledExactlyOnceWith(directory, run, journal, {
outcome: 'passed',
completedAt: expect.any(String)
});
expect(stage.discard).toHaveBeenCalledExactlyOnceWith();
if (kind !== 'publish') expect(stage.publish).not.toHaveBeenCalled();
}
apps/worker/src/workflows/prepare-reports/run.ts
+21 -2
View File
@@ -1,10 +1,29 @@
import { prepareReports } from '@hledger-reports/hledger';
import { createReportStage, readReportPublication } from '@hledger-reports/snapshots';
import { checkJournal, JournalCheckError, prepareReports } from '@hledger-reports/hledger';
import {
createReportStage,
readReportPublication,
writeValidation
} from '@hledger-reports/snapshots';
import type { PrepareReportsWorkflow } from '@hledger-reports/jobs';
/** Execute one requested preparation; journal edits never enqueue follow-up work. */
export function createPrepareReports(journal: string, directory: string): PrepareReportsWorkflow {
return async ({ run }) => {
let checkFailure: JournalCheckError | undefined;
try {
await checkJournal(journal);
} catch (error) {
// Interrupted/unknown execution is not a completed validation result.
if (!(error instanceof JournalCheckError) || error.kind !== 'nonzero-exit') throw error;
checkFailure = error;
}
const outcome = checkFailure ? 'could-not-validate' : 'passed';
await writeValidation(directory, run, journal, {
outcome,
completedAt: new Date().toISOString()
});
if (checkFailure) throw checkFailure;
// Explicit preparation may replace missing or corrupt saved data.
const previous = await readReportPublication(directory).catch(() => null);
const stage = await createReportStage(
docs/architecture.md
+72 -29
View File
@@ -1,11 +1,12 @@
# Architecture
The product has three straightforward paths:
The product has straightforward read, request and produce paths:
```text
Overview / Transactions → read saved reports → render
Prepare reports form → request durable job → hledger → save complete set
Live query Run → request durable job → hledger → return exact result
Overview / registers → read saved financial files → render
Overview / Settings → read saved validation outcome/time → render
Settings form → request durable job → check + record → prepare complete set
Live query Run → request durable job → hledger → return exact result
```
The web app and worker share configuration and a private data directory. They
@@ -14,19 +15,21 @@ Storage is a library used by both processes, not another running service.
## Ownership
| Owner | Responsibility |
| -------------------- | --------------------------------------------------------------------------------- |
| `packages/hledger` | Hledger execution, exact decoding, financial interpretation and report-ready data |
| `apps/worker` | Execute requested jobs, publish their result, own service resources |
| `packages/snapshots` | Stable account IDs, current-format complete publication and pinned reads |
| `packages/jobs` | Shared job definitions, input contracts and queue connection lifetime |
| `apps/web` | Request validation, page composition, presentation and ordinary pagination |
| `packages/config` | Explicit journal-only configuration and fixed deployment paths |
| Owner | Responsibility |
| -------------------- | -------------------------------------------------------------------------------------- |
| `packages/hledger` | Hledger execution, exact decoding, financial interpretation and report-ready data |
| `apps/worker` | Execute requested jobs, publish their result, own service resources |
| `packages/snapshots` | Stable account IDs, complete financial publication/pinned reads and validation records |
| `packages/jobs` | Shared job definitions, input contracts and queue connection lifetime |
| `apps/web` | Request validation, page composition, presentation and ordinary pagination |
| `packages/config` | Explicit journal-only configuration and fixed deployment paths |
Web never opens journals, invokes hledger, interprets hledger JSON or calculates
financial balances. The library's private `process.ts` is the sole hledger
launch point: small text reads and streamed prepared output share limits,
cancellation and cleanup there. Financial calls happen in worker jobs; checking the
launch point: small text reads and streamed prepared output share cancellation
and cleanup there. There are no application-imposed native execution deadlines,
output/row/tuple limits or saved-file size caps; native and Node memory still matter.
Financial calls happen in worker jobs; checking the
installed version and the explicit journal-inspection command are operational
exceptions. There are no standalone Overview/register execution APIs or generic
CLI endpoint. Chart coordinates are approximate presentation only; tables
@@ -35,8 +38,8 @@ interpret accounting. Report storage does not recalculate or regroup report rows
## Prepared reports
A preparation job uses one cached hledger journal view for Overview and all actual
declared/used accounts. It streams account rows into private staging files, then
After recording a passed standalone preflight, a preparation job uses one cached
hledger financial journal view for Overview and all actual declared/used accounts. It streams account rows into private staging files, then
publishes the complete directory. Storage commits files; the workflow returns the
Overview it calculated. Readers independently select the latest complete set by
request ordering, and pin it for the request.
@@ -52,21 +55,24 @@ data, never paths. The server reads one account's complete JSON and slices 50 ro
before returning page data. Account changes and balances are already calculated by hledger;
pagination does not reconstruct them. No separate indexes or persisted page files.
Format 4 retains original exported transaction/posting metadata, recursively exact
amounts and account-relative facts in `PreparedAccountRow`: `accountDate` is
Financial format remains **5**. It retains original exported transaction/posting
metadata, recursively exact amounts and account-relative facts in `PreparedAccountRow`: `accountDate` is
distinct from the original `transaction.date`. The hledger library owns payee parsing, status
inheritance and canonical account/subtree posting membership. The current unfiltered
register changes only the contextual date; the mapper verifies that contract rather
than silently dropping other differences. Transaction indexes are scoped to the
prepared journal view, not durable cross-preparation IDs.
The existing Transactions UI remains unchanged. Its loader projects only the
selected 50 rows into the current table's public fields; source/assertion positions
and unused details stay private. Richer snapshots prepare the data boundary for a
later UI change, not a new job, transaction database or additional native export.
The existing six-month Overview and Cash/Investment registers remain unchanged.
Register loaders project only the selected page's public fields; source/assertion
positions and unused details stay private. This change adds no financial-format
transition, period controls or account sidebar.
Failed preparation retains last-good current-format files. Readers support only the
current complete report shape: Overview contains all four sections, and the journal
Failure before the financial directory commit retains the previous complete set.
After commit, an inspection-only `current` alias update can fail the job while the
new valid complete set remains selectable. Completed sets are never rolled back
or deleted, and a job failure must not be presented as proof that reports are unchanged.
Readers support only the current complete report shape: Overview contains all four sections, and the journal
identity, account directory and pinned set location are always present. If the latest
saved set uses an unsupported format, no report is returned; the user explicitly
prepares again. Older files are neither interpreted nor used as a fallback.
@@ -75,16 +81,53 @@ Reports for another configured journal are withheld. Journal identity is private
operational context, not an input fingerprint or freshness assertion. Files are left
untouched: there is no automatic cleanup, migration or regeneration.
## Last recorded validation
The worker awaits standalone `checkJournal` before financial staging. This adds one
strict process/parse to the existing two-process discovery/financial preparation:
three in total. A completed exit 0 records **Passed**; a completed normal nonzero
records **Could not validate** and stops preparation. Nonzero can mean rejected,
malformed, missing or unreadable input, not necessarily invalid accounting. Launch,
abort, signal, stream or unexpected incomplete failures preserve prior evidence.
No diagnostics are parsed or stored. Evidence-write failure stops financial work.
Snapshots stores one immutable operational record per completed check under
`data/snapshots/.validation/`, outside financial `.sets/`. Selection uses request
creation ordering (including the run-ID separator), then completion time and UUID
within that request; a delayed older request cannot displace a newer one. Temporary
files are ignored. Corrupt/unsupported selected evidence never falls back to an
older pass. Records are retained without cleanup or a history API/UI. Atomic rename
protects against partial publication, not power loss; an unwritten observation can be lost.
Overview reads evidence and financial files independently, without queue/native
access. Settings reads both alongside queue state. Missing/unsupported/other-journal
evidence is unknown; unreadable evidence is unavailable and does not hide readable
finances. Exact journal matching is private scoping, not a content fingerprint.
Only outcome/completion time reaches the browser, never journal/request identity,
set paths or native diagnostics.
The check and financial batch can observe different edits. The final financial batch
remains independently `--strict`; its cached-view guarantees are unchanged. A saved
pass survives later financial/storage failure, including post-commit alias failure,
and is not evidence that a complete financial set exists or inputs are fresh.
Preparation time remains distinct from check completion time. Read-only
`worker:inspect` runs native checking but never writes this evidence.
## Explicit execution
Only the Prepare reports form submits preparation. Concurrent submissions join
already queued/running work; a later explicit submission can prepare again, including
Only the inline Prepare reports form in Settings (`/settings`) submits preparation.
Concurrent submissions join already queued/running work; a later explicit submission can prepare again, including
after failure or without journal changes. Existing durable workflow identities remain
registered so queued work can finish safely.
Overview and Transactions do not read queue status. Preparation status belongs to
`/prepare` and describes the page load; reload to check again. There is no report API,
source watcher, filesystem-signature loop, input hashing, automatic preparation or
Overview and registers do not read queue status. Preparation status belongs to
Settings and describes this deployment's queue at page load, not journal-scoped
validation; the job still has void input. Reload to check again. The ordinary form
works without JavaScript and redirects with 303 to `/settings`; exact-origin,
form-content-type, empty/2-KiB body and no-store guards remain. `/prepare` is removed,
with no redirect or POST alias. The Settings utility entry replaces the standalone
Live queries utility link; Settings links to the existing `/queries` screen.
There is no report API, source watcher, filesystem-signature loop, input hashing, automatic preparation or
browser polling. Link hovering does not preload data; ordinary SvelteKit navigation
may read route data when the user follows a link. Already requested durable jobs can resume after worker restart;
restart does not create a new preparation request.
docs/development.md
+30 -7
View File
@@ -34,16 +34,16 @@ This uses the checked-in Compose files and your ignored live configuration.
A mise/fnox Docker task is an owner-local convenience, not part of the checked-in
project. The remaining commands below run from the repository root.
Open **http://127.0.0.1:5173/** and sign in. Overview and Transactions read saved reports; if none exist, they say so. Use **Prepare reports** (`/prepare`) to request a complete calculation, then reload that page to check status and navigate to the saved results. Startup, journal edits and browsing never request preparation automatically. Saved reports remain available while a requested job runs or when the worker is unavailable. The separate automated-test `lots.journal` fixture anchors at **2026-04-16**, giving **November 2025–April 2026** regardless of today's date; no fixture edits are needed. Overview's bars, categories, net-worth trend and coverage now share a complete preparation with all account registers. Use **Transactions** in the primary navigation, then select an account; Next/Previous links read one complete account JSON file and return a 50-row slice with native running balances. Account history is not restricted to the Overview window. Only complete current-format reports are displayed. Unsupported saved formats show “No prepared reports available.”; explicitly prepare again to create the current set. Existing saved files are not rewritten or deleted automatically. That fixture's net-worth point is 11825.00 USD at April 16, not the 12010.00 USD from later April 30 prices. Native whole-journal coverage includes 9 transactions dated April 1–16 in that fixture; its date span can include secondary/posting dates, while the financial window is anchored to the latest primary transaction date. See the [web contract](../apps/web/README.md) for interaction behavior and the [native library](../packages/hledger/README.md#overview-data) for dataset semantics.
Open **http://127.0.0.1:5173/** and sign in. Overview and Transactions read saved reports; if none exist, they say so. Use the inline **Prepare reports** form in **Settings** (`/settings`) to request a complete calculation, then reload Settings to check status and navigate to saved results. `/prepare` is removed without a redirect or POST alias; Settings links to the existing `/queries` tool. Startup, journal edits and browsing never request preparation automatically. Saved reports remain available while a requested job runs or when the worker is unavailable. The separate automated-test `lots.journal` fixture anchors at **2026-04-16**, giving **November 2025–April 2026** regardless of today's date; no fixture edits are needed. Overview's bars, categories, net-worth trend and coverage now share a complete preparation with all account registers. Use a configured **Cash register** or **Investment register** in the primary navigation, then select an account; ordinary pagination reads saved account activity with native running balances. Account history is not restricted to the Overview window. Only complete financial format-5 reports are displayed; this change keeps format 5 and the existing six-month Overview/register behavior. Unsupported saved formats show “No prepared reports available.”; explicitly prepare again to create the current set. Existing saved files are not rewritten or deleted automatically. That fixture's net-worth point is 11825.00 USD at April 16, not the 12010.00 USD from later April 30 prices. Native whole-journal coverage includes 9 transactions dated April 1–16 in that fixture; its date span can include secondary/posting dates, while the financial window is anchored to the latest primary transaction date. See the [web contract](../apps/web/README.md) for interaction behavior and the [native library](../packages/hledger/README.md#overview-data) for dataset semantics.
At `/queries`, submit `type:X` as of `2026-04-30` for the fixture expense total
Follow **Live queries** from Settings to the existing `/queries` screen; submit `type:X` as of `2026-04-30` for the fixture expense total
**100.00 USD**. Run waits for the worker through the same queue. Query reload still
clears its display; workflow output remains in SQLite, not a saved-report URL.
App/package source directories are mounted read-only. Vite updates the UI and
Node's watcher restarts the worker on imported source changes. Dependency,
Dockerfile, or non-mounted configuration changes require another `up --build`.
After journal/include/price edits, request **Prepare reports** again when you want a
After journal/include/price edits, request **Prepare reports** in Settings again when you want a
new saved set. There is no file watcher, input hashing, freshness comparison or automatic
follow-up job. Native includes are resolved when the requested calculation runs.
Explicit query Runs still read current input directly. Configuration changes require
@@ -76,7 +76,7 @@ full financial results and preparation history, while the inspectable report set
`data/snapshots/current/`. See [prepared reports](../packages/snapshots/README.md#prepared-reports)
for publication, recovery, growth and untouched unsupported files.
The current set has one `overview.json` with journal/report details and the account directory, plus one `transactions/<account-id>.json` for every account's complete activity. IDs are eight lowercase letters/digits, stable across preparations by exact native name; names never become paths. The server parses that one account file and returns a 50-row slice. It does not read every account or execute hledger. Whole sets are prepared privately and published together; `current` is the inspection link into hidden immutable sets. Each request pins its selected set. Larger preparations can still take substantial native time/memory: the combined producer is bounded to 180 seconds and 1 GiB output, not a universal capacity guarantee. Unsupported outputs and older complete sets are left untouched, but are not a fallback for the current format. Re-preparing unsupported data creates a new set and new account IDs; IDs are reused only from a supported set for the same journal. Do not remove active account files individually.
The current set has one `overview.json` with journal/report details and the account directory, plus one `transactions/<account-id>.json` for every account's complete activity. IDs are eight lowercase letters/digits, stable across preparations by exact native name; names never become paths. The server parses that one account file and returns a 50-row slice. It does not read every account or execute hledger. Whole sets are prepared privately and published together; `current` is the inspection link into hidden immutable sets. Each request pins its selected set. Larger preparations can still take substantial native time/memory. There are no application-imposed native execution deadlines, output/row/tuple limits or saved-file size caps, and no universal journal-capacity guarantee. Unsupported outputs and older complete sets are left untouched, but are not a fallback for the current format. Re-preparing unsupported data creates a new set and new account IDs; IDs are reused only from a supported set for the same journal. Do not remove active account files individually.
**SQLite query retention is enabled**. See [retention scope, scheduling and backup precautions](../apps/worker/README.md#query-job-retention);
it is not prepared-file cleanup.
@@ -165,7 +165,7 @@ docker compose -f docker-compose.dev.yml up --build --wait
Both command chains stop on any failure: inspection/copy must succeed before startup. The alternative refuses to merge into an already-populated host folder. Use your actual old
volume name if you changed the Compose project name. If you deliberately start
with an empty folder, the services create a fresh queue; use Prepare reports to create
with an empty folder, the services create a fresh queue; use Prepare reports in Settings to create
saved reports. The old volume still remains available. Never use `down -v` to migrate.
## Manual startup checks
@@ -174,7 +174,7 @@ Use the normal development Compose file and **http://127.0.0.1:5173/**; the owne
- **Warm boot:** keep the data folder, stop/start services and open Overview. Saved cards render immediately and retain their preparation timestamp. Editing inputs while stopped does not change that: a new set requires an explicit request.
- **Worker unavailable:** leave web running and stop worker. Browsing saved reports still works. A submitted preparation waits in the queue; starting the worker executes that already-requested job. Restart alone must not create a new request.
- **Journal changes:** preserve a copy of the fictional fixture, then edit a salary or applicable price. Reloading Overview still shows the saved values. Visit Prepare reports, submit, reload its status, then reload Overview to see the new values. Restore the fixture and explicitly prepare again afterward.
- **Journal changes:** preserve a copy of the fictional fixture, then edit a salary or applicable price. Reloading Overview still shows the saved values. Visit Settings, submit Prepare reports, reload its status, then reload Overview to see the new values. Restore the fixture and explicitly prepare again afterward.
- **Failure/retry:** request preparation against an invalid fictional journal. The job fails safely, worker health remains available and saved reports remain readable. Fix the source and explicitly request preparation again. Neither source restoration nor page reload retries the failed job. No raw native diagnostics should reach the browser.
- **Cold boot:** automated tests use disposable deployments. For an optional manual check, stop services and move only the prepared directory aside using the same worker image and existing data bind mount:
@@ -185,7 +185,30 @@ docker compose -f docker-compose.dev.yml run --rm --no-deps --entrypoint node wo
docker compose -f docker-compose.dev.yml up --build --wait
```
Overview now reports that no saved reports exist. Visit Prepare reports and submit the form; reload Overview after preparation finishes. The printed backup is preserved, as are the SQLite queue, configuration and journals. To restore it, stop services, move the newly prepared directory to a separate backup, then rename the saved backup to `snapshots` using that same one-off Node command pattern. Never delete the volume to simulate a cold start.
Overview now reports that no saved reports exist. Visit Settings and submit Prepare reports; reload Overview after preparation finishes. The printed backup is preserved, as are the SQLite queue, configuration and journals. To restore it, stop services, move the newly prepared directory to a separate backup, then rename the saved backup to `snapshots` using that same one-off Node command pattern. Never delete the volume to simulate a cold start.
Overview and Settings read the last recorded strict check outcome/time from immutable
`data/snapshots/.validation/` records, independently of complete financial sets and
their preparation time. Homepage loads never consult the queue or native journal;
Settings alone reads this deployment's queue state. Evidence-read failure does not
hide readable finances. Journal/request identities, storage paths and native diagnostics
remain private. Records are retained without cleanup or a history UI; no freshness
proof is inferred from their presence or timestamps.
Explicit preparation now runs standalone `checkJournal` before staging: a third native
process/strict parse before the existing two-process financial path. Completed exit 0
records Passed; completed normal nonzero records Could not validate and stops financial
work. Missing/unreadable input can also produce nonzero; it is not an invalid-accounting
diagnosis. Launch/abort/signal/stream/unexpected incomplete failures preserve earlier
evidence. Evidence-write failure stops preparation before financial work. The final
financial batch still independently uses `--strict` and its cached journal; the check
may have seen different edits. A recorded pass is not a completed financial set.
Failure before the financial directory commit keeps the previous complete set and
recorded pass. An inspection-only `current` alias failure after commit may fail the
job with the new valid complete set still selectable. No rollback or deletion follows;
reload saved reports to inspect the available result. The read-only `worker:inspect`
command does not write validation evidence.
Prepared files identify their configured journal privately so reports from a known
different journal are not shown. The preparation timestamp describes saved data,
docs/philosophy.md
+22 -6
View File
@@ -6,12 +6,17 @@ clear inputs and outputs, not scattered across its consumers.
## Read, request, produce
- **Browsing reads saved reports.** If none exist, say so. Do not prepare, inspect
journals, infer freshness or query job status to render Overview and Transactions.
- **Users request work.** The preparation page submits a job explicitly. Live queries
are a separate explicit operation. Startup and journal edits do not prepare reports.
- **Jobs produce data.** The worker calls hledger and saves a complete result. A page
navigation or reload observes saved data; open pages do not update themselves.
- **Browsing reads saved files.** If no complete financial set exists, say so.
Overview reads last recorded validation outcome/time independently of finances;
neither proves current-input freshness. Do not inspect journals, query job status
or request work to render saved financial views.
- **Users request work.** Settings (`/settings`) owns inline preparation/status and
links to the existing `/queries` tool. `/prepare` is removed without an alias.
Live queries are a separate explicit operation. Startup and journal edits do not
prepare reports.
- **Jobs produce data.** The worker records a completed standalone strict check
before financial work, then publishes a complete result only after a pass.
A page navigation or reload observes saved data; open pages do not update themselves.
## Ownership before abstraction
@@ -21,6 +26,17 @@ through jobs; version checking and explicit journal inspection are operational
exceptions. Shared job code defines requests and owns queue access; web code requests
work; workers execute it; storage persists reports; web presents saved data. Keep
those responsibilities separate without wrapping the SDK in another job framework.
A completed exit-0 preflight records Passed; a normal nonzero records Could not
validate, not a diagnosis of invalid accounting. Incomplete checks preserve earlier
evidence. The extra check adds a third parse; financial production remains independently
strict and uses its own cached journal. Read-only inspection does not write evidence.
Validation records are immutable operational artifacts outside financial sets, retained
without cleanup or a history product. Only outcome/time is public; journal/request
identity and storage provenance remain private. A recorded pass is not a completed
financial publication or proof of identical inputs. Before financial commit, failure
retains the previous complete set; after commit, inspection-alias failure can fail the
job while leaving a new valid selectable set. Do not roll back or delete that set.
If a consumer compensates for an awkward output, improve the producer contract
before extracting that compensation into more helpers.
packages/hledger/README.md
+29 -24
View File
@@ -4,8 +4,8 @@ Hledger library, pinned to **1.99.5**. It reads journal data and organizes neutr
- `prepareReports(path, { writeRows, signal? })` calculates Overview and all account activity by running hledger for the manual preparation job. Unlike storage's `readReportPublication`, it prepares new data rather than reading a saved report.
- `getHledgerVersion()` verifies the inspected engine version.
- `checkJournal(path)` runs read-only strict checking with ambient hledger config disabled.
- `runCostBalance(path, expression, asOf)` runs one bounded native `balance --strict --cost --flat` expression against the supplied journal path and its current includes. Native strict validation happens on every execution, not only at worker startup. It freezes relative-date context, uses an inclusive as-of cutoff, validates the returned structure, and preserves reported quantities as strings. It does not perform market valuation or implicit FX conversion. Runtime/output/row limits fail rather than truncate; they do not establish journal capacity or a native-memory budget.
- `checkJournal(path): Promise<void>` runs read-only strict checking with ambient hledger config disabled. It returns only after complete consumption, normal exit 0 and cleanup; sanitized `JournalCheckError.kind` distinguishes `nonzero-exit` from `not-completed`. A normal nonzero supports **Could not validate**, not an invalid-accounting diagnosis: input may be missing, unreadable or malformed. Launch/abort/signal/stream/unexpected errors mean incomplete execution.
- `runCostBalance(path, expression, asOf)` runs one native `balance --strict --cost --flat` expression against the supplied journal path and its current includes. Native strict validation happens on every execution, not only at worker startup. It freezes relative-date context, uses an inclusive as-of cutoff, validates the returned structure, and preserves reported quantities as strings. It does not perform market valuation or implicit FX conversion. There are no application-imposed native runtime/output/row caps; this is not a journal-capacity or native-memory guarantee.
- Exact JSON ingestion reads integer mantissas as BigInt from original JSON tokens,
never rounded JavaScript numbers. The existing codec fails closed on
scientific-notation mantissa tokens; exactness is not a universal claim for
@@ -35,7 +35,7 @@ src/
├── queries/
│ └── cost-balance.ts # Recorded-cost query
├── amounts.ts # Shared exact quantities and amount projection
├── report-stream.ts # Bounded incremental report framing
├── report-stream.ts # Incremental report framing
├── journal-stats.ts # Journal metadata interpretation
└── report-date.ts # Shared ISO report-date formatting
```
@@ -153,9 +153,16 @@ all native `accounts --find --directives` identity qualifications, and then all
full-history `aregister --historical --empty -Ojson` reports. Every published
financial value comes from this second loaded journal. Its anchor, inferred
currency and actual account list must match discovery. Both processes disable
ambient config. This is not an atomic filesystem snapshot. The caller controls
when to run a manual preparation job and owns final publication; browsing saved
reports does not invoke preparation.
ambient config. The financial batch remains independently `--strict`. This is not
an atomic filesystem snapshot. The worker now calls standalone `checkJournal` before
this unchanged two-process financial path: **three native processes/parses** for an
explicit preparation. The check can observe different edits from financial production;
a passed check is not proof of identical inputs, freshness or a completed financial set.
The worker/storage, not this library, record completed outcomes; incomplete checks
preserve prior evidence. Read-only `worker:inspect` also uses `checkJournal` but never
writes evidence. The caller owns final publication; browsing saved reports does not
invoke preparation. Financial format remains **5**, with the existing six-month
Overview and full-history registers unchanged.
The account summaries contain only native used/declared accounts, sorted by literal name;
it does not expand implicit parents. An explicitly declared parent has native
@@ -239,39 +246,37 @@ explicit-zero and unnamed-commodity entry is preserved; empty bags stay empty.
Scientific-notation mantissas still fail closed.
Output is streamed with fatal UTF-8 decoding, strict incremental top-level row
array framing and callback backpressure. Only the bounded metadata/financial
prefix, one raw tuple and an internal chunk of at most 50 projected rows are retained,
array framing and callback backpressure. Only the metadata/financial prefix,
one raw tuple and an internal chunk of at most 50 projected rows are retained,
not the complete native JSON or an entire account history. Chunk boundaries are
not persisted pagination. `writeRows(accountName, rows)` is awaited before consuming more
rows, including during cancellation. Abort/deadline kills the native child
rows, including during cancellation. Abort kills the native child
immediately, but the active callback must settle before this facade returns or
throws; staging can then be discarded without a late write recreating files.
Callback I/O is caller-owned, so a stalled callback can extend call wall time
beyond the native execution deadline. **Writes must remain private/staged**: callbacks can run before a later
native error. Only successful EOF, zero exit and completed validation allow the
Callback I/O is caller-owned, so a stalled callback can delay completion and cleanup.
**Writes must remain private/staged**: callbacks can run before a later native error. Only successful EOF, zero exit and completed validation allow the
returned Overview/account summaries and staged rows to be published together. Caller code
owns cleanup and atomic publication; this library does not store snapshots.
Callback failure, cancellation, malformed/truncated/extra output and resource
failures kill and reap the child and return a generic error without diagnostics.
The combined path enforces these **execution limits**: a 180-second native execution budget
(discovery itself at most 30 seconds/1 MiB), 1 GiB aggregate streamed native batch
stdout, a 1-MiB metadata/financial prefix and a 4-MiB physical raw tuple. These are
not capacity claims or a 1-GiB native-memory ceiling; hledger's own rendering
memory still matters and requires production profiling. There are no separately
There are no application-imposed native execution deadlines, stdout/prefix/tuple/row
caps or saved-file size caps. Sequential callback chunks of at most 50 rows are
backpressure, not capacity limits. Hledger rendering and Node parsing can still use
substantial memory; no journal-capacity guarantee is implied. There are no separately
exported Overview or account-register execution APIs.
## One hledger process boundary
`process.ts` is the only production module that launches hledger. It owns
argument-array execution, stdout limits, monotonic deadlines, SIGKILL cancellation,
backpressure, EOF/exit validation and cleanup. There is no shell command builder,
argument-array execution, SIGKILL cancellation, backpressure, EOF/exit validation
and cleanup. There is no shell command builder,
provider registry or configurable executable. Native stderr and nested process
errors never become public diagnostics.
Its private `readHledger` buffers small responses with fatal UTF-8 decoding, defaulting
to 30 seconds and 1 MiB. The prepared batch consumes bounded byte chunks through
`consumeHledger`; both use the same process lifecycle. There is no separate buffered
Its private `readHledger` buffers responses with fatal UTF-8 decoding. The prepared
batch consumes byte chunks through `consumeHledger`; both use the same process
lifecycle, without application time/output caps. There is no separate buffered
execution implementation. Consumers cannot return a successful partial read before
EOF or successful process exit. Cancellation kills native work immediately but waits
for active caller-owned writes before returning, so cleanup cannot race a late write.
@@ -285,7 +290,7 @@ not a reason to expose a generic CLI API now.
`amounts.ts` owns the exact numeric codec and common reported-amount shape/projection.
Projection preserves order, duplicate commodities and exact quantity strings. Query
response limits and cost-annotation restrictions stay in the cost-balance operation.
input validation and cost-annotation restrictions stay in the cost-balance operation.
`transactions/account-rows.ts` owns the private hledger register tuple;
`organizeAccountRow` produces `PreparedAccountRow` with account-relative facts
and canonical posting selection; `transactions/transaction.ts` owns full
@@ -295,6 +300,6 @@ Tests cover the pure financial plan/decoding/organization and the actual combine
producer. Independent native fixture oracles remain test-only; removed standalone
Overview/register execution APIs are not kept alive just to support tests. Process
safety tests live next to `process.ts`. The richer record does not change the
shared process/stream protocol, commands, callback backpressure or resource limits.
financial stream protocol, commands or callback backpressure.
Development uses source exports and the root Node loader; production uses compiled `dist/src`. `pnpm build`, `check`, `lint`, and `test` operate on this package.
packages/hledger/src/checks.test.ts
+59 -9
View File
@@ -1,8 +1,17 @@
import { mkdtemp, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { describe, expect, it, vi } from 'vitest';
import { checkJournal, getHledgerVersion } from './checks.js';
import { readHledger } from './process.js';
import { checkJournal, getHledgerVersion, JournalCheckError } from './checks.js';
import { HledgerExecutionError, readHledger } from './process.js';
vi.mock('./process.js', () => ({ readHledger: vi.fn() }));
vi.mock('./process.js', async (importOriginal) => ({
...(await importOriginal<typeof import('./process.js')>()),
readHledger: vi.fn()
}));
const safeCheckMessage =
'hledger strict journal validation failed. Run hledger check --strict directly for diagnostics.';
describe('read-only hledger boundary', () => {
it('uses the shared native reader by default, including empty check output', async () => {
@@ -51,16 +60,57 @@ describe('read-only hledger boundary', () => {
});
});
it('does not expose native journal diagnostics in operational error messages', async () => {
await expect(
checkJournal('/journal', async () => {
throw new Error('SECRET PAYEE AND BALANCE');
})
).rejects.toThrow(/^hledger strict journal validation failed\./);
it.each(['nonzero-exit', 'not-completed'] as const)(
'exposes only the safe check classification for %s',
async (kind) => {
const error = await checkJournal('/private/journal', async () => {
throw new HledgerExecutionError(kind);
}).catch((error: unknown) => error);
expect(error).toBeInstanceOf(JournalCheckError);
expect(error).toMatchObject({ message: safeCheckMessage, kind });
expect((error as Error).cause).toBeUndefined();
}
);
it('treats unknown errors as incomplete without exposing private diagnostics', async () => {
const error = await checkJournal('/journal', async () => {
throw Object.assign(new Error('SECRET PAYEE AND BALANCE'), { kind: 'nonzero-exit' });
}).catch((error: unknown) => error);
expect(error).toBeInstanceOf(JournalCheckError);
expect(error).toMatchObject({ message: safeCheckMessage, kind: 'not-completed' });
expect((error as Error).cause).toBeUndefined();
await expect(
getHledgerVersion(async () => {
throw new Error('private runtime details');
})
).rejects.toThrow(/^Unable to execute hledger\./);
});
it('classifies real strict rejection and missing input as could-not-validate, not invalid accounting', async () => {
const native = await vi.importActual<typeof import('./process.js')>('./process.js');
const directory = await mkdtemp(join(tmpdir(), 'hledger-check-'));
const journal = join(directory, 'fictional.journal');
try {
const input = `commodity USD
account assets:cash
account equity:opening
2025-01-01 Fictional opening
assets:cash 1 USD
equity:opening -1 USD
`;
await writeFile(journal, input);
await expect(checkJournal(journal, native.readHledger)).resolves.toBeUndefined();
await writeFile(journal, input.replace('-1 USD', '-2 USD'));
await expect(checkJournal(journal, native.readHledger)).rejects.toMatchObject({
message: safeCheckMessage,
kind: 'nonzero-exit'
});
await expect(
checkJournal(join(directory, 'missing.journal'), native.readHledger)
).rejects.toMatchObject({ message: safeCheckMessage, kind: 'nonzero-exit' });
} finally {
await rm(directory, { recursive: true, force: true });
}
});
});
packages/hledger/src/checks.ts
+16 -4
View File
@@ -1,4 +1,4 @@
import { readHledger, type CommandRunner } from './process.js';
import { HledgerExecutionError, readHledger, type CommandRunner } from './process.js';
export const supportedHledgerVersion = '1.99.5';
export async function getHledgerVersion(run: CommandRunner = readHledger): Promise<string> {
@@ -14,17 +14,29 @@ export async function getHledgerVersion(run: CommandRunner = readHledger): Promi
return text;
}
/** Safe check outcome, not a diagnosis of invalid accounting. */
export class JournalCheckError extends Error {
readonly kind: HledgerExecutionError['kind'];
constructor(kind: HledgerExecutionError['kind']) {
super(
'hledger strict journal validation failed. Run hledger check --strict directly for diagnostics.'
);
this.kind = kind;
}
}
export async function checkJournal(
journal: string,
run: CommandRunner = readHledger
): Promise<void> {
try {
await run(['--no-conf', '--lang=en', '-f', journal, 'check', '--strict']);
} catch {
} catch (error) {
// Native diagnostics can include complete postings and amounts. Do not put
// them into operational logs; the operator can run hledger directly.
throw new Error(
'hledger strict journal validation failed. Run hledger check --strict directly for diagnostics.'
throw new JournalCheckError(
error instanceof HledgerExecutionError ? error.kind : 'not-completed'
);
}
}
packages/hledger/src/index.test.ts
-13
View File
@@ -1,13 +0,0 @@
import { expect, it } from 'vitest';
import * as hledger from './index.js';
it('exposes the report operations and operational checks used by the apps, not native plumbing', () => {
expect(Object.keys(hledger).sort()).toEqual([
'checkJournal',
'getHledgerVersion',
'prepareReports',
'runCostBalance',
'runHoldings',
'supportedHledgerVersion'
]);
});
packages/hledger/src/index.ts
+6 -1
View File
@@ -1,4 +1,9 @@
export { checkJournal, getHledgerVersion, supportedHledgerVersion } from './checks.js';
export {
checkJournal,
getHledgerVersion,
JournalCheckError,
supportedHledgerVersion
} from './checks.js';
export type { OverviewData } from './overview/report.js';
export { runCostBalance } from './queries/cost-balance.js';
export type { CostBalanceReport } from './queries/cost-balance.js';
packages/hledger/src/process.test.ts
+59 -12
View File
@@ -2,7 +2,7 @@ import type { ChildProcess, SpawnOptions } from 'node:child_process';
import { spawn } from 'node:child_process';
import { getEventListeners } from 'node:events';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { consumeHledger, readHledger } from './process.js';
import { consumeHledger, HledgerExecutionError, readHledger } from './process.js';
const owned = vi.hoisted(() => ({
script: '',
@@ -28,6 +28,7 @@ vi.mock('node:child_process', async (importOriginal) => {
});
const safeError = 'Unable to complete hledger execution.';
const notCompleted = { message: safeError, kind: 'not-completed' };
const delay = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));
function gate() {
let resolve!: () => void;
@@ -64,7 +65,10 @@ describe('owned native process', () => {
it('rejects a nonzero exit even after valid complete output', async () => {
owned.script = 'process.stdout.write("valid"); process.exitCode = 7';
await expect(readHledger([])).rejects.toThrow(safeError);
const error = await readHledger([]).catch((error: unknown) => error);
expect(error).toBeInstanceOf(HledgerExecutionError);
expect(error).toMatchObject({ message: safeError, kind: 'nonzero-exit' });
expect((error as Error).cause).toBeUndefined();
expect(owned.children[0].exitCode).toBe(7);
expect(owned.closed.has(owned.children[0])).toBe(true);
});
@@ -74,7 +78,8 @@ describe('owned native process', () => {
throw new Error('/secret/journal spawn details');
});
const error = await readHledger([]).catch((error: unknown) => error);
expect(error).toEqual(new Error(safeError));
expect(error).toBeInstanceOf(HledgerExecutionError);
expect(error).toMatchObject(notCompleted);
expect((error as Error).cause).toBeUndefined();
});
@@ -89,14 +94,16 @@ describe('owned native process', () => {
child.once('close', () => owned.closed.add(child));
return child;
});
await expect(readHledger([])).rejects.toThrow(safeError);
await expect(readHledger([])).rejects.toMatchObject(notCompleted);
expect(owned.closed.has(owned.children[0])).toBe(true);
});
it('does not launch a pre-aborted request', async () => {
const controller = new AbortController();
controller.abort(new Error('private abort reason'));
await expect(readHledger([], { signal: controller.signal })).rejects.toThrow(safeError);
await expect(readHledger([], { signal: controller.signal })).rejects.toMatchObject(
notCompleted
);
expect(spawn).not.toHaveBeenCalled();
});
@@ -104,7 +111,7 @@ describe('owned native process', () => {
owned.script = 'setInterval(() => {}, 1000)';
const controller = new AbortController();
const result = readHledger([], { signal: controller.signal });
const rejected = expect(result).rejects.toThrow(safeError);
const rejected = expect(result).rejects.toMatchObject(notCompleted);
controller.abort();
await rejected;
expect(owned.children[0].signalCode).toBe('SIGKILL');
@@ -126,7 +133,7 @@ describe('owned native process', () => {
}).finally(() => {
settled = true;
});
const rejected = expect(result).rejects.toThrow(safeError);
const rejected = expect(result).rejects.toMatchObject(notCompleted);
await ended.promise;
await delay(20);
expect(settled).toBe(false);
@@ -142,13 +149,15 @@ describe('owned native process', () => {
consumeHledger([], {}, async () => {
throw new Error('SECRET FINANCE');
})
).rejects.toThrow(safeError);
await expect(consumeHledger([], {}, async () => 'too early')).rejects.toThrow(safeError);
).rejects.toMatchObject(notCompleted);
await expect(consumeHledger([], {}, async () => 'too early')).rejects.toMatchObject(
notCompleted
);
await expect(
consumeHledger([], {}, async (output) => {
for await (const _chunk of output) break;
})
).rejects.toThrow(safeError);
).rejects.toMatchObject(notCompleted);
expect(owned.children.every((child) => owned.closed.has(child))).toBe(true);
expect(owned.children.every((child) => child.signalCode === 'SIGKILL')).toBe(true);
});
@@ -199,12 +208,50 @@ describe('owned native process', () => {
} finally {
release.resolve();
}
expect(await result).toEqual(new Error(safeError));
expect(await result).toMatchObject(notCompleted);
expect(callerFinished).toBe(true);
expect(getEventListeners(controller.signal, 'abort')).toHaveLength(0);
expect(owned.children[0].listenerCount('error')).toBe(0);
});
it('does not classify signal termination as a completed nonzero exit', async () => {
owned.script = 'process.kill(process.pid, "SIGTERM")';
await expect(readHledger([])).rejects.toMatchObject(notCompleted);
expect(owned.children[0].signalCode).toBe('SIGTERM');
expect(owned.closed.has(owned.children[0])).toBe(true);
});
it('does not let a normal nonzero exit override consumer or stream failure', async () => {
owned.script = 'process.stdout.write("valid"); process.exitCode = 7';
await expect(
consumeHledger([], {}, async (output) => {
for await (const _chunk of output) {
// Consume all bytes before failing caller work.
}
const child = owned.children[0];
if (!owned.closed.has(child)) {
await new Promise<void>((resolve) => child.once('close', () => resolve()));
}
throw new HledgerExecutionError('nonzero-exit');
})
).rejects.toMatchObject(notCompleted);
await expect(
consumeHledger([], {}, async (output) => {
for await (const _chunk of output) {
// Wait for a normal close, then observe a stream error before returning.
}
const child = owned.children[1];
if (!owned.closed.has(child)) {
await new Promise<void>((resolve) => child.once('close', () => resolve()));
}
child.stdout!.emit('error', new Error('SECRET STREAM DETAILS'));
})
).rejects.toMatchObject(notCompleted);
expect(owned.children.every((child) => child.exitCode === 7)).toBe(true);
expect(owned.children.every((child) => owned.closed.has(child))).toBe(true);
});
it('removes cancellation listeners after successful consumption', async () => {
const controller = new AbortController();
await readHledger([], { signal: controller.signal });
@@ -234,7 +281,7 @@ describe('UTF-8 reads', () => {
it.each(['[0xff]', '[0xe2, 0x82]'])('rejects malformed UTF-8 %s', async (bytes) => {
owned.script = `process.stdout.write(Buffer.from(${bytes}))`;
await expect(readHledger([])).rejects.toThrow(safeError);
await expect(readHledger([])).rejects.toMatchObject(notCompleted);
expect(owned.closed.has(owned.children[0])).toBe(true);
});
});
packages/hledger/src/process.ts
+22 -6
View File
@@ -6,7 +6,17 @@ export interface ProcessOptions {
export type CommandRunner = (args: readonly string[]) => Promise<string>;
const failure = () => new Error('Unable to complete hledger execution.');
/** Internal lifecycle result; never includes native diagnostics or a private cause. */
export class HledgerExecutionError extends Error {
readonly kind: 'nonzero-exit' | 'not-completed';
constructor(kind: 'nonzero-exit' | 'not-completed') {
super('Unable to complete hledger execution.');
this.kind = kind;
}
}
const failure = () => new HledgerExecutionError('not-completed');
/** Own the child until both the consumer's work and process cleanup have settled. */
export async function consumeHledger<T>(
@@ -30,6 +40,7 @@ export async function consumeHledger<T>(
let eof = false;
let didClose = false;
let exitCode: number | null = null;
let exitSignal: NodeJS.Signals | null = null;
const stop = () => {
failed = true;
@@ -49,9 +60,10 @@ export async function consumeHledger<T>(
}
};
const closed = new Promise<void>((resolve) => {
child.once('close', (code) => {
child.once('close', (code, signal) => {
didClose = true;
exitCode = code;
exitSignal = signal;
resolve();
});
});
@@ -69,16 +81,16 @@ export async function consumeHledger<T>(
eof = true;
}
const output = streamedOutput();
let result: T;
try {
check();
// Deliberately not raced with cancellation: callers may be writing artifacts.
const result = await consume(output, check);
result = await consume(output, check);
check();
if (!eof) throw failure();
await closed;
check();
if (exitCode !== 0) throw failure();
return result;
if (exitSignal !== null || typeof exitCode !== 'number') throw failure();
} catch {
stop();
throw failure();
@@ -87,13 +99,17 @@ export async function consumeHledger<T>(
try {
await output.return(undefined);
} catch {
// The public boundary always reports the same safe error.
failed = true;
}
await closed;
signal?.removeEventListener('abort', stop);
child.removeListener('error', stop);
stdout.removeListener('error', stop);
}
// A completed nonzero result is meaningful only after consumption and cleanup.
check();
if (exitCode !== 0) throw new HledgerExecutionError('nonzero-exit');
return result;
}
export async function readHledger(
packages/snapshots/README.md
+57 -12
View File
@@ -1,8 +1,8 @@
# @hledger-reports/snapshots
Server-side saved-report storage, independent of native execution and either
application. Browsing reads saved reports only; an absent/empty store returns
`null`, as does a store whose latest completed set has an unsupported format. Preparation is explicitly requested by the separate `/prepare` form and
application. Browsing reads saved reports and independent validation evidence only; an absent/empty store returns
`null`, as does a store whose latest completed set has an unsupported format. Preparation is explicitly requested by the inline form in `/settings` and
performed by a worker. Store reads never open journals, run hledger, enqueue work,
watch sources, hash inputs, reconcile state or assess freshness.
@@ -12,6 +12,8 @@ watch sources, hash inputs, reconcile state or assess freshness.
- `readOverview(snapshotDirectory): Promise<OverviewData | null>`
- `createReportStage(snapshotDirectory, run, previousAccounts?): Promise<ReportStage>`
- `readAccountTransactions(publication, id): Promise<{ id, name, transactions }>`
- `writeValidation(snapshotDirectory, run, journal, result): Promise<void>`
- `readValidation(snapshotDirectory): Promise<ValidationRecord | null>`
Pass the snapshots directory. `run` contains `{ id, createdAt: Date }`.
A stage streams contiguous account batches through
@@ -33,7 +35,7 @@ Each execution owns a unique hidden stage. One directory rename commits the
**entire** immutable completed set under `.sets/`. Each set contains:
```text
overview.json # { format: 4, journal, overview, accounts }
overview.json # { format: 5, journal, overview, accounts }
transactions/<8-character-id>.json # { id, name, transactions }
```
@@ -59,7 +61,7 @@ by **exact full native name** from the previous catalog. Pass only
checked against all previous and newly assigned IDs. Names remain unchanged data,
never filesystem paths. There is no separate registry or account-page index.
Every account file contains **all** its rows, provisionally limited to 64 MiB.
Every account file contains **all** its rows, with no application-imposed file-size cap.
Each `PreparedAccountRow` retains the library-owned original transaction, postings, comments,
ordered tags, exact amounts and reported metadata alongside account-relative
facts (`accountDate`, `otherAccounts`, `isSplit`, `change`, `balance`) and selected
@@ -68,23 +70,63 @@ that prepared set, not durable identities across edits. Transaction source posit
and assertion source positions remain **private** in these files; callers must
explicitly project browser-safe fields, never serialize whole stored rows.
Readers bound the read, validate the catalog/header/count and required transaction,
postings and posting-index containers, and return an ordinary array; callers paginate
Readers check regular files and stable byte length, validate the catalog/header/count
and required transaction, postings and posting-index containers, and return an ordinary array; callers paginate
after reading. Financial contents and exact quantity strings (including nested
amounts) remain producer-owned and opaque. This is the detail exported by hledger's
account register, **not a verbatim journal archive or a complete investment
cost/basis record**: annotations stripped by hledger cannot be recovered here.
Ordinary JSON serialization rejects BigInt/cycles. Storage neither computes report
dates nor copies input journals. Richer records do not raise the 64 MiB bound.
dates nor copies input journals. Whole-file reads still require sufficient Node memory;
no capacity guarantee is implied.
## Recorded validation
Validation evidence is operational, independent of financial format **5**. The worker
writes after standalone strict checking completes, before financial staging. The
store does not run checks or infer outcomes. `JournalValidationResult` contains
`{ outcome: 'passed' | 'could-not-validate', completedAt: string }`; `ValidationRecord`
contains `{ format: 1, journal, request: { id, createdAt }, result }`.
Each record is immutable:
```text
data/snapshots/.validation/<request-ms>-<run-id>-<completed-ms>-<uuid>.json
```
Both millisecond fields are padded to 16 digits. Selection uses the financial
request creation key including its trailing `/` separator, then completion time
and UUID within the same request. A recovered run can record another check; a delayed
older request cannot supersede a newer request. These are ordering rules, not global
clock or stale-worker fencing guarantees.
Writes use an exclusive private temporary file, close it, then rename it to its
unique final name. Temporary names are ignored. Reads select one record and validate
version, outcome, timestamps and request/completion identity against its filename.
Absent/unsupported evidence returns `null`; malformed or unreadable selected evidence
fails safely. Neither case falls back to an older Passed record. The web compares the
selected journal exactly, withholding other-journal evidence without searching backward,
and serializes only outcome/time. Journal/request identity and paths remain private.
Records are retained without cleanup or a history API/UI; there is no mutable latest
pointer, input hash, freshness proof or financial-data duplication. Atomic rename
prevents partial selection, not power loss; an unwritten observation can be lost. Check
completion and financial preparation times can describe different requests/inputs.
An incomplete check preserves prior evidence; evidence-write failure stops the worker
before financial work. A recorded pass survives later financial failure, including
post-commit alias failure. Evidence-read failure does not hide readable financial files.
The separate read-only native inspection command never writes these records.
## Current format and preservation
Only format-4 envelopes `{ format: 4, journal, overview, accounts }` are read.
Format 3's table-only rows are unsupported; explicitly **Prepare reports again**
to obtain rich records. There is no legacy reader or record adapter.
Only format-5 envelopes `{ format: 5, journal, overview, accounts }` are read.
Financial format remains **5**; validation evidence does not change it or require
re-preparing supported sets. Format 4 and older shapes, including format 3's table-only
rows, are unsupported; explicitly **Prepare reports again** in Settings to create
a supported set. There is no legacy reader or record adapter.
No completed set, or an unsupported format in the latest completed set, returns
`null`: there is no fallback to an older supported set. Callers can show
“No prepared reports available.” with a manual Prepare link. Store reads never
“No prepared reports available.” with a Settings link for manual preparation. Store reads never
prepare automatically.
Malformed JSON or invalid current publications fail with a generic read error.
@@ -104,7 +146,10 @@ Directories are private **0700**, files **0600**. Final directory/file no-follow
checks reject symlinks; path ancestors must still be trusted, not concurrently
replaced by hostile writers. Errors omit private paths and data. Failed stages
clean only their own hidden directories after closing file handles. Completed sets
are never discarded, including after post-commit alias failures. All successful
are never discarded, including after post-commit alias failures. Failure before the
directory rename retains the previous complete set; failure updating `current` after that commit may
fail the job while the new valid complete set remains selectable. There is no rollback.
Validation writes neither commit nor roll back financial data. All successful
sets and orphaned completions are retained; there is no garbage collection,
migration, history UI or user-data deletion.
packages/snapshots/src/files.ts
+63
View File
@@ -0,0 +1,63 @@
import { constants } from 'node:fs';
import { mkdir, open, readdir } from 'node:fs/promises';
import { join } from 'node:path';
export async function privateDirectory(directory: string, create = false): Promise<void> {
if (create) await mkdir(directory, { recursive: true, mode: 0o700 });
const folder = await open(
directory,
constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW
);
try {
if (create) await folder.chmod(0o700);
} finally {
await folder.close();
}
}
export async function namesIn(directory: string): Promise<string[]> {
try {
await privateDirectory(directory);
return await readdir(directory);
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return [];
throw error;
}
}
export async function readJson(directory: string, name: string): Promise<unknown> {
const file = await open(
join(directory, name),
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK
);
try {
const stat = await file.stat();
if (!stat.isFile()) throw new Error();
// One extra byte detects growth; require exactly the size we inspected.
const buffer = Buffer.alloc(stat.size + 1);
let length = 0;
while (length < buffer.length) {
const { bytesRead } = await file.read(buffer, length, buffer.length - length, null);
if (!bytesRead) break;
length += bytesRead;
}
if (length !== stat.size) throw new Error();
return JSON.parse(buffer.toString('utf8', 0, length));
} finally {
await file.close();
}
}
export async function writeJson(directory: string, name: string, value: unknown): Promise<void> {
const file = await open(
join(directory, name),
constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW,
0o600
);
try {
const json = JSON.stringify(value, null, 2);
await file.writeFile(json, 'utf8');
} finally {
await file.close();
}
}
packages/snapshots/src/index.ts
+2
View File
@@ -5,3 +5,5 @@ export {
readAccountTransactions
} from './reports.js';
export type { ReportPublication, StoredAccount, ReportStage } from './reports.js';
export { writeValidation, readValidation } from './validation.js';
export type { JournalValidationResult, ValidationRecord } from './validation.js';
packages/snapshots/src/reports.test.ts
-9
View File
@@ -1135,12 +1135,3 @@ it.each(['', ' ', null, undefined, 42])(
expect(await fs.readdir(join(snapshots, '.sets'))).toEqual([basename(pinned.directory)]);
}
);
it('exports only saved-report storage operations', async () => {
expect(Object.keys(await import('./index.js')).sort()).toEqual([
'createReportStage',
'readAccountTransactions',
'readOverview',
'readReportPublication'
]);
});
packages/snapshots/src/reports.ts
+2 -61
View File
@@ -5,9 +5,10 @@ import type {
} from '@hledger-reports/hledger';
import { randomInt, randomUUID } from 'node:crypto';
import { constants } from 'node:fs';
import { mkdir, open, readdir, rename, rm, symlink, unlink } from 'node:fs/promises';
import { mkdir, open, rename, rm, symlink, unlink } from 'node:fs/promises';
import type { FileHandle } from 'node:fs/promises';
import { basename, dirname, join } from 'node:path';
import { namesIn, privateDirectory, readJson, writeJson } from './files.js';
const runIdPattern = /^[A-Za-z0-9_-]{1,128}(?![\s\S])/;
const setPattern =
@@ -131,66 +132,6 @@ function isCatalog(value: unknown): value is StoredAccount[] {
return true;
}
async function privateDirectory(directory: string, create = false): Promise<void> {
if (create) await mkdir(directory, { recursive: true, mode: 0o700 });
const folder = await open(
directory,
constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW
);
try {
if (create) await folder.chmod(0o700);
} finally {
await folder.close();
}
}
async function namesIn(directory: string): Promise<string[]> {
try {
await privateDirectory(directory);
return await readdir(directory);
} catch (error) {
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return [];
throw error;
}
}
async function readJson(directory: string, name: string): Promise<unknown> {
const file = await open(
join(directory, name),
constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK
);
try {
const stat = await file.stat();
if (!stat.isFile()) throw new Error();
// One extra byte detects growth; require exactly the size we inspected.
const buffer = Buffer.alloc(stat.size + 1);
let length = 0;
while (length < buffer.length) {
const { bytesRead } = await file.read(buffer, length, buffer.length - length, null);
if (!bytesRead) break;
length += bytesRead;
}
if (length !== stat.size) throw new Error();
return JSON.parse(buffer.toString('utf8', 0, length));
} finally {
await file.close();
}
}
async function writeJson(directory: string, name: string, value: unknown): Promise<void> {
const file = await open(
join(directory, name),
constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW,
0o600
);
try {
const json = JSON.stringify(value, null, 2);
await file.writeFile(json, 'utf8');
} finally {
await file.close();
}
}
function requestCreationKey(token: string): string {
// The separator in latestSet preserves request ordering for prefix run IDs.
return setPattern.exec(token)![1];
packages/snapshots/src/validation.test.ts
+220
View File
@@ -0,0 +1,220 @@
import * as crypto from 'node:crypto';
import * as fs from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { afterEach, beforeEach, expect, it, vi } from 'vitest';
import { readValidation, writeValidation } from './index.js';
import type { JournalValidationResult, ValidationRecord } from './index.js';
vi.mock('node:fs/promises', async (original) => ({
...(await original<typeof import('node:fs/promises')>())
}));
vi.mock('node:crypto', async (original) => ({
...(await original<typeof import('node:crypto')>())
}));
let root: string;
let snapshots: string;
let directory: string;
const older = { id: 'older', createdAt: new Date('2026-05-01T00:00:00.000Z') };
const newer = { id: 'newer', createdAt: new Date('2026-05-02T00:00:00.000Z') };
const journal = '/private/books/main.journal';
const passed: JournalValidationResult = {
outcome: 'passed',
completedAt: '2026-05-03T00:00:00.000Z'
};
const failed: JournalValidationResult = {
outcome: 'could-not-validate',
completedAt: '2026-05-04T00:00:00.000Z'
};
const safeError = (message: string) => (error: Error) =>
error.message === message && error.cause === undefined;
const expected = (run = older, result = passed, identity = journal): ValidationRecord => ({
format: 1,
journal: identity,
request: { id: run.id, createdAt: run.createdAt.toISOString() },
result
});
async function pathFor(run = older): Promise<string> {
const name = (await fs.readdir(directory)).find((name) => name.includes(`-${run.id}-`));
if (!name) throw new Error('Missing test record');
return join(directory, name);
}
beforeEach(async () => {
root = await fs.mkdtemp(join(tmpdir(), 'validation-snapshot-'));
snapshots = join(root, 'snapshots');
directory = join(snapshots, '.validation');
});
afterEach(async () => {
vi.restoreAllMocks();
await fs.rm(root, { recursive: true, force: true });
});
it('round-trips private immutable evidence independently of financial storage', async () => {
expect(await writeValidation(snapshots, older, journal, passed)).toBeUndefined();
expect(await readValidation(snapshots)).toEqual(expected());
const oldPath = await pathFor();
const oldBytes = await fs.readFile(oldPath, 'utf8');
expect(JSON.parse(oldBytes)).toEqual(expected());
expect(await fs.readdir(directory)).toEqual([
expect.stringMatching(/^0001777593600000-older-0001777766400000-[0-9a-f-]{36}\.json$/)
]);
for (const folder of [snapshots, directory])
expect((await fs.stat(folder)).mode & 0o777).toBe(0o700);
expect((await fs.stat(oldPath)).mode & 0o777).toBe(0o600);
const rename = fs.rename;
vi.spyOn(fs, 'rename').mockImplementation(async (source, destination) => {
// The complete temporary JSON is not selected before its rename.
expect(String(source)).toMatch(/\.tmp$/);
expect(JSON.parse(await fs.readFile(source, 'utf8'))).toEqual(
expected(newer, failed, '/private/books/other.journal')
);
expect(await readValidation(snapshots)).toEqual(expected());
await rename(source, destination);
});
await writeValidation(snapshots, newer, '/private/books/other.journal', failed);
// Journal matching belongs to the server, not this private reader.
expect(await readValidation(snapshots)).toEqual(
expected(newer, failed, '/private/books/other.journal')
);
expect(await fs.readFile(oldPath, 'utf8')).toBe(oldBytes);
expect(await fs.readdir(directory)).toHaveLength(2);
expect(await fs.readdir(snapshots)).toEqual(['.validation']);
});
it('returns null for missing evidence and ignores temporary or foreign names', async () => {
expect(await readValidation(snapshots)).toBeNull();
expect(await fs.readdir(root)).toEqual([]);
await fs.mkdir(directory, { recursive: true });
await fs.writeFile(join(directory, '.unfinished.tmp'), 'private partial JSON');
await fs.writeFile(join(directory, 'current.json'), 'private foreign JSON');
expect(await readValidation(snapshots)).toBeNull();
await writeValidation(snapshots, older, journal, passed);
expect(await readValidation(snapshots)).toEqual(expected());
});
it('selects request creation before completion time or delayed write order', async () => {
await writeValidation(snapshots, newer, journal, passed);
await writeValidation(snapshots, older, journal, failed);
expect(await readValidation(snapshots)).toEqual(expected(newer));
});
it('preserves trailing-separator request ordering for prefix run IDs', async () => {
const prefix = { ...newer, id: 'a' };
const extension = { ...newer, id: 'a-z' };
await writeValidation(snapshots, prefix, journal, passed);
await writeValidation(snapshots, extension, journal, failed);
expect(await readValidation(snapshots)).toEqual(expected(prefix));
});
it('selects same-request recovery by completion time, then UUID for ties', async () => {
const low = '00000000-0000-4000-8000-000000000000';
const high = 'ffffffff-ffff-4fff-bfff-ffffffffffff';
vi.spyOn(crypto, 'randomUUID').mockReturnValueOnce(low).mockReturnValueOnce(high);
await writeValidation(snapshots, older, journal, failed);
await writeValidation(snapshots, older, journal, passed);
expect(await readValidation(snapshots)).toEqual(expected(older, failed));
vi.spyOn(crypto, 'randomUUID').mockReturnValueOnce(high);
await writeValidation(snapshots, older, journal, {
...passed,
completedAt: failed.completedAt
});
expect(await readValidation(snapshots)).toEqual(
expected(older, { ...passed, completedAt: failed.completedAt })
);
expect(await fs.readdir(directory)).toHaveLength(3);
});
it('does not fall back from unsupported selected evidence or alter it', async () => {
await writeValidation(snapshots, older, journal, passed);
await writeValidation(snapshots, newer, journal, failed);
const path = await pathFor(newer);
const raw = JSON.stringify({ format: 2, private: 'unsupported evidence' });
await fs.writeFile(path, raw);
expect(await readValidation(snapshots)).toBeNull();
expect(await fs.readFile(path, 'utf8')).toBe(raw);
});
it('fails safely on corrupt selected JSON without reading an older pass', async () => {
await writeValidation(snapshots, older, journal, passed);
await writeValidation(snapshots, newer, journal, failed);
const path = await pathFor(newer);
await fs.writeFile(path, 'private broken JSON');
const open = fs.open;
const oldPath = await pathFor();
vi.spyOn(fs, 'open').mockImplementation(async (path, ...args) => {
expect(String(path)).not.toBe(oldPath);
return open(path, ...args);
});
await expect(readValidation(snapshots)).rejects.toSatisfy(safeError('Unable to read validation'));
});
it('validates current evidence fields and filename identity without an older-pass fallback', async () => {
await writeValidation(snapshots, older, journal, passed);
await writeValidation(snapshots, newer, journal, failed);
const path = await pathFor(newer);
const record = expected(newer, failed);
const malformed = [
{ ...record, journal: '' },
{ ...record, request: { ...record.request, id: 'different' } },
{ ...record, request: { ...record.request, createdAt: older.createdAt.toISOString() } },
{ ...record, result: { ...failed, outcome: 'invalid' } },
{ ...record, result: { ...failed, completedAt: '2026-02-30T00:00:00.000Z' } },
{ ...record, result: passed }
];
for (const value of malformed) {
await fs.writeFile(path, JSON.stringify(value));
await expect(readValidation(snapshots)).rejects.toSatisfy(
safeError('Unable to read validation')
);
}
});
it('keeps prior evidence and closes the exclusive temporary file after a write failure', async () => {
await writeValidation(snapshots, older, journal, passed);
const oldPath = await pathFor();
const oldBytes = await fs.readFile(oldPath, 'utf8');
const open = fs.open;
const closed = vi.fn();
vi.spyOn(fs, 'open').mockImplementation(async (path, ...args) => {
const file = await open(path, ...args);
if (String(path).endsWith('.tmp')) {
const close = file.close.bind(file);
vi.spyOn(file, 'close').mockImplementation(async () => {
closed();
await close();
});
vi.spyOn(file, 'writeFile').mockRejectedValue(new Error('private disk failure'));
}
return file;
});
const rename = vi.spyOn(fs, 'rename');
await expect(writeValidation(snapshots, newer, journal, failed)).rejects.toSatisfy(
safeError('Unable to write validation')
);
expect(closed).toHaveBeenCalledOnce();
expect(rename).not.toHaveBeenCalled();
expect(await readValidation(snapshots)).toEqual(expected());
expect(await fs.readFile(oldPath, 'utf8')).toBe(oldBytes);
});
it('rejects selected file and evidence-directory symlinks rather than following them', async () => {
await writeValidation(snapshots, older, journal, passed);
const path = await pathFor();
const savedFile = join(root, 'saved.json');
await fs.rename(path, savedFile);
await fs.symlink(savedFile, path);
await expect(readValidation(snapshots)).rejects.toSatisfy(safeError('Unable to read validation'));
await fs.unlink(path);
await fs.rename(savedFile, path);
const savedDirectory = join(root, 'saved-validation');
await fs.rename(directory, savedDirectory);
await fs.symlink(savedDirectory, directory);
await expect(readValidation(snapshots)).rejects.toSatisfy(safeError('Unable to read validation'));
await expect(writeValidation(snapshots, newer, journal, failed)).rejects.toSatisfy(
safeError('Unable to write validation')
);
});
packages/snapshots/src/validation.ts
+104
View File
@@ -0,0 +1,104 @@
import { randomUUID } from 'node:crypto';
import { rename } from 'node:fs/promises';
import { join } from 'node:path';
import { namesIn, privateDirectory, readJson, writeJson } from './files.js';
const recordPattern =
/^(\d{16})-([A-Za-z0-9_-]{1,128})-(\d{16})-([0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12})\.json(?![\s\S])/;
export interface JournalValidationResult {
outcome: 'passed' | 'could-not-validate';
completedAt: string;
}
/** Private operational evidence; consumers must withhold unrelated journal identities. */
export interface ValidationRecord {
format: 1;
journal: string;
request: { id: string; createdAt: string };
result: JournalValidationResult;
}
function isObject(value: unknown): value is Record<string, unknown> {
return value !== null && typeof value === 'object' && !Array.isArray(value);
}
function timestampKey(value: unknown): string | undefined {
if (typeof value !== 'string') return;
const date = new Date(value);
if (!Number.isFinite(date.getTime()) || date.toISOString() !== value) return;
const milliseconds = String(date.getTime()).padStart(16, '0');
return /^\d{16}$/.test(milliseconds) ? milliseconds : undefined;
}
function isRecord(value: unknown, identity: RegExpExecArray): value is ValidationRecord {
if (!isObject(value) || value.format !== 1) return false;
if (typeof value.journal !== 'string' || !value.journal.trim()) return false;
if (!isObject(value.request) || !isObject(value.result)) return false;
const [, createdKey, runId, completedKey] = identity;
if (value.request.id !== runId || timestampKey(value.request.createdAt) !== createdKey)
return false;
if (value.result.outcome !== 'passed' && value.result.outcome !== 'could-not-validate')
return false;
return timestampKey(value.result.completedAt) === completedKey;
}
function selectionKey(name: string): string {
const [, created, id, completed, uuid] = recordPattern.exec(name)!;
// Match financial request ordering, including the separator for prefix run IDs.
return `${created}-${id}/${completed}-${uuid}`;
}
export async function writeValidation(
snapshotDirectory: string,
run: { id: string; createdAt: Date },
journal: string,
result: JournalValidationResult
): Promise<void> {
try {
const record: ValidationRecord = {
format: 1,
journal,
request: { id: run.id, createdAt: run.createdAt.toISOString() },
result: { outcome: result.outcome, completedAt: result.completedAt }
};
const uuid = randomUUID();
const name = `${timestampKey(record.request.createdAt)}-${run.id}-${timestampKey(record.result.completedAt)}-${uuid}.json`;
const identity = recordPattern.exec(name);
if (!identity || !isRecord(record, identity)) throw new Error();
await privateDirectory(snapshotDirectory, true);
const directory = join(snapshotDirectory, '.validation');
await privateDirectory(directory, true);
// Failed or interrupted writes remain ignored temporary names, never selected evidence.
const temporary = `.${uuid}.tmp`;
await writeJson(directory, temporary, record);
await rename(join(directory, temporary), join(directory, name));
} catch {
throw new Error('Unable to write validation');
}
}
export async function readValidation(snapshotDirectory: string): Promise<ValidationRecord | null> {
try {
const directory = join(snapshotDirectory, '.validation');
const latest = (await namesIn(directory))
.filter((name) => recordPattern.test(name))
.sort((a, b) => {
const left = selectionKey(a);
const right = selectionKey(b);
if (left < right) return -1;
if (left > right) return 1;
return 0;
})
.at(-1);
if (!latest) return null;
const record = await readJson(directory, latest);
if (!isObject(record)) throw new Error();
if (record.format !== 1) return null;
if (!isRecord(record, recordPattern.exec(latest)!)) throw new Error();
return record;
} catch {
throw new Error('Unable to read validation');
}
}