overview digest #51

Merged

eleith merged 40 commits from gemini/feat/overview-digest into main 2025-08-23 16:02:50 +00:00

Conversation 0 Commits 40 Files Changed 45

+2345 -1696

eleith commented 2025-08-22 04:35:24 +00:00

Owner

Copy link

Copy source

• make one call to retrieve ALL unread entries (as opposed to a fetch PER category)
• email all digests in one email (as opposed to one email per digest)

- make one call to retrieve ALL unread entries (as opposed to a fetch PER category) - email all digests in one email (as opposed to one email per digest)

eleith added 34 commits 2025-08-22 04:35:24 +00:00

feat: Implement core overview digest data flow ... da9056298b

Refactors data fetching, grouping, and initial processing for the new overview digest.
Updates configuration, main application loop, and preview script.
Adjusts related tests for compatibility.

refactor: break circular dependency between digest and llm ... d869e22c6e

This commit refactors the code to break a circular dependency between the `digest` and `llm` packages.

The `digest` package was responsible for both creating the LLM prompt and parsing the response, while also depending on the `llm` package to make the actual API call. This created a tight coupling between the two packages.

This commit introduces a new `DigestLLMService` in the `digest` package, which is responsible for all LLM-related logic within the digest context. The `llm` package now provides a more generic `LLMService` that is only responsible for making the API call.

This change improves the separation of concerns and makes the code more modular and easier to maintain.

refactor: break circular dependency between digest and llm packages 91d117ab11

refactor: restructure llm package 72ec13f7a9

feat: Implement entry sorting based on config 060480c824

feat: Update MockDigestService for sortBy ... 6efef9b15d

This commit updates the MockDigestService in internal/testutil/mocks.go to reflect the change in the BuildDigestData signature, which now includes the sortBy parameter. This ensures consistency between the mock and the actual interface, and allows for proper testing of the sorting functionality.

refactor: Make getHTML in archive package reusable ... 71dbcb3b9d

This commit refactors the getHTML function in internal/archive/archive.go to accept an html/template.Template as an argument. This makes the function reusable for different HTML templates, such as the overview digest and grouped entries.

feat: Implement HTML generation for overview and grouped entries ... 79d22b93da

This commit implements the HTML generation for the overview digest and grouped entries.

- Renamed internal/templates/entries.gohtml to internal/templates/grouped-entries.gohtml.
- Created internal/templates/overview.gohtml.
- Made getHTML function in internal/archive/archive.go reusable.
- Modified internal/archive/archive.go to create daily archive directories and render HTML files for both the overview and grouped entries.

feat: Implement email functionality for overview and grouped entries ... b8d169832e

This commit implements the email functionality for the overview digest and grouped entries.

- Modified internal/email/email.go to send one email with the Overview Digest as the body and attach all Group Digest HTML files.
- Updated internal/app/interfaces.go and internal/testutil/mocks.go to reflect the changes in the Send function signature.
- Updated cmd/miniflux-digest/main.go and internal/processor/processor.go to reflect the changes in the Send function signature and to pass the data to the Send function.

feat: Update preview script to use ArchiveService for HTML generation ... 375622efb1

This commit updates the scripts/preview/main.go to use the ArchiveService to generate the overview and grouped entry HTML files.

- Removed the generateHTML and writeHTMLToFile functions.
- Called application.ArchiveService.MakeArchiveHTML to generate the HTML files.
- Updated the email sending logic to use the files returned by MakeArchiveHTML.

rework services for isolation and avoid cycles bd7ee12aac

refactor: rename HTMLTemplateData to OverviewTemplateData ... 5f9ffc1381

Renamed the generic HTMLTemplateData struct to OverviewTemplateData
to better reflect its purpose as the data model for the digest overview.
This improves clarity and consistency with GroupedEntriesTemplateData.

Updated all references across the codebase, including:
- internal/app/services/*
- internal/archive/archive.go
- internal/digest/digest.go
- internal/email/email.go
- internal/email/email_test.go
- internal/models/models.go
- internal/processor/processor.go
- internal/templates/templates.go
- internal/templates/templates_test.go
- internal/testutil/mocks.go
- scripts/preview/main.go

feat: Refactor web server logic into internal/webserver 61a2add5df

feat: Implement local preview server with graceful shutdown 06612da67a

feat: Produce overview and multiple group entry digests 1779ceb847

feat: update email digest and attachment handling ... 8f83db3a88

modifies email service to account for attaching multiple digests in one
email.

updates preview-email for testing

refactor: Improve archive cleanup logic ... 0a9576480c

This commit refactors the archive cleanup functionality to be more robust and general.

The previous implementation relied on directory names to identify old archives. This new implementation crawls the directory tree and removes files based on their modification time. It then does a second pass to remove any empty directories.

This approach is more resilient to changes in the directory structure and accurately reflects the user's intent to clean up old files based on their age.

rework web server for better sanitization and avoiding directory listing 5dfe4be0b5

decouple preview serving from html. clean up preview script. fix directory listing blocking 000185a47e

rework config defaults and accepted values ... 21eeecf532

fix up preview to handle miniflux and mock data separately

bring back mark as read ... 77e3a1b316

mark entries in digest as read

remove unused miniflux calls 6cd6aba85c

decouple miniflux models from internal app models b623aaa159

ensure llm summaries get pulled into the right model 76850a6a5a

revise stat line to always include feed count and show group summary stat in overview 475082c3b2

bring back llm summaries due to bad struct field and test struct mismatch b9a6685312

fix entry total for ai group 68e6da4178

feat: Add test for initScheduler 06ddde7cf7

feat: Add tests for app.go NewApp and With* functions 48e1e033c0

feat: Add happy path test for ProcessAndSendDigest 5352931433

feat: Add tests for digest.BuildDigestData (non-AI) f27f07d8ab

feat: Add tests for overview.gohtml and textToHTML func 61e2a7b388

feat: Add tests for archive.go functions 2498dcfba7

clear out superflous comments 7ea8855acb

eleith added 6 commits 2025-08-23 15:55:56 +00:00

fix issue when llm responds incorrectly ... 01a1e3eec4

default to feed sub_group_by instead of date

adjust models so primary goups have IDs and allow allow subgroupers to work with these primary groups directly 33aab97394

continue to iterate with gemini calls to improve output success and add more logging e195701db3

improve llm test speed by mocking sleep 220dd25b68

fix favicon rendering 6720bec04e

fix miniflux preview favicon generation 026ac4b802

eleith merged commit 4ce33975ee into main 2025-08-23 16:02:50 +00:00

eleith deleted branch gemini/feat/overview-digest 2025-08-23 16:02:50 +00:00

eleith referenced this issue from a commit 2025-08-23 16:02:52 +00:00

Merge pull request 'overview digest' (#51) from gemini/feat/overview-digest into main

eleith referenced this issue from a commit 2025-12-10 22:24:46 +00:00

Merge pull request 'overview digest' (#51) from gemini/feat/overview-digest into main

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Assignees

Clear assignees

eleith (eleith)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: eleith/miniflux-digest#51