eleith/miniflux-digest
1
1 Fork 0
Code Issues Pull Requests Releases Activity

gemini/refactor/generalize-chunk-processing #60

Merged
eleith merged 12 commits from gemini/refactor/generalize-chunk-processing into main 2025-09-16 18:35:24 +00:00
Conversation 0 Commits 12 Files Changed 11
+825 -292
12 Commits
Author SHA1 Message Date
eleith be5c5c3301 refactor large methods in view/ai.go
ci/woodpecker/push/build-docker Pipeline was successful
Details
2025-09-16 11:26:45 -07:00
eleith 1f403ebd93 improve prompt 2025-09-16 10:59:13 -07:00
eleith 1f33b2e452 move helper method in llm service 2025-09-16 09:58:53 -07:00
eleith a5ba746e79 refactor(llm): Move ExecuteLLMRequestAndParse to llm/util.go 
Move the `ExecuteLLMRequestAndParse` function from
`internal/digest/view/ai.go` to `internal/llm/util.go`. This improves
readability and reduces code duplication.
 2025-09-16 16:26:16 +00:00
eleith 70dbbd2bb8 refactor(ai): Refactor getSummaryForGroup to use executeLLMRequestAndParse 
Refactor the `getSummaryForGroup` function to use the new
`executeLLMRequestAndParse` function. This improves readability and
reduces code duplication.
 2025-09-16 16:15:47 +00:00
eleith 59422b3f96 refactor(ai): Refactor consolidatePrimaryGroups to use executeLLMRequestAndParse 
Refactor the `consolidatePrimaryGroups` function to use the new
`executeLLMRequestAndParse` function. This improves readability and
reduces code duplication.
 2025-09-16 16:15:10 +00:00
eleith c54324a2ac refactor(ai): Abstract LLM entry preparation and execution 
Abstract out reusable logic for preparing LLM entries and executing
LLM requests. This improves readability and reduces code duplication.
 2025-09-16 16:14:31 +00:00
eleith dd0779827b refactor(ai): Generalize chunk processing 
Remove redundant `processSingleChunk` function and simplify
`ProcessPrimaryGroupWithAI` to always process chunks concurrently. This
improves readability and reduces code duplication.
 2025-09-16 16:11:42 +00:00
eleith 6d14ef5168 refactor(digest): Rename job entry limit constant 
Renames `MaxEntriesForSummarization` to `MaxEntriesPerJob` to reflect its general use in all AI processing jobs.
 2025-09-16 15:41:22 +00:00
eleith f8c75a7c11 test for util llm functions 2025-09-16 08:29:44 -07:00
eleith 8ba0024b5b refactor(digest): Parallelize summarization and sub-grouping 
Refactors `ProcessPrimaryGroupWithAI` to use the `llm.ProcessInChunks` helper.

For large entry groups, summarization and sub-grouping tasks are now chunked and run in parallel to prevent timeouts.

Tests for this function have been updated to cover the new chunking and failure-handling logic.
 2025-09-16 15:29:02 +00:00
eleith 0ae0f0efe2 refactor(digest): Add generic parallel processing helper 
Adds a generic llm.ProcessInChunks function to run jobs on data chunks in parallel.

Refactors GroupAIEntries to use this new helper function, replacing its own concurrency management.
 2025-09-16 15:04:11 +00:00