CSV_TO_PDF_REPORT/backlog/sprints/sprint-3.md

Sprint Backlog

Sprint: 3 Dates: 2026-09-21 - 2026-09-25 Sprint Goal: Close the high-volume render throughput risk with a confirmed root cause and a measured mitigation, and deliver the first slice of CSV field mapping (load headers, bind a dynamic field, rebind it).

Committed Items

Story	Size	Status	Tasks
Investigate and address high-volume render throughput degradation	8 points	Done	- [x] Build a scaled-down repeatable throughput probe (extend Sprint 2's benchmark scenario) that runs in minutes, not tens of minutes, so hypotheses can be iterated on quickly - [x] Test hypothesis: does periodic Save-to-file + release/reopen a fresh PDFLibrary instance reset the per-page cost curve? Record results against BENCHMARK.md's root-cause section - [x] Confirm or rule out the Debenu-internal-document-model hypothesis based on the above; document the evidence - [x] Design and implement one mitigation in production code: extend IPdfRenderer/DebenuPdfRenderer to batch pages across multiple underlying Debenu documents (save+reopen every N pages) and merge/concatenate the batches into the single final output PDF - [x] Add/extend unit tests for the batching + merge logic (fake-renderer path plus a real-Debenu integration test for actual merge correctness) - [x] Re-run the full 100k-record benchmark to completion (not time-boxed) with the mitigation in place; record elapsed time, throughput curve, and output PDF validity/size - [x] Document the result in code/BENCHMARK.md against the 10-minute target (met / not met, by how much, what remains open) and qualitatively re-assess project_config.md's 1,000,000-record ceiling - [x] Regression pass: confirm existing render correctness (392-record sample, exit codes, PROGRESS stream) is unchanged; update CLI_CONTRACT.md if the mitigation changes any observable CLI behavior (no contract changes needed — behavior unchanged, confirmed live)
Load CSV headers and sample rows into the desktop app	3 points	Done	- [x] Add a CSV file-open action to the desktop app (file picker supporting local and UNC paths) using the existing CsvHelper dependency - [x] Parse the header row plus a bounded sample of data rows; surface a clear operator-facing error for malformed CSV instead of crashing - [x] Display column headers and sample rows in a new grid/list view in the designer - [x] Unit tests (valid header+rows, malformed CSV, empty file) plus a live manual check against code/sample-data/87700 - 999999 - Wilson Township.csv
Create a dynamic text token from a CSV column	5 points	Done	- [x] Surface the loaded CSV header list (from the story above) in the designer as a selectable field source - [x] Extend the existing “Add Dynamic Placeholder” flow (TemplateDesignerForm) so a new dynamic field is bound to an operator-selected real CSV column instead of a generic placeholder (drag-and-drop or an explicit bind action, whichever is faster to deliver) - [x] Persist the bound column using the render-time column="Column Name" attribute format in TemplateLayoutDocument/the XML serializer — confirmed against TEMPLATE_FORMAT.md/TemplateXmlParser and Sprint 2's already-existing serializer that this (not a {{Column Name}} token) is the real, already-renderable format; see the epic's verification note - [x] Render the bound field on canvas with a visible placeholder representation distinct from static text - [x] Unit tests covering token creation, persistence round-trip, and arbitrary (non-hardcoded) column names - [x] Live manual verification against the built .exe: load the real sample CSV, create a mapped field, save, and confirm the saved XML token format
Re-map an existing dynamic field	2 points	Done	- [x] Add a rebind control (properties panel or mapping pane) listing the currently loaded CSV's columns for a selected dynamic field - [x] Implement rebind logic: update the bound column without recreating or repositioning the element; persist on save - [x] Unit tests for rebind (persistence, no repositioning, correct column reflected) plus a live check that a subsequent preview/render run reflects the new mapping

Notes

• Capacity signal: velocity range of 19-20 points (Sprint 1: 20 completed, Sprint 2: 19 completed) — still low-confidence off two data points. Committing 18 points, leaving 1-2 points of buffer per AGENTS.md's guidance to reserve roughly 10% of capacity for grooming rather than packing to the exact ceiling, especially with the throughput story carrying more open-ended risk than a typical 8-pointer.
• Planning rationale: pulled in the order recommended at Sprint 2 review (backlog/backlog.md) — the throughput-degradation story first (highest priority: threatens a hard product constraint and compounds at the stated 1,000,000-record ceiling), then CSV Integration stories in existing backlog order starting with the no-dependency “Load CSV headers” story (deferred from Sprint 2 as a stretch item), no cherry-picking.
• Sizing decision: “Investigate and address high-volume render throughput degradation” was unsized entering this sprint. dev-team sized it at 8 points during planning — see the sizing note now recorded in backlog/epics/05_cli_rendering_engine_and_debenu_integration.md. Its own acceptance criteria already bound it to a time-boxed shape (confirm one hypothesis, implement one mitigation, honest “not met” is an acceptable documented outcome), and code inspection found a clean, testable seam for the leading mitigation candidate — so it was sized directly rather than split into a separate spike-then-mitigation pair.
• Deliberately not committed this sprint (capacity discipline, “don't start what you can't finish”): “Collapse blank optional address lines consistently” (5 points — would push the total to 23, clearly over the capacity range) and “Harden production configuration delivery for CLI runtime settings” (2 points — not urgent per the product owner's note on that story, and would consume the sprint's grooming buffer). Both are clean pulls for Sprint 4.
• Impediments: template asset path strategy (absolute vs relative) and UNC timeout/retry behavior remain open in logs/impediment_log.md. Neither blocks this sprint's committed items.
• Carried over from previous sprint: none (Sprint 2 completed all 5 committed stories).
• Sequencing risk: “Re-map an existing dynamic field” depends on “Create a dynamic text token” landing first, both committed in this same sprint — swarm to finish that story before starting this one's implementation.

Daily Scrum Log

Day	Date	Completed	Planned	Blocked/At risk
1	2026-09-21	Batch 1 (“Investigate and address high-volume render throughput degradation”, 8 points) done, all 8 tasks and all 4 ACs met with real evidence, no simulation. Built a scaled-down probe (real Debenu DLL, 1,000-4,000 pages, runs in 2-3 min) that reproduced the Sprint 2 spike's degradation curve at a fraction of the wall time. Confirmed the Debenu-internal-document-model hypothesis directly: save+release/reopen a fresh PDFLibrary instance repeatably resets the per-page cost curve (validated at both 4,000-page and sustained 10,000-page/20-batch scale). Measured a genuine throughput-vs-file-size trade-off across batch sizes 100-1,000 and picked 300 pages/batch for production (evidence in code/BENCHMARK.md). Implemented the mitigation in DebenuPdfRenderer (batch across multiple Debenu documents, merge via Debenu's real MergeFileListFast), with a documented fast path that keeps small/typical renders byte-for-byte unchanged. Added 8 new unit/integration tests (201/201 passing total, up from 195). Re-ran the full 100,352-record/300 DPI benchmark to completion (not time-boxed): 315 seconds, ~47.5% under the 10-minute target, valid 100,352-page PDF (397 MB), content-correctness spot-checked at start/middle/end. Documented the result and qualitatively re-assessed the 1,000,000-record ceiling in code/BENCHMARK.md and logs/technical_debt_log.md: throughput looks fine extrapolated, but a new file-size risk (~4.5-5 GB extrapolated, over the 2 GB constraint) was found and logged as open technical debt rather than silently absorbed or overstated as solved. Regression pass complete: live-verified the real built CLI's --help, no-args, missing-template, and full 392-record render paths all match CLI_CONTRACT.md exactly; no contract changes needed since batching is entirely internal to DebenuPdfRenderer.	Next: begin Batch 2 (“Load CSV headers and sample rows into the desktop app”) per the Execution Order table below.	None. The one open item is the newly-discovered 1,000,000-record file-size risk (logs/technical_debt_log.md, 2026-09-21 entry) — not a blocker for this sprint (no committed story renders at that scale), tracked as open technical debt for a future sprint to pick up before any story commits to rendering near that ceiling.
2	2026-09-22	Batches 2-4 (the full CSV Integration chain, 3+5+2 = 10 points) all done in dependency order, swarmed to completion one at a time rather than started in parallel. Batch 2 (“Load CSV headers and sample rows into the desktop app”): added CsvPreviewLoader (EnvelopeRenderer.Desktop.Core.Csv, same CsvHelper package the CLI already uses) and a “Load CSV...” action + preview grid in TemplateDesignerForm; 7 new tests (valid rows, bounded sample, empty file, malformed/unterminated-quote CSV, missing file, blank path, header-only file). Batch 3 (“Create a dynamic text token from a CSV column”): before writing any code, confirmed in TEMPLATE_FORMAT.md/TemplateXmlParser and Sprint 2's existing TemplateLayoutXmlSerializer that the real render-time binding is a column="<header>" attribute, not the {{Column Name}} token mentioned in this story's older conversation notes — built on the existing, already-renderable format instead of inventing a second one. Extended the existing “Add Dynamic Placeholder” flow with an explicit column-picker combo box (populated from Batch 2's loaded headers, disabled until a CSV is loaded) and added a light highlight fill so dynamic elements are visibly distinct on canvas, not just textually ({Column} bracket notation already existed from Sprint 2). 9 new tests cover token creation and XML round-trip with arbitrary column names (including +, &, ', embedded quotes). Batch 4 (“Re-map an existing dynamic field”): added a “CSV column:” rebind combo to the existing properties panel (TextElementPropertiesEditor.SetColumnName, touches only ColumnName, never X/Y), enabled only for a selected dynamic element with a loaded CSV. 5 new tests cover rebind-without-moving, static-element no-op, blank-input rejection, and an arbitrary column name. 223/223 tests passing across both test projects (up from 201). Live verification against the actual built .exe (real Win32/UI-Automation input injection, not an in-process harness): loaded the real sample CSV (confirmed 22 columns/20 sample rows in the grid); bound a new dynamic field to “Full Name” and confirmed the highlighted {Full Name} placeholder at (29.70, 547.20) pt; rebound it live to “G2 City St ZIP+4” and confirmed the canvas text updated to {G2 City St ZIP+4} at the exact same (29.70, 547.20) pt (no repositioning); saved and inspected the real XML on disk — column="G2 City St ZIP+4" with matching x/y; ran the real EnvelopeRenderer.Cli.exe against that saved template and the real sample CSV — exit 0, PROGRESS complete ... completed=392, valid 392-page PDF, confirming the new mapping is honored on the next render. One honestly-reported gap: every one of the 392 sample records has a blank G2 City St ZIP+4 value (confirmed via Import-Csv), so that specific rendered page is visually blank at the field — not a defect, just this sample file's data shape; to get a positive visual confirmation of the same coordinate pipeline, additionally rendered a second template bound to the populated Precinct column at the identical x/y and confirmed via a rasterized page crop that “15” (the real first-record value) renders exactly where expected.	Sprint's committed backlog is now fully done (18/18 points across all 4 committed stories). Next: sprint review and retrospective per process/03_sprint_execution.md's exit criteria.	None.

Execution Order

Sequenced by dependency and priority; the throughput story is pulled first since it's the highest-priority known risk and has no in-sprint dependency, followed by the CSV Integration chain in backlog order.

Batch	Story	Why it's gated here
1	Investigate and address high-volume render throughput degradation	No dependency on any other Sprint 3 story; highest priority per Sprint 2 review (threatens a hard product constraint). Pulled first.
2	Load CSV headers and sample rows into the desktop app	No dependency on Batch 1; needed before any dynamic-field-binding work since fields must bind to real loaded CSV columns.
3	Create a dynamic text token from a CSV column	Needs loaded CSV headers (Batch 2) to bind fields to, and the canvas placement work from Sprint 2 (already Done).
4	Re-map an existing dynamic field	Needs an existing dynamic field to rebind, i.e. Batch 3 done first.