17KB

Rå Blame Historik

title: ‘Add XLSX Jurisdiction Import’ slug: ‘add-xlsx-jurisdiction-import’ created: ‘2026-03-13’ status: ‘implemented’ stepsCompleted: [1, 2, 3, 4] tech_stack: [‘Classic ASP’, ‘VBScript’, ‘IIS’, ‘ADO’, ‘Microsoft Jet/ACE OLEDB’, ‘FreeASPUpload’, ‘Session-backed MVC helpers’] files_to_modify: ['/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp’, ‘/workspace/App/Views/Jurisdiction/import.asp’, ‘/workspace/App/DomainModels/JurisdictionRepository.asp’] code_patterns: [‘Controller action with inline file-processing logic’, ‘Repository-based persistence with raw SQL and Automapper models’, ‘Session-backed Flash/FormCache/CSRF helpers’, ‘Server-rendered forms with HTML helper methods’] test_patterns: [‘ASPUnit helper tests only’, ‘No existing controller/import coverage’, ‘Manual IIS verification required for upload/import flows’]

Tech-Spec: Add XLSX Jurisdiction Import

Created: 2026-03-13

Overview

Problem Statement

The current jurisdiction import only supports .csv and .txt, relies on the Jet text driver, and does not persist the imported jurisdiction updates. The business now needs an .xlsx-only import flow for a known spreadsheet format that updates the Jurisdiction table and gives the user progress feedback while the import runs.

Solution

Replace the current jurisdiction import implementation with an XLSX-based import pipeline that reads a fixed workbook format, skips row 1, uses row 2 as headers, extracts JCode and normalized Name from the Jurisdiction column, joins City & Township and ZIP + 4 into CSZ, regenerates IMB_Digits and IMB using existing logic, updates existing Jurisdiction rows by extracted JCode, inserts missing JCode rows, and exposes AJAX progress updates in the import UI.

Scope

In Scope:

Replace the current import UI and backend flow with .xlsx-only support
Support the known workbook shape represented by Data\BRM Permit Info February '26 updated.xlsx
Ignore row 1 and treat row 2 as the header row
Extract JCode from text inside parentheses in the Jurisdiction column
Normalize Name from the text before parentheses
If the normalized name ends with CITY, transform it to CITY OF <name without trailing CITY>
Build CSZ from City & Township plus ZIP + 4
Update Mailing_Address, CSZ, IMB_Digits, and IMB
Regenerate IMB through the existing GetIMBCodec.EncodeDigits(...) path
Add AJAX-driven progress feedback to the import screen

Out of Scope:

Supporting .csv or .txt imports on the jurisdiction screen
Supporting arbitrary workbook layouts or multiple worksheet formats
Broad refactors to unrelated jurisdiction CRUD features

Context for Development

Codebase Patterns

The current jurisdiction import lives in JurisdictionController.ImportPost and uses FreeASPUpload plus server-side file processing.
The current CSV/TXT path already contains business logic for JCode, Name, CSZ, IMB_Digits, and IMB derivation.
The repo is a Classic ASP / VBScript application running on IIS with existing shared helpers in MVC/.
Environment-sensitive behavior should stay minimal and localized because this codebase is tightly coupled to Windows/IIS/runtime dependencies.
Controller methods are allowed to contain substantial workflow logic in this codebase; there is no separate service layer pattern to follow.
Persistence updates should still flow through JurisdictionRepository instead of ad hoc SQL inside views.
Session-backed helpers already exist for flash state and CSRF tokens, which makes Session a viable anchor for import progress state if the implementation stays inside the web app.
MVC/lib.json.asp exists, so lightweight JSON responses for AJAX polling fit the current stack without introducing a new dependency.

Files to Reference

File	Purpose
`/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp`	Current import entry point and existing CSV/TXT derivation logic
`/workspace/App/Views/Jurisdiction/import.asp`	Current upload UI to be changed for XLSX and progress feedback
`/workspace/App/DomainModels/JurisdictionRepository.asp`	Jurisdiction lookup and update persistence path
`/workspace/MVC/lib.Upload.asp`	Existing upload helper used by controller imports
`/workspace/MVC/lib.json.asp`	Existing JSON helper if the progress endpoint returns polling status
`/workspace/Data/BRM Permit Info February '26 updated.xlsx`	Sample workbook shape to anchor header and column mapping
`/workspace/_bmad-output/project-context.md`	Brownfield implementation rules and constraints

Technical Decisions

The import screen will become XLSX-only.
Matching is by extracted JCode from the Jurisdiction column.
The implementation must preserve and reuse the existing IMB encoding path rather than inventing a new barcode algorithm.
Progress feedback is required and should be exposed through AJAX rather than a synchronous full-page post only.
The current Jet text-driver import path cannot read .xlsx; the implementation needs a Windows/IIS-compatible Excel-reading strategy.
Primary workbook-read strategy should use Microsoft.ACE.OLEDB.12.0 on Windows/IIS. If ACE is unavailable, the import should fail with a clear user-facing error.
The controller will likely need to split import work into at least two responsibilities: kickoff/upload and progress/status reporting.
The repository currently supports FindByJCode and Update, which is enough for update-by-JCode if the controller loads and mutates each row model before persisting.
There is no existing automated controller/import test pattern, so verification will rely on manual IIS-based import testing plus any helper-level tests that can be isolated.
The import page should use a kickoff action plus an AJAX polling endpoint. Progress state can be stored in Session under a generated import token.
Because this runs under Classic ASP/IIS, true live per-row progress may be constrained by request/session locking behavior. The implementation should use a two-phase design if needed and may expose checkpoint-based progress if that is the most reliable bounded solution.
Required workbook headers should be validated before processing begins: County, Jurisdiction, Mailing Address, City & Township, ZIP + 4, and Mailer ID Option 1.
Rows with extracted JCode values not found in Jurisdiction should be inserted and counted separately in the final summary.
If duplicate JCode rows appear in the workbook, process them in file order and let the last valid row win while counting duplicates in the final summary.
Name normalization should trim whitespace and apply the CITY rewrite only when the extracted name ends with the standalone word CITY.
Row-level failures should be isolated, counted, and reported in the completion summary rather than aborting the entire import after partial progress.
Progress state should remain session-scoped rather than being persisted to the database or temp files.
Persistence should prefer the existing FindByJCode + mutate + JurisdictionRepository.Update flow before introducing a new repository bulk-update abstraction.
Header binding should validate the expected row-2 header names and then resolve columns by header names rather than raw column positions only.
The final import summary should include at minimum: total rows seen, rows updated, rows inserted, duplicate rows encountered, invalid rows skipped, and row-level failures.
The final result should also expose row-level operator feedback with row number, failure reason, and full imported record details for malformed Jurisdiction values, IMB_Digits build failures, ACE read failures, and other row-level processing errors.

Implementation Plan

Tasks

Task 1: Replace the current jurisdiction import contract with XLSX-only validation
- File: /workspace/App/Controllers/Jurisdiction/JurisdictionController.asp
- Action: Update ImportPost so it only accepts .xlsx uploads, removes the CSV/TXT-specific text-driver path, and returns a clear validation error for any non-XLSX file.
- Notes: Keep the existing FreeASPUpload flow for multipart upload handling unless the ACE-based workbook read requires a different saved-file path convention.
Task 2: Add workbook-reading and header-validation logic for the known spreadsheet format
- File: /workspace/App/Controllers/Jurisdiction/JurisdictionController.asp
- Action: Read the uploaded .xlsx workbook through Microsoft.ACE.OLEDB.12.0, target the first worksheet, ignore row 1, treat row 2 as headers, and validate the required headers before row processing begins.
- Notes: Fail fast with a user-facing error if ACE is unavailable, the workbook cannot be opened, or required headers are missing.
Task 3: Implement row-mapping and normalization rules for jurisdiction updates
- File: /workspace/App/Controllers/Jurisdiction/JurisdictionController.asp
- Action: For each workbook row, extract JCode from the Jurisdiction value inside parentheses, derive normalized Name from the text before parentheses, apply the CITY to CITY OF ... rewrite when appropriate, join City & Township and ZIP + 4 into CSZ, compute IMB_Digits, and regenerate IMB via GetIMBCodec.EncodeDigits(...).
- Notes: Trim whitespace on all derived values and treat malformed or unparseable rows as row-level failures instead of fatal process errors.
Task 4: Persist updates by existing JCode and track import result counters
- File: /workspace/App/Controllers/Jurisdiction/JurisdictionController.asp
- File: /workspace/App/DomainModels/JurisdictionRepository.asp
- Action: For each valid parsed row, load the existing jurisdiction by JCode, mutate the model fields, and call JurisdictionRepository.Update; insert missing JCode rows with the parsed values; detect duplicates in workbook order and allow the last valid row to win.
- Notes: Only add repository support if a small helper materially reduces controller duplication; otherwise stay with the existing FindByJCode + Update pattern.
Task 5: Introduce bounded progress-state tracking and a polling endpoint
- File: /workspace/App/Controllers/Jurisdiction/JurisdictionController.asp
- Action: Split the import flow into a kickoff path and a progress/status path, storing progress state in Session under an import token and returning JSON-compatible status for AJAX polling.
- Notes: If Classic ASP session locking prevents true live row-by-row progress, expose checkpoint-based phases such as uploaded, opening workbook, validating headers, processing rows, and complete.
Task 6: Replace the import page UI with XLSX-only upload and progress display
- File: /workspace/App/Views/Jurisdiction/import.asp
- Action: Update the form copy and file input to XLSX-only, add client-side AJAX kickoff/polling behavior, render a progress bar/status area, and display final summary counts plus row-level error details.
- Notes: The UX should remain usable if progress is phase-based rather than exact per-row percentage.
Task 7: Define the final operator summary and failure reporting
- File: /workspace/App/Controllers/Jurisdiction/JurisdictionController.asp
- File: /workspace/App/Views/Jurisdiction/import.asp
- Action: Return or render a completion result that includes total rows seen, rows updated, rows inserted, duplicate rows encountered, invalid rows skipped, row-level failures, and row-specific error details.
- Notes: Include row number, failure reason, and full record details for malformed Jurisdiction values, IMB digit failures, insert/update failures, and workbook-read errors where applicable.
Task 8: Update user-facing messaging and implementation notes for runtime dependency risk
- File: /workspace/App/Views/Jurisdiction/import.asp
- File: /workspace/App/Controllers/Jurisdiction/JurisdictionController.asp
- Action: Make sure operator-facing errors clearly describe ACE dependency issues and workbook validation failures.
- Notes: This keeps deployment/runtime failures diagnosable without digging through server logs first.

Acceptance Criteria

AC 1: Given the user opens the jurisdiction import page, when the screen renders, then it only advertises and accepts .xlsx uploads.
AC 2: Given a non-.xlsx file is submitted, when the import starts, then the user receives a validation error and no jurisdiction rows are changed.
AC 3: Given the uploaded workbook cannot be opened through ACE/OLEDB, when the import starts, then the user receives a clear workbook/ACE error and no jurisdiction rows are changed.
AC 4: Given the workbook is missing one of the required row-2 headers, when validation runs, then the import stops before row processing and reports the missing header(s).
AC 5: Given a row contains Jurisdiction text in the format ALCONA TOWNSHIP (01040), when the row is processed, then JCode is parsed as 01040 and Name is parsed as ALCONA TOWNSHIP.
AC 6: Given a parsed jurisdiction name ends with the standalone word CITY, when normalization runs, then the stored Name becomes CITY OF <name without trailing CITY>.
AC 7: Given a workbook row has City & Township and ZIP + 4 values, when the row is processed, then the database CSZ field is updated with those values joined in the same import flow.
AC 8: Given a valid workbook row matches an existing jurisdiction by extracted JCode, when the row is processed, then Name, Mailing_Address, CSZ, IMB_Digits, and IMB are updated in the Jurisdiction table.
AC 9: Given a valid workbook row is processed, when IMB_Digits is derived, then IMB is regenerated using the existing GetIMBCodec.EncodeDigits(...) functionality.
AC 10: Given a workbook row has an extracted JCode that does not exist in the database, when the row is processed, then a new Jurisdiction row is inserted with the parsed/imported values.
AC 11: Given the same JCode appears more than once in the workbook, when processing completes, then the last valid occurrence wins and duplicates are counted in the final summary.
AC 12: Given a row has malformed Jurisdiction text that cannot produce a valid JCode, when the row is processed, then that row is counted as failed or invalid, the import continues with subsequent rows, and the final output includes the full imported record details.
AC 13: Given the user starts an import, when the operation is in progress, then the page shows AJAX-driven progress feedback for the current import token.
AC 14: Given true live row-by-row progress is not technically reliable under Classic ASP/IIS, when the import runs, then the user still sees checkpoint-based progress states instead of a frozen screen.
AC 15: Given the import completes, when final results are shown, then the user sees total rows seen, rows updated, rows inserted, duplicates encountered, invalid rows skipped, and failures.
AC 16: Given any malformed Jurisdiction or IMB_Digits build failures occur, when results are shown, then the user can see row numbers, failure reasons, and the full source record for those rows.

Additional Context

Dependencies

Windows/IIS runtime with Classic ASP enabled
FreeASPUpload remains the upload mechanism unless implementation constraints force a localized alternative
Microsoft.ACE.OLEDB.12.0 must be installed and available on the target server for workbook access
Existing GetIMBCodec.EncodeDigits(...) functionality must remain available during import processing
Existing MVC JSON helper in /workspace/MVC/lib.json.asp can support polling responses

Testing Strategy

Manual IIS test: upload a valid copy of [BRM Permit Info February ‘26 updated.xlsx](/workspace/Data/BRM Permit%20Info%20February%20'26%20updated.xlsx) and verify updated jurisdiction rows in the database
Manual IIS test: upload a non-XLSX file and verify validation failure
Manual IIS test: upload a workbook with one required header removed and verify fast-fail header validation
Manual IIS test: upload a workbook containing malformed Jurisdiction values and verify row-level error reporting with continued processing and full source-record details
Manual IIS test: upload a workbook containing missing JCode rows and verify the rows are inserted into Jurisdiction
Manual IIS test: upload a workbook containing invalid Mailer ID Option 1 or ZIP + 4 values and verify the full source record is shown in the error output
Manual IIS test: upload a workbook containing duplicate JCode rows and verify the last valid row wins
Manual IIS test: verify progress bar/status polling updates during import and resolves to a final summary state
Optional helper-level verification: isolate and test any extracted parsing/normalization helper routines if they are moved out of the controller into reusable VBScript functions

Notes

Highest implementation risk is the interaction between Classic ASP request lifecycle, Session locking, and AJAX polling; the implementation should prefer a reliable bounded progress model over an unreliable “live” illusion.
ACE/OLEDB availability is a deployment/runtime dependency, not just a coding detail; missing-provider handling must be first-class.
Keep the change bounded to the existing jurisdiction import workflow and avoid turning this into a generalized spreadsheet-import framework.
If the controller becomes too large, extraction into small local helper functions is acceptable, but avoid broad architectural refactors.

17KB Rå Blame Historik