--- title: 'Add XLSX Jurisdiction Import' slug: 'add-xlsx-jurisdiction-import' created: '2026-03-13' status: 'implemented' stepsCompleted: [1, 2, 3, 4] tech_stack: ['Classic ASP', 'VBScript', 'IIS', 'ADO', 'Microsoft Jet/ACE OLEDB', 'FreeASPUpload', 'Session-backed MVC helpers'] files_to_modify: ['/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp', '/workspace/App/Views/Jurisdiction/import.asp', '/workspace/App/DomainModels/JurisdictionRepository.asp'] code_patterns: ['Controller action with inline file-processing logic', 'Repository-based persistence with raw SQL and Automapper models', 'Session-backed Flash/FormCache/CSRF helpers', 'Server-rendered forms with HTML helper methods'] test_patterns: ['ASPUnit helper tests only', 'No existing controller/import coverage', 'Manual IIS verification required for upload/import flows'] --- # Tech-Spec: Add XLSX Jurisdiction Import **Created:** 2026-03-13 ## Overview ### Problem Statement The current jurisdiction import only supports `.csv` and `.txt`, relies on the Jet text driver, and does not persist the imported jurisdiction updates. The business now needs an `.xlsx`-only import flow for a known spreadsheet format that updates the `Jurisdiction` table and gives the user progress feedback while the import runs. ### Solution Replace the current jurisdiction import implementation with an XLSX-based import pipeline that reads a fixed workbook format, skips row 1, uses row 2 as headers, extracts `JCode` and normalized `Name` from the `Jurisdiction` column, joins `City & Township` and `ZIP + 4` into `CSZ`, regenerates `IMB_Digits` and `IMB` using existing logic, updates existing `Jurisdiction` rows by extracted `JCode`, inserts missing `JCode` rows, and exposes AJAX progress updates in the import UI. ### Scope **In Scope:** - Replace the current import UI and backend flow with `.xlsx`-only support - Support the known workbook shape represented by `Data\BRM Permit Info February '26 updated.xlsx` - Ignore row 1 and treat row 2 as the header row - Extract `JCode` from text inside parentheses in the `Jurisdiction` column - Normalize `Name` from the text before parentheses - If the normalized name ends with `CITY`, transform it to `CITY OF ` - Build `CSZ` from `City & Township` plus `ZIP + 4` - Update `Mailing_Address`, `CSZ`, `IMB_Digits`, and `IMB` - Regenerate `IMB` through the existing `GetIMBCodec.EncodeDigits(...)` path - Add AJAX-driven progress feedback to the import screen **Out of Scope:** - Supporting `.csv` or `.txt` imports on the jurisdiction screen - Supporting arbitrary workbook layouts or multiple worksheet formats - Broad refactors to unrelated jurisdiction CRUD features ## Context for Development ### Codebase Patterns - The current jurisdiction import lives in `JurisdictionController.ImportPost` and uses `FreeASPUpload` plus server-side file processing. - The current CSV/TXT path already contains business logic for `JCode`, `Name`, `CSZ`, `IMB_Digits`, and `IMB` derivation. - The repo is a Classic ASP / VBScript application running on IIS with existing shared helpers in `MVC/`. - Environment-sensitive behavior should stay minimal and localized because this codebase is tightly coupled to Windows/IIS/runtime dependencies. - Controller methods are allowed to contain substantial workflow logic in this codebase; there is no separate service layer pattern to follow. - Persistence updates should still flow through `JurisdictionRepository` instead of ad hoc SQL inside views. - Session-backed helpers already exist for flash state and CSRF tokens, which makes Session a viable anchor for import progress state if the implementation stays inside the web app. - `MVC/lib.json.asp` exists, so lightweight JSON responses for AJAX polling fit the current stack without introducing a new dependency. ### Files to Reference | File | Purpose | | ---- | ------- | | `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` | Current import entry point and existing CSV/TXT derivation logic | | `/workspace/App/Views/Jurisdiction/import.asp` | Current upload UI to be changed for XLSX and progress feedback | | `/workspace/App/DomainModels/JurisdictionRepository.asp` | Jurisdiction lookup and update persistence path | | `/workspace/MVC/lib.Upload.asp` | Existing upload helper used by controller imports | | `/workspace/MVC/lib.json.asp` | Existing JSON helper if the progress endpoint returns polling status | | `/workspace/Data/BRM Permit Info February '26 updated.xlsx` | Sample workbook shape to anchor header and column mapping | | `/workspace/_bmad-output/project-context.md` | Brownfield implementation rules and constraints | ### Technical Decisions - The import screen will become XLSX-only. - Matching is by extracted `JCode` from the `Jurisdiction` column. - The implementation must preserve and reuse the existing IMB encoding path rather than inventing a new barcode algorithm. - Progress feedback is required and should be exposed through AJAX rather than a synchronous full-page post only. - The current Jet text-driver import path cannot read `.xlsx`; the implementation needs a Windows/IIS-compatible Excel-reading strategy. - Primary workbook-read strategy should use `Microsoft.ACE.OLEDB.12.0` on Windows/IIS. If ACE is unavailable, the import should fail with a clear user-facing error. - The controller will likely need to split import work into at least two responsibilities: kickoff/upload and progress/status reporting. - The repository currently supports `FindByJCode` and `Update`, which is enough for update-by-`JCode` if the controller loads and mutates each row model before persisting. - There is no existing automated controller/import test pattern, so verification will rely on manual IIS-based import testing plus any helper-level tests that can be isolated. - The import page should use a kickoff action plus an AJAX polling endpoint. Progress state can be stored in `Session` under a generated import token. - Because this runs under Classic ASP/IIS, true live per-row progress may be constrained by request/session locking behavior. The implementation should use a two-phase design if needed and may expose checkpoint-based progress if that is the most reliable bounded solution. - Required workbook headers should be validated before processing begins: `County`, `Jurisdiction`, `Mailing Address`, `City & Township`, `ZIP + 4`, and `Mailer ID Option 1`. - Rows with extracted `JCode` values not found in `Jurisdiction` should be inserted and counted separately in the final summary. - If duplicate `JCode` rows appear in the workbook, process them in file order and let the last valid row win while counting duplicates in the final summary. - Name normalization should trim whitespace and apply the `CITY` rewrite only when the extracted name ends with the standalone word `CITY`. - Row-level failures should be isolated, counted, and reported in the completion summary rather than aborting the entire import after partial progress. - Progress state should remain session-scoped rather than being persisted to the database or temp files. - Persistence should prefer the existing `FindByJCode` + mutate + `JurisdictionRepository.Update` flow before introducing a new repository bulk-update abstraction. - Header binding should validate the expected row-2 header names and then resolve columns by header names rather than raw column positions only. - The final import summary should include at minimum: total rows seen, rows updated, rows inserted, duplicate rows encountered, invalid rows skipped, and row-level failures. - The final result should also expose row-level operator feedback with row number, failure reason, and full imported record details for malformed `Jurisdiction` values, `IMB_Digits` build failures, ACE read failures, and other row-level processing errors. ## Implementation Plan ### Tasks - [x] Task 1: Replace the current jurisdiction import contract with XLSX-only validation - File: `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` - Action: Update `ImportPost` so it only accepts `.xlsx` uploads, removes the CSV/TXT-specific text-driver path, and returns a clear validation error for any non-XLSX file. - Notes: Keep the existing `FreeASPUpload` flow for multipart upload handling unless the ACE-based workbook read requires a different saved-file path convention. - [x] Task 2: Add workbook-reading and header-validation logic for the known spreadsheet format - File: `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` - Action: Read the uploaded `.xlsx` workbook through `Microsoft.ACE.OLEDB.12.0`, target the first worksheet, ignore row 1, treat row 2 as headers, and validate the required headers before row processing begins. - Notes: Fail fast with a user-facing error if ACE is unavailable, the workbook cannot be opened, or required headers are missing. - [x] Task 3: Implement row-mapping and normalization rules for jurisdiction updates - File: `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` - Action: For each workbook row, extract `JCode` from the `Jurisdiction` value inside parentheses, derive normalized `Name` from the text before parentheses, apply the `CITY` to `CITY OF ...` rewrite when appropriate, join `City & Township` and `ZIP + 4` into `CSZ`, compute `IMB_Digits`, and regenerate `IMB` via `GetIMBCodec.EncodeDigits(...)`. - Notes: Trim whitespace on all derived values and treat malformed or unparseable rows as row-level failures instead of fatal process errors. - [x] Task 4: Persist updates by existing `JCode` and track import result counters - File: `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` - File: `/workspace/App/DomainModels/JurisdictionRepository.asp` - Action: For each valid parsed row, load the existing jurisdiction by `JCode`, mutate the model fields, and call `JurisdictionRepository.Update`; insert missing `JCode` rows with the parsed values; detect duplicates in workbook order and allow the last valid row to win. - Notes: Only add repository support if a small helper materially reduces controller duplication; otherwise stay with the existing `FindByJCode` + `Update` pattern. - [x] Task 5: Introduce bounded progress-state tracking and a polling endpoint - File: `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` - Action: Split the import flow into a kickoff path and a progress/status path, storing progress state in `Session` under an import token and returning JSON-compatible status for AJAX polling. - Notes: If Classic ASP session locking prevents true live row-by-row progress, expose checkpoint-based phases such as `uploaded`, `opening workbook`, `validating headers`, `processing rows`, and `complete`. - [x] Task 6: Replace the import page UI with XLSX-only upload and progress display - File: `/workspace/App/Views/Jurisdiction/import.asp` - Action: Update the form copy and file input to XLSX-only, add client-side AJAX kickoff/polling behavior, render a progress bar/status area, and display final summary counts plus row-level error details. - Notes: The UX should remain usable if progress is phase-based rather than exact per-row percentage. - [x] Task 7: Define the final operator summary and failure reporting - File: `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` - File: `/workspace/App/Views/Jurisdiction/import.asp` - Action: Return or render a completion result that includes total rows seen, rows updated, rows inserted, duplicate rows encountered, invalid rows skipped, row-level failures, and row-specific error details. - Notes: Include row number, failure reason, and full record details for malformed `Jurisdiction` values, IMB digit failures, insert/update failures, and workbook-read errors where applicable. - [x] Task 8: Update user-facing messaging and implementation notes for runtime dependency risk - File: `/workspace/App/Views/Jurisdiction/import.asp` - File: `/workspace/App/Controllers/Jurisdiction/JurisdictionController.asp` - Action: Make sure operator-facing errors clearly describe ACE dependency issues and workbook validation failures. - Notes: This keeps deployment/runtime failures diagnosable without digging through server logs first. ### Acceptance Criteria - [x] AC 1: Given the user opens the jurisdiction import page, when the screen renders, then it only advertises and accepts `.xlsx` uploads. - [x] AC 2: Given a non-`.xlsx` file is submitted, when the import starts, then the user receives a validation error and no jurisdiction rows are changed. - [x] AC 3: Given the uploaded workbook cannot be opened through ACE/OLEDB, when the import starts, then the user receives a clear workbook/ACE error and no jurisdiction rows are changed. - [x] AC 4: Given the workbook is missing one of the required row-2 headers, when validation runs, then the import stops before row processing and reports the missing header(s). - [x] AC 5: Given a row contains `Jurisdiction` text in the format `ALCONA TOWNSHIP (01040)`, when the row is processed, then `JCode` is parsed as `01040` and `Name` is parsed as `ALCONA TOWNSHIP`. - [x] AC 6: Given a parsed jurisdiction name ends with the standalone word `CITY`, when normalization runs, then the stored `Name` becomes `CITY OF `. - [x] AC 7: Given a workbook row has `City & Township` and `ZIP + 4` values, when the row is processed, then the database `CSZ` field is updated with those values joined in the same import flow. - [x] AC 8: Given a valid workbook row matches an existing jurisdiction by extracted `JCode`, when the row is processed, then `Name`, `Mailing_Address`, `CSZ`, `IMB_Digits`, and `IMB` are updated in the `Jurisdiction` table. - [x] AC 9: Given a valid workbook row is processed, when `IMB_Digits` is derived, then `IMB` is regenerated using the existing `GetIMBCodec.EncodeDigits(...)` functionality. - [x] AC 10: Given a workbook row has an extracted `JCode` that does not exist in the database, when the row is processed, then a new `Jurisdiction` row is inserted with the parsed/imported values. - [x] AC 11: Given the same `JCode` appears more than once in the workbook, when processing completes, then the last valid occurrence wins and duplicates are counted in the final summary. - [x] AC 12: Given a row has malformed `Jurisdiction` text that cannot produce a valid `JCode`, when the row is processed, then that row is counted as failed or invalid, the import continues with subsequent rows, and the final output includes the full imported record details. - [x] AC 13: Given the user starts an import, when the operation is in progress, then the page shows AJAX-driven progress feedback for the current import token. - [x] AC 14: Given true live row-by-row progress is not technically reliable under Classic ASP/IIS, when the import runs, then the user still sees checkpoint-based progress states instead of a frozen screen. - [x] AC 15: Given the import completes, when final results are shown, then the user sees total rows seen, rows updated, rows inserted, duplicates encountered, invalid rows skipped, and failures. - [x] AC 16: Given any malformed `Jurisdiction` or `IMB_Digits` build failures occur, when results are shown, then the user can see row numbers, failure reasons, and the full source record for those rows. ## Additional Context ### Dependencies - Windows/IIS runtime with Classic ASP enabled - `FreeASPUpload` remains the upload mechanism unless implementation constraints force a localized alternative - `Microsoft.ACE.OLEDB.12.0` must be installed and available on the target server for workbook access - Existing `GetIMBCodec.EncodeDigits(...)` functionality must remain available during import processing - Existing MVC JSON helper in `/workspace/MVC/lib.json.asp` can support polling responses ### Testing Strategy - Manual IIS test: upload a valid copy of [BRM Permit Info February '26 updated.xlsx](/workspace/Data/BRM Permit%20Info%20February%20'26%20updated.xlsx) and verify updated jurisdiction rows in the database - Manual IIS test: upload a non-XLSX file and verify validation failure - Manual IIS test: upload a workbook with one required header removed and verify fast-fail header validation - Manual IIS test: upload a workbook containing malformed `Jurisdiction` values and verify row-level error reporting with continued processing and full source-record details - Manual IIS test: upload a workbook containing missing `JCode` rows and verify the rows are inserted into `Jurisdiction` - Manual IIS test: upload a workbook containing invalid `Mailer ID Option 1` or `ZIP + 4` values and verify the full source record is shown in the error output - Manual IIS test: upload a workbook containing duplicate `JCode` rows and verify the last valid row wins - Manual IIS test: verify progress bar/status polling updates during import and resolves to a final summary state - Optional helper-level verification: isolate and test any extracted parsing/normalization helper routines if they are moved out of the controller into reusable VBScript functions ### Notes - Highest implementation risk is the interaction between Classic ASP request lifecycle, Session locking, and AJAX polling; the implementation should prefer a reliable bounded progress model over an unreliable “live” illusion. - ACE/OLEDB availability is a deployment/runtime dependency, not just a coding detail; missing-provider handling must be first-class. - Keep the change bounded to the existing jurisdiction import workflow and avoid turning this into a generalized spreadsheet-import framework. - If the controller becomes too large, extraction into small local helper functions is acceptable, but avoid broad architectural refactors.