diff --git a/docs/kci-processing-jobs/README.md b/docs/kci-processing-jobs/README.md new file mode 100644 index 0000000..84b949a --- /dev/null +++ b/docs/kci-processing-jobs/README.md @@ -0,0 +1,34 @@ +# KCI Processing Jobs Documentation + +This folder documents KCI jobs and processes used to process files, PDFs, imports, exports, scheduled work, cleanup tasks, and related operational workflows. + +The goal is to make each job understandable, supportable, verifiable, and safer to change. + +## Start Here + +- [Job Inventory](job-inventory.md) +- [Job Runbook Template](job-runbook-template.md) +- [File Processing Conventions](file-processing-conventions.md) +- [Troubleshooting Guide](troubleshooting-guide.md) + +## Job Runbooks + +- [Mail Manager](jobs/mail-manager.md) +- [Purple Envelope](jobs/purple-envelope.md) + +## Documentation Rules + +- Do not store passwords, credentials, private keys, connection strings, or sensitive customer data in these files. +- Mark unknowns clearly instead of guessing. +- Separate environment-specific details when production, test, and local behavior differ. +- For every job, define how success is verified. +- For file-processing jobs, document input locations, output locations, naming conventions, retention, and reprocessing rules. + +## Suggested Documentation Flow + +1. Add the job to [Job Inventory](job-inventory.md). +2. Copy [Job Runbook Template](job-runbook-template.md) into `jobs/.md`. +3. Fill in verified facts first. +4. Mark unresolved items as `Unknown`. +5. Review security notes before committing. +6. Have someone familiar with the job validate the runbook. diff --git a/docs/kci-processing-jobs/file-processing-conventions.md b/docs/kci-processing-jobs/file-processing-conventions.md new file mode 100644 index 0000000..1e58664 --- /dev/null +++ b/docs/kci-processing-jobs/file-processing-conventions.md @@ -0,0 +1,88 @@ +# File Processing Conventions + +Use this file to document conventions shared by KCI processing jobs. + +Mark unknowns instead of guessing. + +## File Locations + +| Location Type | Path or Description | Environment | Notes | +| --- | --- | --- | --- | +| Input | Unknown | Unknown | Unknown | +| Output | Unknown | Unknown | Unknown | +| Archive | Unknown | Unknown | Unknown | +| Error / reject | Unknown | Unknown | Unknown | +| Logs | Unknown | Unknown | Unknown | + +## File Naming + +Document any naming conventions used by processing jobs. + +Examples to fill in: + +- Prefix: +- Date format: +- Sequence number: +- Customer/account identifier: +- Extension: +- Temporary file naming: +- Completed file naming: +- Error file naming: + +## File States + +Recommended state model: + +```text +received +validated +processing +processed +archived +rejected +error +``` + +Document the actual KCI state model when known. + +## PDF Processing Notes + +Capture shared PDF rules here: + +- Expected PDF source: +- Required page size: +- Required orientation: +- Naming rules: +- Merge/split rules: +- OCR rules: +- Validation rules: +- Output destination: + +## Retention + +| File Type | Retention Period | Cleanup Method | Owner | +| --- | --- | --- | --- | +| Input files | Unknown | Unknown | Unknown | +| Output files | Unknown | Unknown | Unknown | +| Logs | Unknown | Unknown | Unknown | +| Error files | Unknown | Unknown | Unknown | + +## Reprocessing Principles + +Before rerunning a job, identify: + +- original input file +- output files from the failed or partial run +- database records created or changed +- archive state +- downstream systems already notified +- duplicate risks + +Do not rerun a job until duplicate and partial-processing risks are understood. + +## Security And Privacy + +- Do not put customer data samples in documentation unless they are sanitized. +- Do not commit credentials or connection strings. +- Document where configuration lives without exposing secret values. +- Avoid copying full production paths if they reveal sensitive structure; use environment labels when needed. diff --git a/docs/kci-processing-jobs/job-inventory.md b/docs/kci-processing-jobs/job-inventory.md new file mode 100644 index 0000000..038d790 --- /dev/null +++ b/docs/kci-processing-jobs/job-inventory.md @@ -0,0 +1,32 @@ +# KCI Processing Job Inventory + +Use this file as the high-level map of KCI processing jobs. + +Do not put full procedures here. Link to the detailed runbook for each job. + +| Job | Purpose | Trigger | Schedule | Environment | Owner | Runbook | Status | +| --- | --- | --- | --- | --- | --- | --- | --- | +| Mail Manager | Unknown | Unknown | Unknown | Unknown | Unknown | [Runbook](jobs/mail-manager.md) | Draft | +| Purple Envelope | Unknown | Unknown | Unknown | Unknown | Unknown | [Runbook](jobs/purple-envelope.md) | Draft | + +## Status Values + +- `Draft`: runbook exists but facts are incomplete. +- `Verified`: runbook was reviewed by someone familiar with the job. +- `Needs Review`: job has changed or important facts are uncertain. +- `Deprecated`: job no longer runs but is kept for history. + +## Inventory Checklist + +For each job, confirm: + +- [ ] Purpose is known. +- [ ] Trigger is known. +- [ ] Schedule is known. +- [ ] Runtime location is known. +- [ ] Inputs are documented. +- [ ] Outputs are documented. +- [ ] Success criteria are documented. +- [ ] Failure handling is documented. +- [ ] Reprocessing rules are documented. +- [ ] Owner or escalation contact is documented. diff --git a/docs/kci-processing-jobs/job-runbook-template.md b/docs/kci-processing-jobs/job-runbook-template.md new file mode 100644 index 0000000..758ca35 --- /dev/null +++ b/docs/kci-processing-jobs/job-runbook-template.md @@ -0,0 +1,144 @@ +# Job Name + +Status: Draft + +## Purpose + +What business or operational outcome does this job produce? + +## Owner + +- Primary owner: +- Backup owner: +- Escalation contact: + +## Summary + +Short explanation of what the job does in plain language. + +## Trigger + +How does the job start? + +- Scheduled task: +- SQL Agent job: +- Service: +- Script: +- Manual action: +- Application-triggered: +- Other: + +## Schedule + +- Frequency: +- Expected start time: +- Expected duration: +- Time zone: +- Blackout windows: + +## Runtime Location + +- Server/workstation: +- Application: +- Script path: +- Working directory: +- Account/context: + +Do not document passwords or secrets. + +## Inputs + +| Input | Location | Format | Required | Notes | +| --- | --- | --- | --- | --- | +| Unknown | Unknown | Unknown | Unknown | Unknown | + +## Outputs + +| Output | Location | Format | Recipient/System | Notes | +| --- | --- | --- | --- | --- | +| Unknown | Unknown | Unknown | Unknown | Unknown | + +## Dependencies + +- File shares: +- Databases: +- External systems: +- Printers: +- Email: +- Third-party tools: +- Permissions: +- Network access: + +## Processing Steps + +Document the expected flow. + +```text +INPUT + ↓ +VALIDATION + ↓ +PROCESSING + ↓ +OUTPUT + ↓ +ARCHIVE / CLEANUP + ↓ +VERIFICATION +``` + +## Success Criteria + +How do we know the job worked? + +- Expected files created: +- Expected records updated: +- Expected logs: +- Expected notifications: +- Expected downstream result: + +## Failure Modes + +| Failure | Symptom | Likely Cause | First Check | Escalation | +| --- | --- | --- | --- | --- | +| Unknown | Unknown | Unknown | Unknown | Unknown | + +## Troubleshooting + +1. Check whether the job started. +2. Check the latest log. +3. Check input files exist and match expected naming/format. +4. Check permissions and file locks. +5. Check downstream/output location. +6. Check whether the job can be safely rerun. + +## Reprocessing Rules + +- Can this job be rerun safely? +- What must be cleaned up before rerun? +- What files or records indicate a partial run? +- What duplicate risks exist? +- Who approves reprocessing? + +## Retention / Cleanup + +- Input retention: +- Output retention: +- Archive location: +- Cleanup schedule: +- Backup requirements: + +## Security Notes + +- Sensitive data handled: +- Access restrictions: +- Secrets/configuration location: +- Logging restrictions: + +Do not include secrets in this runbook. + +## Change History + +| Date | Change | Author | +| --- | --- | --- | +| YYYY-MM-DD | Initial draft | Unknown | diff --git a/docs/kci-processing-jobs/jobs/mail-manager.md b/docs/kci-processing-jobs/jobs/mail-manager.md new file mode 100644 index 0000000..b4d71a9 --- /dev/null +++ b/docs/kci-processing-jobs/jobs/mail-manager.md @@ -0,0 +1,146 @@ +# Mail Manager + +Status: Draft + +## Purpose + +Unknown. + +Initial note: this job/process may need scheduled backup and/or deletion of Mail Manager jobs. + +## Owner + +- Primary owner: Unknown +- Backup owner: Unknown +- Escalation contact: Unknown + +## Summary + +Unknown. + +## Trigger + +- Scheduled task: Unknown +- SQL Agent job: Unknown +- Service: Unknown +- Script: Unknown +- Manual action: Unknown +- Application-triggered: Unknown +- Other: Unknown + +## Schedule + +- Frequency: Unknown +- Expected start time: Unknown +- Expected duration: Unknown +- Time zone: Unknown +- Blackout windows: Unknown + +## Runtime Location + +- Server/workstation: Unknown +- Application: Unknown +- Script path: Unknown +- Working directory: Unknown +- Account/context: Unknown + +Do not document passwords or secrets. + +## Inputs + +| Input | Location | Format | Required | Notes | +| --- | --- | --- | --- | --- | +| Mail Manager jobs | Unknown | Unknown | Unknown | Need to define source and selection criteria. | + +## Outputs + +| Output | Location | Format | Recipient/System | Notes | +| --- | --- | --- | --- | --- | +| Backup/archive | Unknown | Unknown | Unknown | Need to define backup destination and retention. | + +## Dependencies + +- File shares: Unknown +- Databases: Unknown +- External systems: Unknown +- Permissions: Unknown +- Network access: Unknown + +## Processing Steps + +Draft expected flow: + +```text +identify eligible Mail Manager jobs + ↓ +backup/archive selected jobs + ↓ +verify backup exists and is usable + ↓ +delete or mark original jobs according to retention rule + ↓ +log result +``` + +## Success Criteria + +- Selected jobs are backed up before deletion. +- Backup location contains expected job data. +- Deleted jobs match the approved criteria. +- Job history/logs show success. +- No unexpected jobs are removed. + +## Failure Modes + +| Failure | Symptom | Likely Cause | First Check | Escalation | +| --- | --- | --- | --- | --- | +| Backup fails | Jobs are not archived | Path, permissions, disk space, lock | Check backup destination and logs | Unknown | +| Delete criteria wrong | Too many or too few jobs selected | Missing retention rule | Review selection query/path/filter | Unknown | +| Scheduled task fails | Cleanup does not run | Task disabled, credentials, path | Check task history | Unknown | + +## Troubleshooting + +1. Check scheduled task history. +2. Confirm source job location. +3. Confirm backup location is reachable. +4. Confirm selection/deletion criteria. +5. Confirm backup verification method. +6. Confirm whether partial cleanup occurred. + +## Reprocessing Rules + +- Can this job be rerun safely? Unknown. +- What must be cleaned up before rerun? Unknown. +- What duplicate risks exist? Unknown. +- Who approves reprocessing? Unknown. + +## Retention / Cleanup + +- Input retention: Unknown +- Output retention: Unknown +- Archive location: Unknown +- Cleanup schedule: Unknown +- Backup requirements: Unknown + +## Security Notes + +- Sensitive data handled: Unknown +- Access restrictions: Unknown +- Secrets/configuration location: Unknown +- Logging restrictions: Do not log credentials or sensitive content. + +## Open Questions + +- What exactly counts as a Mail Manager job? +- Where are Mail Manager jobs stored? +- What should be backed up? +- Where should backups go? +- What retention rule determines deletion? +- Should deletion run automatically or require manual approval at first? +- How should backup integrity be verified? + +## Change History + +| Date | Change | Author | +| --- | --- | --- | +| 2026-09-01 | Initial draft scaffold | Codex | diff --git a/docs/kci-processing-jobs/jobs/purple-envelope.md b/docs/kci-processing-jobs/jobs/purple-envelope.md new file mode 100644 index 0000000..809b2b5 --- /dev/null +++ b/docs/kci-processing-jobs/jobs/purple-envelope.md @@ -0,0 +1,151 @@ +# Purple Envelope + +Status: Draft + +## Purpose + +Unknown. + +Initial note: Loc needs to run the Purple Envelope jobs, and the process has unresolved weirdness that needs diagnosis. + +## Owner + +- Primary owner: Unknown +- Backup owner: Unknown +- Escalation contact: Unknown +- Team member involved: Loc + +## Summary + +Unknown. + +## Trigger + +- Scheduled task: Unknown +- SQL Agent job: Unknown +- Service: Unknown +- Script: Unknown +- Manual action: Unknown +- Application-triggered: Unknown +- Other: Unknown + +## Schedule + +- Frequency: Unknown +- Expected start time: Unknown +- Expected duration: Unknown +- Time zone: Unknown +- Blackout windows: Unknown + +## Runtime Location + +- Server/workstation: Unknown +- Application: Unknown +- Script path: Unknown +- Working directory: Unknown +- Account/context: Unknown + +Do not document passwords or secrets. + +## Inputs + +| Input | Location | Format | Required | Notes | +| --- | --- | --- | --- | --- | +| Unknown | Unknown | Unknown | Unknown | Need to identify files or batches processed by Purple Envelope. | + +## Outputs + +| Output | Location | Format | Recipient/System | Notes | +| --- | --- | --- | --- | --- | +| Unknown | Unknown | Unknown | Unknown | Need to identify expected output. | + +## Dependencies + +- File shares: Unknown +- Databases: Unknown +- External systems: Unknown +- PDF tooling: Unknown +- Permissions: Unknown +- Network access: Unknown +- Team member access/setup: Loc + +## Processing Steps + +Draft expected flow: + +```text +receive or identify Purple Envelope input + ↓ +run expected job/process + ↓ +produce output + ↓ +verify output + ↓ +archive or hand off result +``` + +## Success Criteria + +- Loc can start the expected Purple Envelope job. +- The job runs without unexpected errors. +- Expected output is created. +- Output is validated. +- Any downstream handoff completes. +- The weird behavior is either resolved or documented with a known workaround. + +## Failure Modes + +| Failure | Symptom | Likely Cause | First Check | Escalation | +| --- | --- | --- | --- | --- | +| Loc cannot run job | Job does not start or fails immediately | Access, setup, permissions, path, missing dependency | Compare Loc setup to known-good setup | Unknown | +| Weird process behavior | Unexpected output or inconsistent run | Unknown | Capture exact step, input, output, and logs | Unknown | +| Output missing | Job completes but files are absent | Wrong destination, permission, failed processing | Check logs and output path | Unknown | + +## Troubleshooting + +1. Ask Loc to capture the exact failure point. +2. Record the input/batch used. +3. Record the command, UI step, or job trigger Loc used. +4. Compare Loc's environment/access to a known-good environment. +5. Check logs for the same time window. +6. Confirm expected output path and file naming. +7. Decide whether this is a setup issue, data issue, process issue, or code issue. + +## Reprocessing Rules + +- Can this job be rerun safely? Unknown. +- What must be cleaned up before rerun? Unknown. +- What duplicate risks exist? Unknown. +- Who approves reprocessing? Unknown. + +## Retention / Cleanup + +- Input retention: Unknown +- Output retention: Unknown +- Archive location: Unknown +- Cleanup schedule: Unknown +- Backup requirements: Unknown + +## Security Notes + +- Sensitive data handled: Unknown +- Access restrictions: Unknown +- Secrets/configuration location: Unknown +- Logging restrictions: Do not log credentials or sensitive content. + +## Open Questions + +- What does Purple Envelope process? +- What command, application, or scheduled job runs it? +- What does Loc need in order to run it? +- What is the exact weirdness? +- Is there a known-good machine/user/process for comparison? +- What output proves the job succeeded? +- Can the job be rerun safely after a failed attempt? + +## Change History + +| Date | Change | Author | +| --- | --- | --- | +| 2026-09-01 | Initial draft scaffold | Codex | diff --git a/docs/kci-processing-jobs/troubleshooting-guide.md b/docs/kci-processing-jobs/troubleshooting-guide.md new file mode 100644 index 0000000..5a41421 --- /dev/null +++ b/docs/kci-processing-jobs/troubleshooting-guide.md @@ -0,0 +1,87 @@ +# KCI Processing Jobs Troubleshooting Guide + +Use this guide when a processing job fails, behaves strangely, or produces unexpected output. + +## First Questions + +1. Which job failed? +2. Which environment failed? +3. When did it last succeed? +4. What input file or batch was involved? +5. Did the job start, partially run, or never start? +6. What changed recently? + +## Initial Checks + +- Confirm the trigger ran. +- Check the job history or scheduled task history. +- Check logs for the failed time window. +- Confirm input files exist. +- Confirm files are not locked. +- Confirm expected output location is reachable. +- Confirm the job account has required permissions. +- Confirm disk space is sufficient. +- Confirm required downstream systems are available. + +## File Processing Checks + +- File name matches expected pattern. +- File extension is expected. +- File size is reasonable. +- File is not empty. +- File is not still being copied. +- File encoding or format is expected. +- PDF can be opened and is not corrupt. +- Required pages, forms, or records are present. + +## Scheduled Task Checks + +- Task is enabled. +- Trigger is correct. +- Last run time matches expectation. +- Last run result is understood. +- Run account is correct. +- Working directory is correct. +- Script path is correct. +- Arguments are correct. +- Highest privileges setting is correct when needed. +- Network paths are accessible from the task account. + +## Safe Recovery + +Before rerunning: + +- identify whether output was partially created +- identify whether database records were changed +- identify whether files were moved or archived +- identify whether downstream notifications were sent +- confirm duplicate-processing risk +- confirm who approves the rerun + +## Escalation Notes + +Escalate when: + +- customer-facing output is wrong +- production data may be duplicated or missing +- credentials or permissions appear broken +- the job is repeatedly failing +- the recovery path is unclear +- rerunning could create duplicates + +## Incident Notes Template + +```text +Date/time: +Job: +Environment: +Input/batch: +Observed symptom: +Expected result: +Last known successful run: +Recent changes: +Checks performed: +Current state: +Next action: +Owner: +```