From 4c71f94d96a7d029a105e51dfd1330767f5099e7 Mon Sep 17 00:00:00 2001 From: Daniel Covington Date: Tue, 10 Nov 2026 13:56:59 -0500 Subject: [PATCH] Make the C128 Vim the default editor and prove the source round-trip (E07-S02, E07-S03). The editor launcher now copies the Vim disk, injects vim128.prg into the copy, and autostarts that disk copy in x128 80-column mode; host-loaded editor sessions reported success without writing to the image. E07-S03 adds a C128 tokenizer fixture and a two-session test that creates a source in Vim, reopens it, and tokenizes the disk file without host conversion. Generated with Codebuff Co-Authored-By: Codebuff --- newLang/docs/architecture.md | 4 +- newLang/docs/decisions.md | 2 + newLang/docs/editor-workflow.md | 20 +- newLang/planning/claims/E07-S02.md | 112 +++++----- newLang/planning/claims/E07-S03.md | 97 +++++++++ newLang/planning/epics-and-stories.md | 3 +- newLang/scripts/build-tokenize-seq.ps1 | 26 +++ newLang/scripts/launch-editor.ps1 | 39 +++- newLang/scripts/lib/Nova128Launcher.psm1 | 63 +++++- newLang/tests/disk/launchers.tests.ps1 | 6 +- newLang/tests/disk/tokenize-seq.s | 234 +++++++++++++++++++++ newLang/tests/disk/vim-roundtrip.tests.ps1 | 186 ++++++++++++++++ 12 files changed, 711 insertions(+), 81 deletions(-) create mode 100644 newLang/planning/claims/E07-S03.md create mode 100644 newLang/scripts/build-tokenize-seq.ps1 create mode 100644 newLang/tests/disk/tokenize-seq.s create mode 100644 newLang/tests/disk/vim-roundtrip.tests.ps1 diff --git a/newLang/docs/architecture.md b/newLang/docs/architecture.md index 06de1e9..54bb6a0 100644 --- a/newLang/docs/architecture.md +++ b/newLang/docs/architecture.md @@ -893,8 +893,8 @@ files. Version 1 does not link the editor and compiler in memory. The development image workflow is strictly sequential: 1. `scripts/launch-editor.ps1` copies `../vim/vim-improved.d81` to - `build/newlang-edit.d81`, launches `x64sc` and autostarts - `../vim/vim128.prg`; + `build/newlang-edit.d81`, writes `../vim/vim128.prg` into the copy, and + launches `x128` in 80-column mode with that C128 editor (D151); 2. the script returns only after the emulator exits and the image is flushed, releasing its session lock; 3. `scripts/launch-compiler.ps1` prepares `build/newlang-dev.d81` (unless diff --git a/newLang/docs/decisions.md b/newLang/docs/decisions.md index a656a40..3002ec6 100644 --- a/newLang/docs/decisions.md +++ b/newLang/docs/decisions.md @@ -83,6 +83,8 @@ documented default. `Proposed` means a later change still needs approval. | D148 | E07-S02 puts both emulator launchers on one shared implementation, `scripts/lib/Nova128Launcher.psm1`. `Enter-Nova128ImageSession` takes an exclusive `.newlang-session.lock` beside the writable image, replaces a stale lock only when its PID is gone, and refuses when the holder is alive or the lock cannot even be read; `Wait-Nova128ImageFlush` reopens the image exclusively with retries so a launcher never returns while the drive is still writing; `Invoke-Nova128Emulator` starts the emulator, waits for exit, then waits for flush and releases the lock. `scripts/launch-editor.ps1` uses `x64sc` with `../vim/vim128.prg` and copies `../vim/vim-improved.d81` to `build/newlang-edit.d81`; `scripts/launch-compiler.ps1` uses `x128` with `build/n128c.prg` and prepares the development image unless `-SkipPrepare`. Both autostart a host-side program path, so no viewer binary has to be on the disk, and both accept `-LimitCycles`, `-Warp` and `-Screenshot`. | The story's rule is that the editor and the compiler never hold the same writable disk at once, which is only reliable if both launchers implement the lock and the flush wait identically. A lock file beside the image is detectable from either script and from a test without a shared daemon; an unreadable lock counts as held because an exclusive holder denies reads too. Copying the editor disk per session keeps `../vim/vim-improved.d81` a read-only original, matching E07-S01's rule for the compiler disk. `x64sc` names the VIC-II exit screenshot `-exitscreenshot` while `x128` uses `-exitscreenshotvicii`, so the option is chosen by emulator name. `docs/editor-workflow.md` carries the user-facing order and recovery rules, and `tests/disk/launchers.tests.ps1` proves the lock, the flush wait, both autostarts and that the source disk hash is unchanged. | +| D151 | The editor workflow now defaults to the C128 builds: `scripts/launch-editor.ps1` starts `x128` in 80-column mode with `../vim/vim128.prg`, injected as `vim128` into the session's copy of `../vim/vim-improved.d81`, and autostarts that disk copy. This supersedes the `x64sc` editor default in E07-S02's acceptance text; the C64 editor build and its disk remain available through explicit arguments. Because `x128`'s plain `-exitscreenshot` captures the active display (VDC in 80-column mode) while `-exitscreenshotvicii` captures the 40-column screen, the shared launcher selects the option by emulator and mode. The E07-S03 round-trip test now creates, reopens and tokenizes the source through `vim128` in `x128`, so the whole path stays on the target machine. The editor must be autostarted from the session disk (`work.d81:vim128`, with the launcher running in the image's folder), not from a host path: a host-injected program has no device number from its load, and `vim128` then reports `written` while the image never receives the file — observed while validating the round-trip, which only passed once the disk copy was autostarted. | The user asked for the C128 Vim to be the default editor everywhere. The C128 build was already present as `../vim/vim128.prg` (80-column, built by `vim/build128.bat`) but no shipped `.d81` held it; injecting it into the per-session disk copy makes the editor available to the session's own disk access without writing `../vim` or adding a new reference image. Capturing the VDC screen is required for automated checks: a VIC-II capture of an 80-column session is blank. Keeping the C64 build reachable as an argument preserves the proven fallback instead of deleting working behaviour. | + ## Deferred - Native C128/80-column Vim port diff --git a/newLang/docs/editor-workflow.md b/newLang/docs/editor-workflow.md index 6282536..8d3c084 100644 --- a/newLang/docs/editor-workflow.md +++ b/newLang/docs/editor-workflow.md @@ -26,17 +26,25 @@ its kind, and a lock left behind by a dead process is replaced automatically. powershell -NoProfile -ExecutionPolicy Bypass -File scripts\launch-editor.ps1 ``` -- Autostarts `../vim/vim128.prg` in `x64sc` with the disk image attached as a - true drive. - Copies `../vim/vim-improved.d81` to `build/newlang-edit.d81` first, so the - editor never opens the source image. -- Waits for `x64sc` to exit, then waits until the image can be opened + editor never opens the source image, then writes `../vim/vim128.prg` into the + copy as `vim128`. +- Starts `x128` in 80-column mode and autostarts `vim128` **from the session + disk**, then attaches that disk as drive 8. Loading the editor from the disk + is what gives it a write device: a host-loaded copy reports `written` and + leaves the image unchanged, because it has no device number from the load. + The 80-column session draws on the VDC, so an automated `-Screenshot` uses + VICE's plain `-exitscreenshot`; the VIC-II option would capture a blank + 40-column screen. +- Waits for `x128` to exit, then waits until the image can be opened exclusively (the image has flushed) before returning. - Verifies the source image's SHA-256 is unchanged. Use `-EditorPrg` and `-SourceImage` to select another editor build or disk. -`-LimitCycles` and `-Screenshot` are for automated checks only; they end the -session early. +`-Emulator x64sc` selects the C64 build instead, but that program is on +`vim-improved.d81` rather than a standalone PRG, so pass `-EditorPrg` with the +extracted `vim` file. `-LimitCycles` and `-Screenshot` are for automated checks +only; they end the session early. ## Compiler session diff --git a/newLang/planning/claims/E07-S02.md b/newLang/planning/claims/E07-S02.md index 0a430d7..0a7c3b6 100644 --- a/newLang/planning/claims/E07-S02.md +++ b/newLang/planning/claims/E07-S02.md @@ -11,87 +11,87 @@ depends-on: [E07-S01, E04-S01] ## Intended change -Two launchers, one shared implementation: `scripts/launch-editor.ps1` starts -`x64sc` on a per-session copy of the Vim disk and autostarts the Vim program, -and `scripts/launch-compiler.ps1` starts `x128` and autostarts the compiler. -Both take an exclusive lock beside the writable image, wait for the emulator to -exit and for the image to be flushed before returning, and refuse a second -writable session. `docs/editor-workflow.md` states the order, recovery for a -stale lock, and the prohibition on simultaneous writable attachment. +Two launchers, one shared implementation: `scripts/launch-editor.ps1` and +`scripts/launch-compiler.ps1`. Both take an exclusive lock beside the writable +image, wait for the emulator to exit and the image to flush, and refuse a second +writable session. D151 moves the editor to the target machine: the launcher +copies `../vim/vim-improved.d81`, writes `../vim/vim128.prg` into the copy as +`vim128`, starts `x128` in 80-column mode, and autostarts the editor **from that +disk**. `docs/editor-workflow.md` states the order, stale-lock recovery and the +one-writable-session rule. ## File area -- `scripts/lib/Nova128Launcher.psm1` (new): `Get-Nova128ViceTool`, - `Enter-Nova128ImageSession`, `Exit-Nova128ImageSession`, - `Wait-Nova128ImageFlush`, `Invoke-Nova128Emulator` +- `scripts/lib/Nova128Launcher.psm1` (new): session lock, flush wait, + `Add-Nova128DiskFile`, `Invoke-Nova128Emulator` (80-column and disk-autostart + support, VDC/VIC-II screenshot choice) - `scripts/launch-editor.ps1`, `scripts/launch-compiler.ps1` (new) - `tests/disk/launchers.tests.ps1` (new) -- `docs/editor-workflow.md` (new), `docs/decisions.md` (D148), +- `docs/editor-workflow.md` (new), `docs/decisions.md` (D148, D151), `docs/architecture.md` (§10) - `planning/claims/E07-S02.md` - generated files under `build/` ## Acceptance evidence -- [x] Editor launcher uses `x64sc` and autostarts Vim: +- [x] Editor launcher uses `x128` in 80-column mode and autostarts the C128 Vim: `scripts/launch-editor.ps1` defaults to `../vim/vim128.prg`, copies - `../vim/vim-improved.d81` to `build/newlang-edit.d81`, and passes the program - path to `x64sc` for autostart. `tests/disk/launchers.tests.ps1` runs it with - `-X64Sc` against a copy of the fixture disk and asserts the emulator exits, - the session lock is released and the image stays writable afterwards. + `../vim/vim-improved.d81` to `build/newlang-edit.d81`, writes the editor into + the copy as `vim128`, and autostarts `work.d81:vim128` in `x128 -80col` from + the image's folder. `tests/disk/launchers.tests.ps1` runs it against a copy of + the fixture disk, asserts the emulator exits, the session lock is released, + the image stays writable, and `vim128` is on the session disk. - [x] Compiler launcher uses `x128` and autostarts the compiler: it defaults to `build/n128c.prg`, prepares `build/newlang-dev.d81` (copy plus injection) - unless `-SkipPrepare` is given, and autostarts the compiler in `x128`; the - test exercises the same sequence with `-SkipPrepare` on a disposable image. + unless `-SkipPrepare`, and autostarts it in `x128`; the test exercises the + same sequence with `-SkipPrepare` on a disposable image. - [x] Launchers wait for process/image flush before returning: the shared - `Exit-Nova128ImageSession` waits for emulator exit and `Wait-Nova128ImageFlush` - then reopens the image exclusively with retries; the test takes the lock in - the foreground and asserts the launcher refuses, and asserts the image is - reopenable for writing immediately after each launcher returns. + `Exit-Nova128ImageSession` waits for emulator exit and + `Wait-Nova128ImageFlush` reopens the image exclusively with retries; the test + holds the lock and asserts refusal, then asserts the image is reopenable + immediately after each launcher returns. - [x] Documentation and warnings prohibit simultaneous writable attachment: `docs/editor-workflow.md` and the launcher banners state that only one writable session may exist and that the previous session must be closed - before editing or inspecting the disk; the lock makes the rule enforced, not - merely documented. + before editing or inspecting the disk; the lock enforces it. ## Coordination notes -- Depends on E07-S01 (image preparation) and E04-S01 (container/autostart - shape); both are complete. The compiler launcher reuses E07-S01's prepare - script rather than duplicating injection. -- Autostart uses a host-side program path, so no viewer binary has to live on - the disk and `../vim` stays read-only; only the session's own image is - writable and only for drive 8. -- VICE names exit screenshots differently per emulator: `x64sc` accepts - `-exitscreenshot` while `x128` accepts `-exitscreenshotvicii`. The module - chooses by emulator name after the tests caught a missing screenshot. +- Depends on E07-S01 (image preparation) and E04-S01 (container/autostart); + both are complete. The compiler launcher reuses `prepare-disk.ps1`. +- D151 changed the editor default from `x64sc` to the C128 build at the user's + request. The C64 editor is still reachable with `-Emulator x64sc`, but its + program lives inside `vim-improved.d81`, so `-EditorPrg` must name an + extracted `vim`. +- Autostarting the editor from the session disk is required, not cosmetic: a + host-loaded copy has no device number from its load, and `vim128` then + reports `written` while the image never receives the file. +- VICE's `x64sc` plain `-exitscreenshot` is the VIC-II screen; `x128` in + 80-column mode needs the same plain option (VDC), while + `-exitscreenshotvicii` captures a blank 40-column screen. ## Work log -- The first test run failed the lock case with "cannot access the lock file" - instead of a lock message: a live holder opens the file without sharing, so - reading it can fail. An unreadable lock is now treated as held and reported as - "another Nova128 session already holds ". -- The screenshot assertion then failed for `x64sc` because `-exitscreenshotvicii` - is the `x128` spelling; the option is now selected by emulator name. +- The lock test first failed with "cannot access the lock file" instead of a + lock message: a live holder opens the file without sharing, so reading it can + fail. An unreadable lock now counts as held. +- The screenshot assertion failed for `x64sc` because + `-exitscreenshotvicii` is the `x128` spelling; the option is now chosen by + emulator and display mode. - The compiler test passed no disk, so the launcher correctly refused with - "disk image not found"; the test now copies its source image to the target - first, which is also what the real workflow does through E07-S01. + "disk image not found"; the test now copies its source image first. +- D151: the C128 editor was validated at the prompt, then its saves failed in + three distinct ways (1541 drive type, true-drive emulation, host autostart) + before the disk-autostart form worked. ## Completion -- Tests run: `tests/disk/launchers.tests.ps1` (lock refusal, flush wait, - `x64sc` autostart, `x128` autostart, source-disk hash), - `scripts/verify-lifetime-test.ps1`, `scripts/verify-parser-test.ps1`, - `git diff --check`. -- Results: the launcher suite prints `launcher tests passed`, covering the disk - copy, lock refusal, flush wait, both autostarts and an unchanged source disk - (`0FF83A621B59B312F1BAE9F1D0785421500D22FD00F833A9CA918EBCA7105983`). Both - wrappers also allow `-LimitCycles`, `-Warp` and `-Screenshot` for unattended - runs. -- Remaining risks: the lock is advisory and identifies a holder by PID, so an - unrelated process that reuses a dead holder's PID in the window before the - lock is read is treated as alive; a launcher killed from the task manager - leaves the lock for the next run to replace; and Vim's own disk writes are - only guaranteed visible after VICE exits cleanly, which the flush wait assumes - but cannot force if the emulator is killed. +- Tests run: `tests/disk/launchers.tests.ps1`, + `tests/disk/vim-roundtrip.tests.ps1`, `git diff --check`. +- Results: `launcher tests passed` covering the disk copy with `vim128` + injected, lock refusal, flush wait, and both x128 sessions; the source disk + stayed `0FF83A621B59B312F1BAE9F1D0785421500D22FD00F833A9CA918EBCA7105983`. +- Remaining risks: the lock is advisory and PID-based, so a recycled PID in the + window before the lock is read is treated as alive; a launcher killed from + outside leaves a lock for the next run to replace; and the flush wait assumes + VICE exits cleanly, which cannot be forced if the process is killed. diff --git a/newLang/planning/claims/E07-S03.md b/newLang/planning/claims/E07-S03.md new file mode 100644 index 0000000..5573ab2 --- /dev/null +++ b/newLang/planning/claims/E07-S03.md @@ -0,0 +1,97 @@ +--- +story: E07-S03 +owner: codebuff +status: done +started: 2026-11-10 +completed: 2026-11-10 +depends-on: [E07-S02, E02-S02] +--- + +# E07-S03 Claim + +## Intended change + +Prove that a Nova128 source written only in the editor is a valid PETSCII SEQ +file that the C128 compiler front end consumes without a host conversion, that +the editor itself can reopen it unchanged, and that the exact bytes match an +independently computed expectation. D151 moves this check to the target machine: +the editor is `../vim/vim128.prg` in `x128` 80-column mode, autostarted from the +session disk. + +## File area + +- `tests/disk/tokenize-seq.s` (new): load `ROUNDTRIP.N8` through `src_load`, + tokenize it, dump tokens in the `lexer-test.s` format, report through the + mailbox +- `scripts/build-tokenize-seq.ps1` (new) +- `tests/disk/vim-roundtrip.tests.ps1` (new): two editor sessions plus the + tokenizer session +- `docs/decisions.md` (D151), `docs/editor-workflow.md`, + `docs/architecture.md` (§10) +- `planning/claims/E07-S03.md` +- generated files under `build/vim-roundtrip/` + +## Acceptance evidence + +- [x] A source file created entirely in Vim tokenizes on C128 without host + conversion: the test types the whole program through VICE `keybuf` into + `vim128` (`file:` prompt, `y` to create, `i`, text, RUN/STOP, `:wq`) and then + runs the new C128 tokenizer, which loads the disk file with `src_load`, lexes + it, and reports 31 tokens that match the host reference lexer line for line. + Nothing rewrites the file between Vim and the C128. +- [x] Reopening it in Vim preserves content: a second `x128` session opens + `roundtrip.n8` and writes the buffer to `copy.n8`; the extracted bytes are + identical (107 bytes), so the editor restored the saved text exactly. +- [x] Extracted bytes match the expected PETSCII/CR fixture: the host computes + the expected bytes from the same ASCII text with `ConvertTo-Nova128Petscii` + (the mapping `prepare-disk.ps1` uses) and compares every byte; `c1541 -read` + only transfers the file. +- [x] Both supported PETSCII letter ranges are exercised: the source mixes + lowercase identifiers (`program`, `alpha`) and uppercase ones (`RoundTrip`, + `BETA`), and the test asserts the written file contains bytes in `$41-$5A` + and `$C1-$DA` before tokenizing. + +## Coordination notes + +- Depends on E07-S02 (launchers; D151 moved the editor to the C128 build) and + E02-S02 (lexer); both are complete. The tokenizer links the production + `source.s` and `lexer.s`, so it proves the reader and lexer rather than a + copy of them. +- The expected bytes are computed on the host from the same text that was + typed; the C128 side never receives a converted copy, which is the property + the story is about. +- `../vim/vim-improved.d81` and `../vim/vim128.prg` are only read; the session + works on `build/vim-roundtrip/work.d81`, and the test checks the source + image's SHA-256 is unchanged. + +## Work log + +- The first run stopped at LOADING: the session had a 40M-cycle limit and no + warp after autostart, so the editor had not loaded. The limit is now 120M + cycles with a 300 ms per-key delay, enough for loading, typing and `:wq`. +- The editor then typed the text but reported `cannot write` because drive 8 + defaulted to a 1541, which cannot write a D81. `-drive8type 1581` fixed the + device; a `-drive8truedrive` variant also failed to store the file. +- With an emulated 1581 the editor reported success and quit, yet the image had + no file: it had been autostarted from the host PRG, and a host-loaded program + has no device number from its load, so the saves went nowhere. Autostarting + `vim128` from the session disk (`work.d81:vim128`, run from the image's + folder) makes the writes land; the launcher and module now use that path. +- The 80-column session draws on the VDC, so the automated screenshot uses + VICE's plain `-exitscreenshot`; `-exitscreenshotvicii` produced a blank + 40-column capture, and `-exitscreenshotvdc` is not a VICE option. + +## Completion + +- Tests run: `tests/disk/vim-roundtrip.tests.ps1` (two `x128` editor sessions + on a copied D81, then one `x128` tokenizer session), + `tests/disk/launchers.tests.ps1`, `git diff --check`. +- Results: `vim source round-trip passed`; the editor wrote 107 bytes with both + PETSCII letter ranges, the reopened copy was byte-identical, and C128 + produced 31 tokens matching the reference lexer. The source image hash stayed + `0FF83A621B59B312F1BAE9F1D0785421500D22FD00F833A9CA918EBCA7105983`. +- Remaining risks: `keybuf` is wall-clock driven, so a heavily loaded host can + deliver keys before a prompt is ready; the test's cycle limit and per-key + delay are the guard, not a handshake. Lines longer than the editor's 80-byte + limit are split by the editor on load and cannot be reproduced by this test, + and the tokenizer only accepts the single fixed file name `ROUNDTRIP.N8`. diff --git a/newLang/planning/epics-and-stories.md b/newLang/planning/epics-and-stories.md index 3410630..a272986 100644 --- a/newLang/planning/epics-and-stories.md +++ b/newLang/planning/epics-and-stories.md @@ -580,7 +580,8 @@ integration fixtures, generated `build/` artifacts. **Acceptance criteria:** -- Editor launcher uses `x64sc` and autostarts Vim. +- Editor launcher uses `x128` in 80-column mode and autostarts the C128 Vim + (`../vim/vim128.prg`; D151 makes this the default instead of `x64sc`). - Compiler launcher uses `x128` and autostarts the compiler. - Launchers wait for process/image flush before returning. - Documentation and warnings prohibit simultaneous writable attachment. diff --git a/newLang/scripts/build-tokenize-seq.ps1 b/newLang/scripts/build-tokenize-seq.ps1 new file mode 100644 index 0000000..96c1df7 --- /dev/null +++ b/newLang/scripts/build-tokenize-seq.ps1 @@ -0,0 +1,26 @@ +# E07-S03: build the single-file tokenizer PRG and a fresh D81 for it. The +# editor-written SEQ file is added to the image after Vim creates it. +[CmdletBinding()] +param() + +$ErrorActionPreference = "Stop" +$project = Split-Path $PSScriptRoot -Parent +Import-Module (Join-Path $PSScriptRoot "lib\Nova128Test.psm1") -Force +$out = Join-Path $project "build\vim-roundtrip" +New-Item -ItemType Directory -Force -Path $out | Out-Null + +$link = Invoke-Nova128Link -Name "tokenize-seq" -BuildDir $out ` + -Config (Join-Path $project "cfg\compiler.cfg") -Sources @( + (Join-Path $project "src\compiler\mmu.s"), + (Join-Path $project "src\compiler\diskio.s"), + (Join-Path $project "src\compiler\source.s"), + (Join-Path $project "src\compiler\lexer.s"), + (Join-Path $project "tests\disk\tokenize-seq.s")) +if ((Get-Nova128Label $link.Labels "test_entry") -ne 0x1C0D) { throw "test_entry is not at `$1C0D" } + +$disk = Join-Path $out "tokenize-seq.d81" +New-Nova128Disk -Path $disk -Files @( + @{ Name = "tokenize-seq"; Path = $link.Program; Type = "prg" }) + +Write-Host ("built {0}" -f $link.Program) +Write-Host ("disk {0}" -f $disk) diff --git a/newLang/scripts/launch-editor.ps1 b/newLang/scripts/launch-editor.ps1 index 180732c..a4bcf75 100644 --- a/newLang/scripts/launch-editor.ps1 +++ b/newLang/scripts/launch-editor.ps1 @@ -1,13 +1,15 @@ -# E07-S02: autostart the Vim editor in x64sc on a disposable disk copy. +# E07-S02: autostart the editor on a disposable disk copy (D151: the C128 Vim +# in x128 80-column mode by default; -Emulator x64sc selects the C64 build). # # The session is strictly sequential: this launcher refuses to start while a -# compiler session owns the same image, waits for x64sc to exit and for the -# image to flush, and never writes the source disk it copies from. +# compiler session owns the same image, waits for the emulator to exit and for +# the image to flush, and never writes the source disk it copies from. [CmdletBinding()] param( [string]$EditorPrg = "", [string]$SourceImage = "", [string]$DiskImage = "", + [ValidateSet("x128", "x64sc")][string]$Emulator = "x128", [int]$LimitCycles = 0, [string]$Screenshot = "", [switch]$Warp @@ -19,7 +21,13 @@ $repository = Split-Path $project -Parent $build = Join-Path $project "build" Import-Module (Join-Path $PSScriptRoot "lib\Nova128Launcher.psm1") -Force -if ($EditorPrg -eq "") { $EditorPrg = Join-Path $repository "vim\vim128.prg" } +if ($EditorPrg -eq "") { + if ($Emulator -eq "x128") { + $EditorPrg = Join-Path $repository "vim\vim128.prg" + } else { + throw "the C64 editor lives on ../vim/vim-improved.d81; extract 'vim' and pass -EditorPrg" + } +} if ($SourceImage -eq "") { $SourceImage = Join-Path $repository "vim\vim-improved.d81" } if ($DiskImage -eq "") { $DiskImage = Join-Path $build "newlang-edit.d81" } $EditorPrg = [IO.Path]::GetFullPath($EditorPrg) @@ -29,7 +37,7 @@ if ($SourceImage) { $SourceImage = [IO.Path]::GetFullPath($SourceImage) } if (-not (Test-Path -LiteralPath $EditorPrg -PathType Leaf)) { throw "editor program not found: $EditorPrg (build Vim or pass -EditorPrg)" } -$null = Get-Nova128ViceTool "x64sc" +$null = Get-Nova128ViceTool $Emulator $sourceHash = $null New-Item -ItemType Directory -Force -Path (Split-Path $DiskImage -Parent) | Out-Null @@ -46,11 +54,28 @@ if ($SourceImage) { Write-Host "fresh editor disk: $DiskImage" } +# The session's own disk carries the editor, so the machine can reload it and +# the source image stays untouched. +if ($Emulator -eq "x128") { + if (-not (Test-Path -LiteralPath $DiskImage)) { + throw "the C128 editor needs a writable disk image: $DiskImage" + } + Add-Nova128DiskFile -Image $DiskImage -HostPath $EditorPrg -DiskName "vim128" -Type prg + Write-Host "injected vim128 into $DiskImage" + # The editor is then started from the disk copy, not from the host path: + # a host-loaded program has no device number, and vim128's saves would + # report success without reaching the image. + $EditorPrg = "vim128" +} + $session = $null try { $session = Enter-Nova128ImageSession -Image $DiskImage -Kind "editor" - $null = Invoke-Nova128Emulator -Emulator "x64sc" -Program $EditorPrg ` - -DiskImage $DiskImage -Screenshot $Screenshot -LimitCycles $LimitCycles -Warp:$Warp + $session = $session + $null = Invoke-Nova128Emulator -Emulator $Emulator -Program $EditorPrg ` + -DiskImage $DiskImage -Screenshot $Screenshot -LimitCycles $LimitCycles ` + -Warp:$Warp -EightyColumn:($Emulator -eq "x128") ` + -ProgramOnDisk:($Emulator -eq "x128") } finally { if ($session) { Exit-Nova128ImageSession -Session $session } } diff --git a/newLang/scripts/lib/Nova128Launcher.psm1 b/newLang/scripts/lib/Nova128Launcher.psm1 index 59b050c..80f35c1 100644 --- a/newLang/scripts/lib/Nova128Launcher.psm1 +++ b/newLang/scripts/lib/Nova128Launcher.psm1 @@ -97,6 +97,30 @@ function Wait-Nova128ImageFlush { # Start an emulator and wait for it to exit. Autostart uses the host program # path, so the program is deterministic; the disk image stays attached for the # session's own reads and writes. +# Write one file into a disk image and prove it appears in the directory. +function Add-Nova128DiskFile { + param( + [Parameter(Mandatory)][string]$Image, + [Parameter(Mandatory)][string]$HostPath, + [Parameter(Mandatory)][string]$DiskName, + [ValidateSet("prg", "seq")][string]$Type = "prg" + ) + $c1541 = Get-Nova128ViceTool "c1541" + if (-not (Test-Path -LiteralPath $HostPath -PathType Leaf)) { + throw "file not found: $HostPath" + } + $destination = if ($Type -eq "seq") { "$DiskName,s" } else { $DiskName } + $output = (& $c1541 $Image -write $HostPath $destination 2>&1) -join "`n" + if ($LASTEXITCODE -ne 0) { + throw "c1541 could not write '$DiskName' to $Image`n$output" + } + $listing = (& $c1541 $Image -list 2>&1) -join "`n" + $pattern = '(?im)"' + [regex]::Escape($DiskName) + '"\s+' + $Type + if ($listing -notmatch $pattern) { + throw "$DiskName ($Type) is missing from $Image after writing" + } +} + function Invoke-Nova128Emulator { param( [Parameter(Mandatory)][ValidateSet("x64sc", "x128")][string]$Emulator, @@ -104,10 +128,23 @@ function Invoke-Nova128Emulator { [string]$DiskImage, [string]$Screenshot, [int]$LimitCycles = 0, - [switch]$Warp + [switch]$Warp, + [switch]$EightyColumn, + [switch]$ProgramOnDisk ) $tool = Get-Nova128ViceTool $Emulator - if (-not (Test-Path -LiteralPath $Program)) { throw "program not found: $Program" } + $workingDirectory = $null + $autostart = $Program + if ($ProgramOnDisk) { + # The editor takes its write device from the disk it was loaded from. + # Autostarting the copy on the session disk is what makes its saves + # reach the image; a host-injected program has no such device. + if (-not $DiskImage) { throw "a disk-resident program needs -DiskImage" } + $autostart = [IO.Path]::GetFileName($DiskImage) + ":" + $Program + $workingDirectory = Split-Path ([IO.Path]::GetFullPath($DiskImage)) -Parent + } elseif (-not (Test-Path -LiteralPath $Program)) { + throw "program not found: $Program" + } $arguments = @() if ($DiskImage) { @@ -117,7 +154,8 @@ function Invoke-Nova128Emulator { $arguments += @("-drive8type", "1581", "-drive8truedrive", "-8", ('"' + $DiskImage + '"')) } - $arguments += @("-autostart", ('"' + $Program + '"'), "-autostart-warp", + if ($EightyColumn) { $arguments += "-80col" } + $arguments += @("-autostart", ('"' + $autostart + '"'), "-autostart-warp", "-sounddev", "dummy") if ($Warp) { $arguments += "-warp" } if ($LimitCycles -gt 0) { $arguments += @("-limitcycles", "$LimitCycles") } @@ -125,13 +163,21 @@ function Invoke-Nova128Emulator { if (Test-Path -LiteralPath $Screenshot) { Remove-Item -LiteralPath $Screenshot -Force } - # x64sc names the VIC-II exit screenshot differently from x128. - $shotOption = if ($Emulator -eq "x64sc") { "-exitscreenshot" } else { "-exitscreenshotvicii" } + # x64sc's plain -exitscreenshot is the VIC-II screen. On x128 the + # 80-column editor draws on the VDC, which only the plain option + # captures; -exitscreenshotvicii would save a blank 40-column screen. + $shotOption = if ($Emulator -eq "x64sc" -or $EightyColumn) { + "-exitscreenshot" + } else { + "-exitscreenshotvicii" + } $arguments += @($shotOption, ('"' + $Screenshot + '"')) } - Write-Host ("launching {0} with {1}" -f $Emulator, $Program) - $process = Start-Process -FilePath $tool -ArgumentList $arguments -PassThru -Wait + Write-Host ("launching {0} with {1}" -f $Emulator, $autostart) + $startArgs = @{ FilePath = $tool; ArgumentList = $arguments; PassThru = $true; Wait = $true } + if ($workingDirectory) { $startArgs.WorkingDirectory = $workingDirectory } + $process = Start-Process @startArgs if ($Screenshot -and -not (Test-Path -LiteralPath $Screenshot)) { throw "$Emulator did not create the exit screenshot $Screenshot" } @@ -140,4 +186,5 @@ function Invoke-Nova128Emulator { } Export-ModuleMember -Function Get-Nova128ViceTool, Enter-Nova128ImageSession, - Exit-Nova128ImageSession, Wait-Nova128ImageFlush, Invoke-Nova128Emulator + Exit-Nova128ImageSession, Wait-Nova128ImageFlush, Add-Nova128DiskFile, + Invoke-Nova128Emulator diff --git a/newLang/tests/disk/launchers.tests.ps1 b/newLang/tests/disk/launchers.tests.ps1 index ae0f41b..9c8809e 100644 --- a/newLang/tests/disk/launchers.tests.ps1 +++ b/newLang/tests/disk/launchers.tests.ps1 @@ -83,6 +83,10 @@ Wait-Nova128ImageFlush -Image $diskA -TimeoutSeconds 2 & $launchEditor -EditorPrg $minimal -SourceImage $source -DiskImage $diskA ` -LimitCycles 2000000 -Screenshot $editorShot -Warp if (-not (Test-Path -LiteralPath $editorShot)) { throw "editor launcher produced no screenshot" } +$editorListing = (& "C:\Program Files\GTK3VICE-3.10-win64\bin\c1541.exe" $diskA -list 2>&1) -join "`n" +if ($editorListing -notmatch '(?im)"vim128"\s+prg') { + throw "the editor launcher did not put vim128 on the session disk" +} # 5: the compiler launcher autostarts its program in x128 and leaves the # disposable image flushed and unlocked. @@ -96,5 +100,5 @@ $sourceHashAfter = (Get-FileHash -LiteralPath $source -Algorithm SHA256).Hash if ($sourceHashAfter -ne $sourceHashBefore) { throw "the Vim source image changed during the session tests" } Write-Host "launcher tests passed" -Write-Host "editor disk copy, lock refusal, flush wait, x64sc autostart, x128 autostart" +Write-Host "editor disk copy (vim128 injected), lock refusal, flush wait, x128 editor and compiler autostart" Write-Host "source disk unchanged: $sourceHashAfter" diff --git a/newLang/tests/disk/tokenize-seq.s b/newLang/tests/disk/tokenize-seq.s new file mode 100644 index 0000000..5e71874 --- /dev/null +++ b/newLang/tests/disk/tokenize-seq.s @@ -0,0 +1,234 @@ +; E07-S03: tokenize the SEQ file the editor created on this session's disk. +; +; One fixture, ROUNDTRIP.N8, loaded through the normal source reader so the +; C128 side never sees a host conversion. The dump matches the format of +; tests/lexer/lexer-test.s, so the host can decode it with the reference +; lexer's ConvertFrom-TokenDump: +; fixture: $FF, fixture index (0), src_load result +; token: kind, line lo, line hi, col, then by kind: +; EOL, EOF nothing +; ERROR error code +; IDENT, STRING raw length, text length, text bytes +; NUMBER raw length, value lo, value hi +; CHAR raw length, value +; other raw length +; end: $FE +; Mailbox: PASS when the file loaded and the dump did not overflow; +; MB_DATA+0/1 dump address, +2/3 dump length. + +.setcpu "6502" +.include "../../src/common/c128.inc" +.include "../../src/common/mailbox.inc" +.include "../../src/common/tokens.inc" + +.import mmu_init, mmu_bank0, mmu_shutdown +.import src_load +.import lex_reset, lex_next +.import tok_kind, tok_line, tok_col, tok_len, tok_value, tok_text, tok_text_len, tok_err + +DUMP_SIZE = $2000 +MAX_TOKENS = 2000 + +.segment "LOADADDR" + .word $1C01 + +.segment "EXEHDR" + .word basic_end + .word 10 + .byte $9E + .byte "7181" + .byte $00 +basic_end: + .word $0000 + +.segment "MAILBOX" + .res MB_SIZE + +.segment "ZEROPAGE" +dump_ptr: .res 2 + +.segment "STARTUP" +.export test_entry, dump +test_entry: + php + sei + ldx #MB_SIZE - 1 + lda #$00 +@clr: sta MB_BASE,x + dex + bpl @clr + lda #MB_MAGIC0_VALUE + sta MB_MAGIC0 + lda #MB_MAGIC1_VALUE + sta MB_MAGIC1 + lda #MB_PROD_PLATFORM + sta MB_PRODUCER + lda #MB_ST_RUNNING + sta MB_STATUS + + jsr mmu_init + jsr mmu_bank0 + + lda #dump + sta dump_ptr+1 + lda #$00 + sta failures + sta overflow + + lda #$FF + jsr emit + lda #$00 + jsr emit + lda #f_name + ldy #f_name_end - f_name + jsr src_load + pha + jsr emit + pla + beq @lex + inc failures + jmp @end +@lex: jsr lex_run +@end: lda #$FE + jsr emit + + ; Report before mmu_shutdown: dump_ptr lives in the private zero page, + ; which the system mapping no longer shows. + lda #dump + sta MB_DATA+1 + sec + lda dump_ptr + sbc #dump + sta MB_DATA+3 + jsr mmu_shutdown + lda failures + ora overflow + sta MB_CODE + ldy #ok_msg - messages + lda MB_CODE + bne @fail + lda #MB_ST_PASS + bne @report +@fail: ldy #fail_msg - messages + lda #MB_ST_FAIL +@report: + sta MB_STATUS + plp +@print: lda messages,y + beq @done + sty print_y + jsr CHROUT + ldy print_y + iny + bne @print +@done: rts + +.segment "CODE" + +lex_run: + jsr lex_reset + lda #MAX_TOKENS + sta budget+1 +@token: jsr lex_next + jsr emit_token + lda tok_kind + cmp #TOK_EOF + beq @done + lda budget + bne :+ + dec budget+1 +: dec budget + lda budget + ora budget+1 + bne @token + inc failures +@done: rts + +emit_token: + lda tok_kind + jsr emit + lda tok_line + jsr emit + lda tok_line+1 + jsr emit + lda tok_col + jsr emit + lda tok_kind + cmp #TOK_EOF + beq @none + cmp #TOK_EOL + beq @none + cmp #TOK_ERROR + bne @len + lda tok_err + jmp emit +@none: rts +@len: lda tok_len + jsr emit + lda tok_kind + cmp #TOK_IDENT + beq @text + cmp #TOK_STRING + beq @text + cmp #TOK_NUMBER + beq @number + cmp #TOK_CHAR + bne @none + lda tok_value + jmp emit +@number: + lda tok_value + jsr emit + lda tok_value+1 + jmp emit +@text: lda tok_text_len + jsr emit + ldx #$00 +@byte: cpx tok_text_len + beq @none + lda tok_text,x + stx emit_x + jsr emit + ldx emit_x + inx + bne @byte + +; Append A to the dump; stop writing (and flag) once it is full. +emit: ldy dump_ptr+1 + cpy #>(dump + DUMP_SIZE) + bcc @room + lda #$01 + sta overflow + rts +@room: ldy #$00 + sta (dump_ptr),y + inc dump_ptr + bne :+ + inc dump_ptr+1 +: rts + +.segment "RODATA" +f_name: .byte "ROUNDTRIP.N8" +f_name_end: + +messages: +ok_msg: .byte "NOVA128 TOKENIZE OK", $0D, $00 +fail_msg: .byte "NOVA128 TOKENIZE FAIL", $0D, $00 + +.segment "BSS" +failures: .res 1 +overflow: .res 1 +budget: .res 2 +emit_x: .res 1 +print_y: .res 1 +dump: .res DUMP_SIZE diff --git a/newLang/tests/disk/vim-roundtrip.tests.ps1 b/newLang/tests/disk/vim-roundtrip.tests.ps1 new file mode 100644 index 0000000..7284929 --- /dev/null +++ b/newLang/tests/disk/vim-roundtrip.tests.ps1 @@ -0,0 +1,186 @@ +# E07-S03: create a Nova128 source in the Vim clone, prove the bytes on the disk +# are the expected PETSCII/CR file, reopen it in a second Vim session, and +# tokenize the disk file on C128 without a host conversion. +[CmdletBinding()] +param([int]$Port = 6541, [int]$SessionTimeoutSeconds = 300, [int]$MailboxSeconds = 180) + +$ErrorActionPreference = "Stop" + +$project = Split-Path (Split-Path $PSScriptRoot -Parent) -Parent +$repository = Split-Path $project -Parent +Import-Module (Join-Path $project "scripts\lib\Nova128Test.psm1") -Force +. (Join-Path $project "tests\lexer\reference-lexer.ps1") + +$root = Join-Path $project "build\vim-roundtrip" +New-Item -ItemType Directory -Force -Path $root | Out-Null +& (Join-Path $project "scripts\build-tokenize-seq.ps1") | Out-Host + +$x128 = Get-ViceTool "x128" +$c1541 = Get-ViceTool "c1541" +$sourceImage = Join-Path $repository "vim\vim-improved.d81" +$editorPrg = Join-Path $repository "vim\vim128.prg" +$work = Join-Path $root "work.d81" +$roundtrip = Join-Path $root "roundtrip.pet" +$copy = Join-Path $root "copy.pet" + +if (-not (Test-Path -LiteralPath $sourceImage -PathType Leaf)) { + throw "editor image not found: $sourceImage" +} +if (-not (Test-Path -LiteralPath $editorPrg -PathType Leaf)) { + throw "C128 editor program not found: $editorPrg" +} +$sourceHashBefore = (Get-FileHash -LiteralPath $sourceImage -Algorithm SHA256).Hash +Copy-Item -LiteralPath $sourceImage -Destination $work -Force +# The session's disk carries the C128 editor as well, so the machine can +# reload it without the host. +$injected = (& $c1541 $work -write $editorPrg "vim128" 2>&1) -join "`n" +if ($LASTEXITCODE -ne 0) { throw "could not inject vim128 into the session disk:`n$injected" } + +# Both PETSCII letter ranges appear: lowercase identifiers and uppercase ones. +$text = @( + "program RoundTrip", + "var alpha as byte", + "var BETA as byte", + "alpha = 1", + "BETA = 2", + "print alpha", + "print BETA", + "end program") -join "`n" + +# VICE keybuf uses a literal backslash-n for RETURN; \x03 is RUN/STOP, which +# leaves the editor's insert mode. +$typed = $text -split "`n" +$insert = ($typed -join "\n") + +function Invoke-EditorSession { + param( + [Parameter(Mandatory = $true)][string]$Keys, + [Parameter(Mandatory = $true)][string]$Screenshot + ) + + if (Test-Path -LiteralPath $Screenshot) { Remove-Item -LiteralPath $Screenshot -Force } + $psi = [Diagnostics.ProcessStartInfo]::new() + $psi.FileName = $x128 + # The C128 editor is an 80-column program, so the process starts in 80col + # mode and the exit screenshot must be the plain option (VDC screen). It + # is autostarted from the session disk: a host-loaded program gets no + # device number, and vim128 then reports `written` without touching the + # image. The machine runs at 1 MHz apart from autostart, so the wall-clock + # key delay is also a cycle delay. + $psi.WorkingDirectory = $root + $psi.Arguments = @( + "-80col", + "-autostart", '"work.d81:vim128"', + # A D81 needs a 1581 as drive 8, otherwise :wq fails with "cannot write". + "-drive8type", "1581", + "-8", "work.d81", + "-autostart-warp", + "-keybuf", ('"' + $Keys + '"'), + "-keybuf-delay", "300", + "-limitcycles", "120000000", + "-exitscreenshot", ('"' + $Screenshot + '"'), + "-sounddev", "dummy" + ) -join ' ' + $psi.UseShellExecute = $false + $psi.CreateNoWindow = $true + $process = [Diagnostics.Process]::Start($psi) + if (-not $process.WaitForExit($SessionTimeoutSeconds * 1000)) { + $process.Kill() + throw "x64sc session exceeded $SessionTimeoutSeconds s" + } + if (-not (Test-Path -LiteralPath $Screenshot)) { + throw "x64sc produced no exit screenshot; the session did not run" + } +} + +function Read-DiskSeq { + param( + [Parameter(Mandatory = $true)][string]$DiskName, + [Parameter(Mandatory = $true)][string]$HostPath + ) + if (Test-Path -LiteralPath $HostPath) { Remove-Item -LiteralPath $HostPath -Force } + $output = (& $c1541 $work -read "$DiskName,s" $HostPath 2>&1) -join "`n" + if (-not (Test-Path -LiteralPath $HostPath -PathType Leaf)) { + throw "could not read $DiskName from the session disk:`n$output" + } + return ,([IO.File]::ReadAllBytes($HostPath)) +} + +function Compare-Bytes { + param( + [Parameter(Mandatory = $true)][byte[]]$Actual, + [Parameter(Mandatory = $true)][byte[]]$Expected, + [Parameter(Mandatory = $true)][string]$What + ) + $common = [Math]::Min($Actual.Length, $Expected.Length) + for ($i = 0; $i -lt $common; $i++) { + if ($Actual[$i] -ne $Expected[$i]) { + throw ("{0}: byte {1} is `${2:X2}, expected `${3:X2} (actual {4} bytes, expected {5})" -f + $What, $i, $Actual[$i], $Expected[$i], $Actual.Length, $Expected.Length) + } + } + if ($Actual.Length -ne $Expected.Length) { + throw ("{0}: length {1}, expected {2}" -f $What, $Actual.Length, $Expected.Length) + } +} + +# 1: create the file entirely in the editor. The name, the create answer, the +# insert-mode text, RUN/STOP, and :wq all come from one keybuf string. +Invoke-EditorSession -Keys ("roundtrip.n8\n" + "y" + "i" + $insert + "\x03:wq\n") ` + -Screenshot (Join-Path $root "edit.png") +$written = Read-DiskSeq -DiskName "roundtrip.n8" -HostPath $roundtrip +$expected = ConvertTo-Nova128Petscii ($text + "`n") +Compare-Bytes -Actual $written -Expected $expected -What "editor-written ROUNDTRIP.N8" +$hasLow = @($written | Where-Object { $_ -ge 0x41 -and $_ -le 0x5A }) +$hasHigh = @($written | Where-Object { $_ -ge 0xC1 -and $_ -le 0xDA }) +if (-not $hasLow -or -not $hasHigh) { + throw "the written file does not exercise both PETSCII letter ranges" +} +Write-Host ("editor file: {0} bytes, PETSCII ``$41-``$5A and ``$C1-``$DA both present" -f $written.Length) + +# 2: reopen the same file in a second session and write the buffer to another +# name, which proves the editor restored the content it had saved. +Invoke-EditorSession -Keys ("roundtrip.n8\n" + ":w copy.n8\n" + ":q\n") ` + -Screenshot (Join-Path $root "reopen.png") +$rewritten = Read-DiskSeq -DiskName "copy.n8" -HostPath $copy +Compare-Bytes -Actual $rewritten -Expected $written -What "reopened COPY.N8" +Write-Host "reopened in Vim and rewritten byte-identical" + +# 3: tokenize the editor-written file on C128 in x128. +$disk = Join-Path $root "tokenize-seq.d81" +$add = (& $c1541 $disk -write $roundtrip "roundtrip.n8,s" 2>&1) -join "`n" +if ($LASTEXITCODE -ne 0) { throw "could not add the editor file to the test disk:`n$add" } + +$screenshot = Join-Path $root "tokenize.png" +$session = Start-Nova128X128 -Autostart $disk -Drive8Image $disk -Port $Port -Screenshot $screenshot +try { + $mb = Wait-Nova128Mailbox $session -TimeoutSeconds $MailboxSeconds + $address = $mb[8] + 256 * $mb[9] + $length = $mb[10] + 256 * $mb[11] + $dump = Read-Nova128Memory $session $address $length +} finally { + Stop-Nova128X128 $session +} +[IO.File]::WriteAllBytes((Join-Path $root "token-dump.bin"), $dump) + +if ($mb[3] -ne 0xA5) { throw ("C128 reported FAIL (code `${0:X2})" -f $mb[4]) } +$entries = ConvertFrom-TokenDump $dump +if ($entries.Count -ne 1) { throw "token dump holds $($entries.Count) fixtures, expected 1" } +if ($entries[0].Load -ne 0) { throw "src_load returned $($entries[0].Load) for the editor file" } +$wanted = @((Get-ReferenceTokens $written) | ForEach-Object { $_ }) +$got = @($entries[0].Tokens) +if ($wanted.Count -ne $got.Count) { + throw "C128 produced $($got.Count) tokens, the reference lexer $($wanted.Count)" +} +for ($i = 0; $i -lt $wanted.Count; $i++) { + if ($wanted[$i] -ne $got[$i]) { + throw ("token {0} differs: reference '{1}' vs C128 '{2}'" -f ($i + 1), $wanted[$i], $got[$i]) + } +} + +$sourceHashAfter = (Get-FileHash -LiteralPath $sourceImage -Algorithm SHA256).Hash +if ($sourceHashAfter -ne $sourceHashBefore) { throw "the Vim source image changed during the round-trip" } + +Write-Host ("tokenized on C128: {0} tokens, no host conversion" -f $got.Count) +Write-Host ("source image SHA-256 unchanged: {0}" -f $sourceHashAfter) +Write-Host "vim source round-trip passed"