C64-dev/newLang/planning/claims/E07-S03.md

---

story: E07-S03 owner: codebuff status: done started: 2026-11-10 completed: 2026-11-10 depends-on: [E07-S02, E02-S02]

E07-S03 Claim

Intended change

Prove that a Nova128 source written only in the editor is a valid PETSCII SEQ file that the C128 compiler front end consumes without a host conversion, that the editor itself can reopen it unchanged, and that the exact bytes match an independently computed expectation. D151 moves this check to the target machine: the editor is ../vim/vim128.prg in x128 80-column mode, autostarted from the session disk.

File area

• tests/disk/tokenize-seq.s (new): load ROUNDTRIP.N8 through src_load, tokenize it, dump tokens in the lexer-test.s format, report through the mailbox
• scripts/build-tokenize-seq.ps1 (new)
• tests/disk/vim-roundtrip.tests.ps1 (new): two editor sessions plus the tokenizer session
• docs/decisions.md (D151), docs/editor-workflow.md, docs/architecture.md (§10)
• planning/claims/E07-S03.md
• generated files under build/vim-roundtrip/

Acceptance evidence

• A source file created entirely in Vim tokenizes on C128 without host conversion: the test types the whole program through VICE keybuf into vim128 (file: prompt, y to create, i, text, RUN/STOP, :wq) and then runs the new C128 tokenizer, which loads the disk file with src_load, lexes it, and reports 31 tokens that match the host reference lexer line for line. Nothing rewrites the file between Vim and the C128.
• Reopening it in Vim preserves content: a second x128 session opens roundtrip.n8 and writes the buffer to copy.n8; the extracted bytes are identical (107 bytes), so the editor restored the saved text exactly.
• Extracted bytes match the expected PETSCII/CR fixture: the host computes the expected bytes from the same ASCII text with ConvertTo-Nova128Petscii (the mapping prepare-disk.ps1 uses) and compares every byte; c1541 -read only transfers the file.
• Both supported PETSCII letter ranges are exercised: the source mixes lowercase identifiers (program, alpha) and uppercase ones (RoundTrip, BETA), and the test asserts the written file contains bytes in $41-$5A and $C1-$DA before tokenizing.

Coordination notes

• Depends on E07-S02 (launchers; D151 moved the editor to the C128 build) and E02-S02 (lexer); both are complete. The tokenizer links the production source.s and lexer.s, so it proves the reader and lexer rather than a copy of them.
• The expected bytes are computed on the host from the same text that was typed; the C128 side never receives a converted copy, which is the property the story is about.
• ../vim/vim-improved.d81 and ../vim/vim128.prg are only read; the session works on build/vim-roundtrip/work.d81, and the test checks the source image's SHA-256 is unchanged.

Work log

• The first run stopped at LOADING: the session had a 40M-cycle limit and no warp after autostart, so the editor had not loaded. The limit is now 120M cycles with a 300 ms per-key delay, enough for loading, typing and :wq.
• The editor then typed the text but reported cannot write because drive 8 defaulted to a 1541, which cannot write a D81. -drive8type 1581 fixed the device; a -drive8truedrive variant also failed to store the file.
• With an emulated 1581 the editor reported success and quit, yet the image had no file: it had been autostarted from the host PRG, and a host-loaded program has no device number from its load, so the saves went nowhere. Autostarting vim128 from the session disk (work.d81:vim128, run from the image's folder) makes the writes land; the launcher and module now use that path.
• The 80-column session draws on the VDC, so the automated screenshot uses VICE's plain -exitscreenshot; -exitscreenshotvicii produced a blank 40-column capture, and -exitscreenshotvdc is not a VICE option.

Completion

• Tests run: tests/disk/vim-roundtrip.tests.ps1 (two x128 editor sessions on a copied D81, then one x128 tokenizer session), tests/disk/launchers.tests.ps1, git diff --check.
• Results: vim source round-trip passed; the editor wrote 107 bytes with both PETSCII letter ranges, the reopened copy was byte-identical, and C128 produced 31 tokens matching the reference lexer. The source image hash stayed 0FF83A621B59B312F1BAE9F1D0785421500D22FD00F833A9CA918EBCA7105983.
• Remaining risks: keybuf is wall-clock driven, so a heavily loaded host can deliver keys before a prompt is ready; the test's cycle limit and per-key delay are the guard, not a handshake. Lines longer than the editor's 80-byte limit are split by the editor on load and cannot be reproduced by this test, and the tokenizer only accepts the single fixed file name ROUNDTRIP.N8.