Word 6 (#278): Faithful Win16 Local-Heap Free-Block Chain, Implementation Plan

Status: design complete (offline, 2026-06-29), NOT yet applied at the time of writing. Tree: the experimental OLE2 kernel branch (kernel-ole2c), deployed only to a dedicated test VM; the main branch is unaffected. All file:line refs are against a local mirror of that branch; re-confirm them against the live tree before applying, since the mirror may lag recent builds.

TL;DR (the load-bearing insight)

Word's statically linked Microsoft C runtime (WINWORD seg231, cs=0x073f) reimplements the local-heap manager itself and walks the DGROUP arena BYTES DIRECTLY. It does NOT call KERNEL.4/5/6 (LocalInit/LocalAlloc/LocalFree) for its own DGROUP heap. Therefore the in-memory byte layout of the DGROUP near-heap reserve must be FAITHFUL at load time. Rewriting the KERNEL Local* ordinals is necessary for the OLE2 DLLs (STORAGE/COMPOBJ do call them) but does nothing for Word's own heap. The fix that actually replaces the LHFIX band-aid is a one-time, load-time byte-level pre-format of [lheap_base, lheap_top) as a coherent single-word-header free arena, terminated exactly at pLast.

A. Current state: what the band-aids do and why LHFIX is geometrically wrong

Three overlapping band-aids, all gated Word-only (g_ole2_k334log / g_win16_pmode):

1. LHFIX, the local-heap "arena-walk terminator" (x86_16.c:1347-1371). The target of this plan.

Hooks per-instruction dispatch at cs==0x073f && ip==0x05f2 (seg231, the MS-C runtime local-heap compaction routine). seg231 walks the arena: load si=pFirst, dx=pLast, then mov bx,[si]; test bl,1 (busy bit), advancing si += (hdr & ~1) + 1. When the header read is 0 (an uninitialised header) with si<dx, LHFIX synthesizes ONE busy bridge block: bridge = ((dx - si - 1) & ~1u) | 1u; x86_16_wr16(cpu,ds,si,bridge); cpu->bx = bridge; so the next advance lands si exactly on dx and cmp si,dx; jz done fires. WHY IT IS WRONG: it does not build a heap; it papers a single fake busy block over the un-formatted ne_heap reserve (which carries a zero header because Word's own LInit never pre-formatted the reserve as a real free arena in our environment). It leaves the arena internally incoherent: seg231's post-walk grow/scan reads garbage past the fake block, wanders into a strlen-style loop over the "WINWORD6.EXE" bytes, and synthesizes a bogus LInit(wSegment=0) that far-returns to cs=0000 (the WILDCS halt seen in pass 5).

1. GlobalSize DGROUP stack-clamp (win16api.c:1703-1713, k_globalsize): for h==g_dgroup_sel, clamps the reported size down to g_dgroup_heap_top (= data_len + ne_heap), paired with the NE-loader heap_is_substantial clause (ne.c:1133-1144) that keeps ltop = data_len + ne_heap instead of extending to sp-0x400. These are the CORRECT half of the fix: they make pLast land below the in-segment stack (SS==DS). They do NOT format the reserve, so LHFIX is still needed today. ROOT CAUSE (cl_word6.md pass 6): __LInit derives pLast from GlobalSize(DGROUP); k_globalsize had returned the FULL autodata size (data_len + ne_heap + ne_stack = 0xa5b2), placing pLast over the stack region [0x68fe, 0xa5b2), creating a permanent gap.
2. W6MEMFIX (x86_16.c:1318-1330): forces DGROUP:0x1a37 bit0 ("memory OK") to suppress Word's "not enough memory" MessageBox. Unrelated to the heap. LEAVE IT ALONE.

B. The faithful model (real Win16 near-heap layout)

A Win16 local heap is a near heap inside one 16-bit data segment (DGROUP, selector 0x075f); all pointers are 16-bit near offsets.

LOCALINFO header (pointed at by bx in seg231; offsets confirmed in cl_word6.md pass 5): +0x00 word pLast end marker (one past last arena byte); here must be 0x6900 +0x02 word pFirst first arena block offset; here 0x36a6 +0x04 word pFreeFirst head of free-list +0x06 word pFreeLast / extra +0x14/0x16/0x18 words moveable handle-table scan bounds +0x1a word heap byte count +0x1e word stored free count

Canonical real-Windows LOCALARENA (KRNL386): busy header 4 bytes +0x00 word pPrev (bit0 = busy flag, 1=busy) +0x02 word pNext free header 8 bytes adds +0x04 pPrevFree, +0x06 pNextFree (free-list doubly linked). Block size is implicit in pNext - pThis.

IMPORTANT nuance for THIS target: seg231 (the MS-C runtime, NOT KRNL386) uses a SIMPLIFIED single-word arena: it reads mov bx,[si] (one header word) and advances si += (hdr & ~1) + 1, i.e. [si] = (size | busy_bit) word-granular. So the byte layout the interpreter must produce to satisfy seg231's own walk is the single-word (size|busy) header form, with a chain that terminates exactly at pLast, NOT the full 4/8-byte KRNL386 arena. (The KRNL386 arena is only needed if/when KRNL386's own LocalAlloc is exercised, i.e. the OLE2 DLLs.)

Moveable handle table (HLOCAL): for LMEM_MOVEABLE the handle is a table index whose entry holds the near pointer + lock count; LocalLock returns the pointer. For LMEM_FIXED the handle IS the near pointer (the current code assumes fixed for everything at win16api.c:1352-1357). Word's MS-C runtime uses fixed blocks here, so fixed-only is the priority; moveable can be a thin table.

C. Algorithm per operation

(near offsets in the target segment; lbase = li.lheap_base, ltop = li.lheap_top, already correct from ne.c:1148-1149 for the substantial-heap case = data_len + ne_heap, ending below the stack.)

LocalInit(wSeg, pStart, pEnd): build a coherent initial arena.

1. tgt = wSeg ? wSeg : DS. Default pStart->lbase (>=2), pEnd->ltop (or 0xFFFE for "to end").
2. For the app's own DGROUP the MS-C runtime owns the LOCALINFO, so do NOT overwrite the header; instead format the arena BYTES [pStart, pEnd).
3. Format as: a busy sentinel at pStart (1 word: 0|1), then ONE big free block spanning [pStart+1, pEnd) with header ((pEnd - (pStart+1) - 1) & ~1) | 0 (bit0=0 free), then a terminating busy sentinel whose advance lands exactly on pEnd so the walk's cmp si,dx; jz fires. This is the faithful replacement for the LHFIX bridge, done ONCE at init, formatting the whole reserve so the later compaction walk reaches pLast naturally with no per-instruction hook.
4. Initialize the free-list anchor (pFreeFirst = the big free block) + per-segment allocator state.

LocalAlloc(wFlags, wBytes): first-fit.

1. Round wBytes to word granularity; add 1-word header overhead.
2. Walk free-list from pFreeFirst; first block with size >= need wins.
3. Split if remainder > minblock (shrink found block, create free tail, relink); else take whole.
4. Set busy bit (hdr |= 1).
5. LMEM_ZEROINIT (0x0040): zero the user bytes (already done at win16api.c:1338-1340).
6. Return near offset of user data (fixed) or handle-table index (moveable). 0 if no fit.

LocalFree(hMem): clear busy bit; coalesce with physical-next if free, then physical-prev if free; insert merged block into free-list; return 0 (NULL) on success.

LocalReAlloc(hMem, wBytes, wFlags):

1. new <= current: shrink in place, split free tail, coalesce. Same handle.
2. new > current and physical-next is free and combined suffices: extend in place. Same handle.
3. else: LocalAlloc new, COPY old bytes (current code does NOT copy, win16api.c:1346-1349, fix this), LocalFree old. Return new handle (fixed) or update table entry (moveable, same handle).

LocalLock/Unlock: fixed -> handle is pointer, return it (current behavior win16api.c:1355-1357 is correct for fixed). Moveable -> return handletable[hMem].ptr, bump/decrement lock count.

LocalSize(hMem): return real byte size from the header chain ((hdr & ~1) minus overhead). Current code returns 0 (win16api.c:1374-1376) which corrupts the MS-C runtime's accounting; must be real.

LocalCompact(wMinFree): walk arena, coalesce adjacent free blocks, return largest free size. LocalHandle(wOffset): fixed -> return offset; moveable -> reverse-lookup the table entry.

D. Where the logic belongs

CRUX (cl_word6.md pass 5): "It operates on the DGROUP ... NONE of our KERNEL.4/5 are involved in this arena, the app builds + walks it entirely itself." So the in-memory byte layout MUST be faithful; the API behavior is secondary. Governing values for Word's DGROUP arena:

1. GlobalSize(DGROUP) -> pLast (already fixed, win16api.c:1703-1713).
2. lheap_base/lheap_top from the NE loader (already fixed, ne.c:1133-1149).
3. The actual BYTES in [lheap_base, lheap_top) at load time: currently raw zeros, which is why the bridge is needed.

PRIMARY change, win16api.c, in win16_api_begin (around :7826 where g_dgroup_sel/g_dgroup_heap_top are set from g_info): after recording DGROUP geometry, PRE-FORMAT the DGROUP near-heap reserve [lheap_base, lheap_top) as a valid single-word-header free arena (LocalInit step 3 above), writing bytes directly into the guest segment via x86_16_wr16(g_cpu, g_dgroup_sel, off, hdr). One-time faithful equivalent of the LHFIX bridge. (Alternatively in ne.c right after win16_api_begin at :1151; win16api.c is preferred because g_dgroup_sel/g_dgroup_heap_top + g_cpu already live there.)

Then DELETE the LHFIX hook (x86_16.c:1331-1371) entirely. With a coherent pre-formatted arena the bx==0 condition never fires. Back up x86_16.c first (x86_16.c.bak-pre-faithful-lheap).

SECONDARY change, win16api.c KERNEL.4/5/6/7/8/9/10 ordinals (k_localinit :910, k_localalloc :1333, k_localrealloc :1346, k_localfree :1367, k_locallock :1355, k_localunlock :1360, k_localsize :1374, plus lheap_alloc_seg :209): replace the per-segment BUMP allocator (g_lseg[], win16api.c:186-226) with a real free-block chain per segment (Section C), operating on guest bytes. Needed for the OLE2 DLLs (STORAGE/COMPOBJ) that legitimately call these ordinals; the bump allocator never reclaims, never copies on realloc, returns size 0 (latent corruption for any DLL that round-trips the API). Keep a per-segment LOCALINFO-like software record {seg, pFirst, pLast, pFreeFirst} replacing lseg_t {seg, next, top} at win16api.c:193. Add k_localcompact/k_localhandle and register them in g_api_table (KERNEL 11/12 area near :7517).

KEEP k_globalsize's DGROUP clamp (win16api.c:1710-1713) and the ne.c heap_is_substantial clause (ne.c:1142-1144): they are the correct half and the pre-format depends on them.

Files to change:

1. exec/win16api.c: pre-format DGROUP arena in win16_api_begin (new code near :7826); rewrite lheap_alloc_seg/lseg_t (:186-226) and k_localinit/alloc/realloc/free/lock/unlock/size (:910, :1328-1377); add k_localcompact/k_localhandle + register (near :7489-7517).
2. exec/x86_16.c: DELETE LHFIX hook (:1331-1371). Leave W6MEMFIX (:1318-1330).
3. exec/ne.c: no change required (geometry correct :1133-1149); confirm li.lheap_base/top are passed for ALL apps so the pre-format has a valid window.

E. Verification plan (project liveness rule)

Build in the kernel-ole2c tree and deploy to the dedicated test VM ONLY; the main branch stays untouched. Bump MAYTERA_BUILD_NUMBER in version.h first. Back up win16api.c and x86_16.c to .bak-pre-faithful-lheap and create a timestamped backup directory with a manifest; add a CHANGELOG.md entry (newest at top) as part of the same work.

1. Boot + serial trace. Launch Word via /CONFIG/WIN16PM.RUN; capture serial (capture the test VM's serial console to a per-build log file). Confirm: no [LHFIX] lines (hook gone); no WILDCS halt, no R6021 regression at/before CreateWindow; Word still emits RegisterClass 'OpusApp' + CreateWindow 'Microsoft Word'; ideally advances PAST the pass-8 app-init failure into the message loop / WM_PAINT. Optionally trace seg231 compaction reaching si==dx==pLast(0x6900) naturally with no synthetic bridge.
2. LIVENESS, two screendumps + clock (mandatory). Take 2+ screendumps of the test VM several seconds apart; confirm the desktop clock advanced (and/or screensaver animated / frames differ) = OS live, not wedged. THEN separately assert the Word window is actually PAINTED (a 'Microsoft Word' frame with content), not merely created-then-destroyed. Cross-check: ping 0% loss + serial responsive during the screendump window. Do NOT claim a rendered window until a screendump shows painted chrome.
3. No-regression (kernel-ole2c). Re-run base + pmode + 386 selftests (must PASS); OLE2 round-trip (PROBEC storage round-trip status=0); and an app that calls the KERNEL Local ordinals (Chips/SkiFree) to confirm the API rewrite did not break old fixed-handle-is-pointer behavior.

Key references

• LHFIX (delete): x86_16.c:1331-1371 ; W6MEMFIX (keep): x86_16.c:1318-1330
• local-heap globals / bump allocator (rewrite): win16api.c:169-226
• KERNEL.4 LocalInit: win16api.c:910-935 ; KERNEL.5-10: win16api.c:1328-1377
• KERNEL.20 GlobalSize DGROUP clamp (keep): win16api.c:1689-1715
• g_dgroup_sel/g_dgroup_heap_top init + pre-format site: win16api.c:7826-7827
• API table registration: win16api.c:7489-7517
• NE-loader heap geometry (keep): ne.c:1108-1150
• interpreter byte-level mem helpers: x86_16_wr16/rd16 at x86_16.c:263,289
• RE evidence: cl_word6.md:700-733 (walk + root cause), :824-844 (correct half), :803-814 (KIMP mislabel correction); pass8.md:46-64 (current terminal state)