feat: what the editor's debugger needs from the link and the compiler

docs/engine/editor-link.md

@@ -273,7 +273,8 @@ for each field, but the mission's file:
 | `global.N` | Global `N`'s value |
 | `variable.NAME` | The game's variable `NAME`, such as `mission_over`; one with no name by its number, where it isn't 0 |
 
-Values are the script's words, unsigned.
+Values are the script's words, unsigned. A reader passes over keys it doesn't know, so that a newer
+game's messages still read.
 
 ### What it does
 
@@ -283,8 +284,8 @@ Values are the script's words, unsigned.
   ([#1057](https://github.com/OpenReliant/openreliant/issues/1057)). It sends tags `0x1A` and `0x1B`.
 - The editor's messages wait while no mission plays, as the original's wait in its block for the
   next mission's start. Each attempt reads the mission's file again, so the flags last for one
-  attempt: an editor sends them again once the file comes with each `Mission started`, a fifth of a
-  second before the start parts run.
+  attempt: an editor sends them again once the file comes with each `Mission started`, and the
+  start parts wait for them.
 - The link opens as the game starts, so that an editor waiting for the game is there before the
   first mission's script starts.
 - The script's start waits on the editor as the original's does, drawing nothing until the editor
@@ -305,8 +306,10 @@ Values are the script's words, unsigned.
   editor had its own copy of the mission and its debugger only heard where the script stopped.
 - As the editor goes, the script is let go: its hold, its step and the editor's pause clear. The
   original keeps the script held, waiting for good.
-- The script's start waits for the editor only while one is there, where the original waits a fifth
-  of a second at every mission's start.
+- While an editor is linked, the script's start waits for its flags before the start parts run, for
+  a second at most, and checks the link every millisecond meanwhile. With no editor linked, it
+  doesn't wait. The original waits a fifth of a second at every mission's start, whether an editor
+  is there or not, and the breakpoints of a slower editor miss the start parts.
 - A tag past `0x17` is logged and passed over, where the original stops the game.
 - While the editor holds the script or pauses the mission, OpenReliant still draws the frame, with
   the frame's work left out.
@@ -359,12 +362,20 @@ A client in Zig can read and write missions exactly as OpenReliant does by impor
 ([Using the library in another project](../port/platform.md#using-the-library-in-another-project)).
 It works the way `openreliant debug` does:
 
-1. Connect to 127.0.0.1 at the port, and read `Hello`.
+1. Connect to 127.0.0.1 at the port, `link.default_port` unless the game was started with
+   another, and read `Hello`.
 2. On `Mission file`, which comes right after `Mission started`, send tag `0x09` with the script's
-   flags within a fifth of a second: a byte for each of the script's bytes, rounded up to a
-   multiple of 4, with bit 0 set on each byte to stop before. The statements end at the opcodes
-   `vm_starts_statement` gives (above), which a step stops before.
+   flags: a byte for each of the script's bytes, rounded up to a multiple of 4, with bit 0 set on
+   each byte to stop before. Send them with no breakpoints too, since the start parts wait for them
+   for up to a second. The statements end at the opcodes `vm_starts_statement` gives (above), which
+   a step stops before.
 3. On tag `0x1A`, the script is held before the byte at the offset it gives: send tag `0x15` with 1
    to run on, 2 to step into the next statement, or 3 to step over it; or with 0 and 1 to pause the
    mission, and 0 and 0 to let it go on.
 4. Send `Get state` for the script's threads, globals and variables.
+
+`engine/link.zig` has a record for each of OpenReliant's own messages, which writes its payload and
+reads it back: `Hello`, `MissionStarted` and `State`. The game writes them, and `openreliant debug`
+reads them. A client that builds missions from source places its breakpoints with the compiler's
+map from the source's lines to the script's statements
+([Building a mission](../formats/mission-source.md#building-a-mission)).

docs/formats/mission-source.md

@@ -292,6 +292,13 @@ kind's records are written in the order of their declarations. The script holds
 triggers run, in the order the triggers first run them, then the parts' blocks in order, each
 written as the developers' compiler wrote its statements ([Statements](dte.md#statements)).
 
+For a debugger, `compileMapped` also gives a map of the script's statements. For each statement, it
+gives the offset in the script of the instruction the statement ends with, as the editor link gives
+a stop, the line and column of its source, and whether the script stops before it. The script stops
+before an assignment, a command, a call, a spawn and an `if`, but not before a `return` or a random
+choice ([Stopping and stepping](../engine/editor-link.md#stopping-and-stepping)).
+`source_parser.buildMapped` parses source and compiles it with the map.
+
 OpenReliant's own Mission 0, the sandbox, is mission source too:
 [`mission0.mission.luau`](../../src/openreliant/mission0.mission.luau), which the build compiles
 with the same compiler, built for the host ([`build_mission.zig`](../../src/tools/build_mission.zig)).

docs/guide/debugging.md

@@ -85,10 +85,10 @@ conditional branch, a return or a spawn, after the instructions that push its op
 numbers the statements at those instructions, and `break <part>:<n>` stops before the `n`th.
 
 Breakpoints stay set from one mission to the next. As a mission starts, the game sends the debugger
-the mission's file, and the debugger places the breakpoints in its script before the start parts
-run, so a breakpoint set before the game starts can stop them. Each attempt at a mission reads its
-file again, and the breakpoints are placed again for it. A breakpoint whose part the mission doesn't
-have waits for one that has it.
+the mission's file, and waits for the debugger to place the breakpoints in its script before the
+start parts run, so a breakpoint set before the game starts can stop them. Each attempt at a
+mission reads its file again, and the breakpoints are placed again for it. A breakpoint whose part
+the mission doesn't have waits for one that has it.
 
 `step` and `next` work from where the script stopped. From a running script, they stop at the next
 statement any thread runs.

src/engine/game/mission/editor.zig

@@ -175,7 +175,7 @@ pub const Session = struct {
         @memcpy(session.file_buffer[0..session.file_length], started.file[0..session.file_length]);
         session.finished_told = null;
         loaded.script.editor.present = session.link.present();
-        loaded.script.editor.connection = .{ .context = session, .vtable = &.{ .stopped = stopped, .check = check, .sleep = sleep } };
+        loaded.script.editor.connection = .{ .context = session, .vtable = &.{ .stopped = stopped, .check = check, .sleep = sleep, .now = now } };
         session.tellMission();
     }
 
@@ -186,7 +186,7 @@ pub const Session = struct {
         loaded.session = null;
         loaded.script.editor.connection = null;
         session.loaded = null;
-        session.link.sendFields(.mission_ended, &.{});
+        session.link.send(@backingInt(link.Own.mission_ended), "");
     }
 
     /// Once a frame, and wherever the script's start checks the link (`editor_link_check`): an
@@ -220,7 +220,10 @@ pub const Session = struct {
             return;
         }
         switch (received[message.tag].action) {
-            .copy => |section| copy(&loaded.bound, section, message.payload),
+            .copy => |section| {
+                copy(&loaded.bound, section, message.payload);
+                if (section == .script_flags) loaded.script.editor.flagged = true;
+            },
             .run_control => runControl(loaded, message.payload),
             .unread, .nothing => {},
             .not_ported => if (!session.logged.isSet(message.tag)) {
@@ -239,26 +242,24 @@ pub const Session = struct {
         }
     }
 
-    /// Tells the editor the script's state (`writeState`).
+    /// Tells the editor the script's state (`stateOf`).
     fn tellState(session: *Session, loaded: *mission.Loaded) void {
-        var text: std.Io.Writer.Allocating = .init(session.link.gpa);
-        defer text.deinit();
-        writeState(&text.writer, loaded) catch {
+        var arena: std.heap.ArenaAllocator = .init(session.link.gpa);
+        defer arena.deinit();
+        const state = stateOf(arena.allocator(), loaded) catch {
             log.warn("the link has no memory left for the script's state", .{});
             return;
         };
-        session.link.send(@backingInt(link.Own.state), text.written());
+        session.link.sendRecord(.state, state);
     }
 
     /// Tells the editor of the mission playing, as it starts or as an editor connects, and sends
-    /// it the mission's file, which it places its breakpoints by before the start parts run.
+    /// it the mission's file. The editor answers with the script's flags, which the script's start
+    /// waits for.
     fn tellMission(session: *Session) void {
         const loaded = session.loaded orelse return;
-        var number: [std.fmt.count("{d}", .{std.math.maxInt(u16)})]u8 = undefined;
-        session.link.sendFields(.mission_started, &.{
-            .{ .key = "number", .value = number[0..std.fmt.printInt(&number, session.number, 10, .lower, .{})] },
-            .{ .key = "file", .value = session.file_buffer[0..session.file_length] },
-        });
+        const started: link.MissionStarted = .{ .number = session.number, .file = session.file_buffer[0..session.file_length] };
+        session.link.sendRecord(.mission_started, started);
         session.link.send(@backingInt(link.Own.mission_file), loaded.bound.image);
     }
 
@@ -292,48 +293,54 @@ pub const Session = struct {
         const session: *Session = @ptrCast(@alignCast(context));
         session.io.sleep(.fromMilliseconds(milliseconds), .awake) catch {};
     }
+
+    fn now(context: *anyopaque) u64 {
+        const session: *Session = @ptrCast(@alignCast(context));
+        return @intCast(@max(0, Io.Clock.awake.now(session.io).toMilliseconds()));
+    }
 };
 
-/// The script's state as `link.Own.state` gives it, a `key=value` line each: its clock (`clock`),
-/// the editor's hold, step and pause (`hold`, `step`, `paused`), and whether the pool's first
-/// thread has finished (`first_finished`); for each thread running, by its slot, where it is in the
-/// script (`thread.N.at`, none outside it), its call depth, the clock value it waits for, 0 for
-/// none, whether it waits for its trigger, and the trigger that started it, if one did; each
-/// global's value (`global.N`); and the game's variables (`variable.NAME`), by their names, and
-/// the unnamed ones by their numbers where they aren't 0. Values are the script's words, unsigned.
-fn writeState(writer: *std.Io.Writer, loaded: *mission.Loaded) std.Io.Writer.Error!void {
+/// The script's state for `link.Own.state` (`link.State`), its lists made in `arena`. A thread
+/// whose instruction is outside the script has no place in it. The game's variables come by their
+/// names, and the ones without a name by their numbers, where they aren't 0.
+fn stateOf(arena: std.mem.Allocator, loaded: *mission.Loaded) std.mem.Allocator.Error!link.State {
     const script = &loaded.script;
-    const editor = script.editor;
-    try writer.print("clock={d}\nhold={t}\nstep={t}\npaused={d}\nfirst_finished={d}\n", .{
-        script.clock,
-        editor.hold,
-        editor.step,
-        @intFromBool(editor.paused),
-        @intFromBool(script.first_finished),
-    });
     const code = loaded.bound.file.entry(.script);
-    var threads = script.liveThreads();
-    while (threads.next()) |found| {
+    var threads: std.ArrayList(link.State.Thread) = .empty;
+    var live = script.liveThreads();
+    while (live.next()) |found| {
         const thread, const index = found;
         const ip = thread.ip orelse continue;
         const at = ip -% code.offset;
-        if (at < @as(u32, code.count) * @sizeOf(u16)) {
-            try writer.print("thread.{d}.at={d}\n", .{ index, at });
-        } else try writer.print("thread.{d}.at=none\n", .{index});
-        try writer.print("thread.{d}.depth={d}\nthread.{d}.wake={d}\nthread.{d}.interrupted={d}\n", .{
-            index, thread.record.call_depth, index, thread.record.wake_time, index, @intFromBool(thread.record.interrupted),
+        try threads.append(arena, .{
+            .slot = @intCast(index),
+            .at = if (at < @as(u32, code.count) * @sizeOf(u16)) at else null,
+            .depth = thread.record.call_depth,
+            .wake = thread.record.wake_time,
+            .interrupted = thread.record.interrupted,
+            .trigger = if (thread.trigger) |trigger| @intCast(trigger) else null,
         });
-        if (thread.trigger) |trigger| try writer.print("thread.{d}.trigger={d}\n", .{ index, trigger });
-    }
-    for (loaded.bound.file.globals() catch &.{}, 0..) |global, index| {
-        try writer.print("global.{d}={d}\n", .{ index, global.value });
     }
+    const file_globals = loaded.bound.file.globals() catch &.{};
+    const globals = try arena.alloc(u32, file_globals.len);
+    for (globals, file_globals) |*value, global| value.* = global.value;
+    var variables: std.ArrayList(link.State.Variable) = .empty;
     for (0..vm.Variables.count) |number| {
         const value = script.variables.slot(@intCast(number)).*;
         if (std.enums.tagName(vm.GameVariable, @fromBackingInt(@intCast(number)))) |name| {
-            try writer.print("variable.{s}={d}\n", .{ name, value });
-        } else if (value != 0) try writer.print("variable.{d}={d}\n", .{ number, value });
+            try variables.append(arena, .{ .name = name, .value = value });
+        } else if (value != 0) try variables.append(arena, .{ .name = try arena.print("{d}", .{number}), .value = value });
     }
+    return .{
+        .clock = script.clock,
+        .hold = script.editor.hold,
+        .step = script.editor.step,
+        .paused = script.editor.paused,
+        .first_finished = script.first_finished,
+        .threads = threads.items,
+        .globals = globals,
+        .variables = variables.items,
+    };
 }
 
 /// `editor_table_copy` (`0x0045A770`): `payload` copied over a section of the mission, as many
@@ -468,16 +475,20 @@ test Session {
     try std.testing.expectEqual(@backingInt(link.Own.hello), (try t.heard(&buffer)).tag);
     const started = try t.heard(&buffer);
     try std.testing.expectEqual(@backingInt(link.Own.mission_started), started.tag);
-    try std.testing.expectEqualStrings("7", link.field(started.payload, "number").?);
-    try std.testing.expectEqualStrings("mission7.dte", link.field(started.payload, "file").?);
+    const started_read: link.MissionStarted = try .read(started.payload);
+    try std.testing.expectEqual(7, started_read.number);
+    try std.testing.expectEqualStrings("mission7.dte", started_read.file);
     try t.expectFile();
     try std.testing.expectEqual(0, try t.word(.first_finished));
 
-    // It flags the part's first instruction, and the part stops before it.
+    // It flags the part's first instruction, which the script's start waits for, and the part
+    // stops before it.
     var flags: [32]u8 = @splat(0);
     flags[Test.first] = 1;
     try t.pipe.sendToGame(0x09, &flags);
+    try std.testing.expect(!t.loaded.script.editor.flagged);
     t.session.poll();
+    try std.testing.expect(t.loaded.script.editor.flagged);
     const part = (try t.loaded.bound.file.parts())[0];
     t.loaded.script.runPart(part);
     try std.testing.expectEqual(Test.first, try t.word(.stopped));
@@ -618,12 +629,15 @@ test "the editor asks for the mission's file and the script's state" {
     var text: [4096]u8 = undefined;
     const state = t.pipe.takeFromGame(&text) orelse return error.TestExpectedMessage;
     try std.testing.expectEqual(@backingInt(link.Own.state), state.tag);
-    try std.testing.expectEqualStrings("held", link.field(state.payload, "hold").?);
-    try std.testing.expectEqualStrings(std.fmt.comptimePrint("{d}", .{Test.first}), link.field(state.payload, "thread.0.at").?);
-    try std.testing.expectEqualStrings("0", link.field(state.payload, "thread.0.depth").?);
-    try std.testing.expectEqualStrings("0", link.field(state.payload, "global.1").?);
-    try std.testing.expectEqualStrings("1", link.field(state.payload, "variable.mission_over").?);
-    try std.testing.expectEqual(null, link.field(state.payload, "variable.200"));
+    var arena: std.heap.ArenaAllocator = .init(gpa);
+    defer arena.deinit();
+    const state_read: link.State = try .read(arena.allocator(), state.payload);
+    try std.testing.expectEqual(vm.editor.Hold.held, state_read.hold);
+    try std.testing.expectEqual(Test.first, state_read.threads[0].at.?);
+    try std.testing.expectEqual(0, state_read.threads[0].depth);
+    try std.testing.expectEqual(0, state_read.globals[1]);
+    try std.testing.expectEqual(1, state_read.variable("mission_over").?);
+    try std.testing.expectEqual(null, state_read.variable("200"));
 
     // A tag only the game sends is passed over.
     try t.pipe.sendToGame(@backingInt(link.Own.state), "");