feat: sltool dte decompile writes a mission as Luau source

docs/README.md

@@ -28,6 +28,7 @@ The [user guide](guide/README.md) covers installing, configuring and playing Ope
 | [`formats/bink.md`](formats/bink.md) | `.bik` movies: the Bink container. |
 | [`formats/frc.md`](formats/frc.md) | `.frc` force-feedback effects. |
 | [`formats/dte.md`](formats/dte.md) | `.DTE` missions: directory, ships, triggers, and the script VM. |
+| [`formats/mission-source.md`](formats/mission-source.md) | Mission source: a mission as Luau text, which `sltool dte decompile` writes. |
 | [`formats/stats.md`](formats/stats.md) | Ship, gun, missile and pilot stat tables. |
 | [`formats/save.md`](formats/save.md) | Saved games: the IFF files of the campaign, the autosave and the restart point. |
 | [`formats/profile.md`](formats/profile.md) | `profile.bin`: the pilot's profile, the call sign and the pilot's record. |

docs/formats/dte.md

@@ -66,7 +66,13 @@ the count says how much of the reserved room is filled, so most missions are exa
 
 Sections 17 to 21 and 25 are empty in all 44 missions. The engine reads nothing of sections 9, 20,
 21 and 23: the binder binds 21 into a local variable of its own, and the rest into globals nothing
-reads. Sections 9 and 23 hold records in some missions, likely the original editor's. Of the
+reads. Sections 9 and 23 hold records in some missions, likely the original editor's. Nor does it
+read the operand tables, sections 1, 22 and 26, but for a mission that leaves section 24 unused
+(below): `mission_operand` (`0x004529D0`) is their only reader, and its only caller,
+`mission_build_part_tables` (`0x00452F50`), hands the entry it gives, by the part's `kind`, to
+`mission_fill_part` (`0x00452FD0`), which pops it unread. **Unverified:** that the original reads
+nothing of section 11, which OpenReliant's engine doesn't read. [Mission
+source](mission-source.md#what-the-source-leaves-out) parks these sections. Of the
 directory's 128 slots before the first section, at `0x400`, the binder reads the first 27. Slot 27
 holds the file's size in most missions and is unused in the rest, and slots 28 on are unused in all
 of them. Section 24 holds one word per command of the [catalogue](#commands) (`dte.CommandFlags`):
@@ -716,11 +722,13 @@ the constant table's filler, which hold stale bytes in the shipped missions.
 
 ## Mission source groundwork
 
-Mission source compiles to a standard `.DTE` ([#1126](https://github.com/OpenReliant/openreliant/issues/1126)).
-Its syntax is a subset of Luau, read only as a mission is built, and its model is the one
-[Mission Source v1](https://github.com/OpenReliant/openreliant/issues/608) defines: explicit IDs,
-typed references, display names apart from IDs, and nothing dropped silently. Its script is the
-statements above. The first implemented slice is source identity and typed symbol resolution
+Mission source compiles to a standard `.DTE`
+([#1126](https://github.com/OpenReliant/openreliant/issues/1126)), and [Mission
+source](mission-source.md) specifies it. Its syntax is a subset of Luau, read only as a mission is
+built, and its model is the one [Mission Source
+v1](https://github.com/OpenReliant/openreliant/issues/608) defines: explicit IDs, typed references,
+display names apart from IDs, and nothing dropped silently. Its script is the statements above. The
+first implemented slice is source identity and typed symbol resolution
 ([#609](https://github.com/OpenReliant/openreliant/issues/609)).
 
 [`dte/source.zig`](../../src/formats/dte/source.zig) provides `Symbols(Record)`, a table for one
@@ -743,8 +751,9 @@ The table borrows immutable records and ID bytes. Rebuild it after changing IDs
 order: a reference then resolves to the record's new index. Display names may repeat or change
 without affecting resolution. The caller selects the script bank explicitly.
 
-**Not implemented:** the Luau syntax, its parser, and building and decompiling missions as text,
-which follow under [#1126](https://github.com/OpenReliant/openreliant/issues/1126). The existing
+**Not implemented:** the parser, and building missions from text, which follow under
+[#1126](https://github.com/OpenReliant/openreliant/issues/1126); `sltool dte decompile` writes
+the text. The existing
 binary reader, writer and assembler remain the format implementation.
 
 ## Prior art

docs/formats/mission-source.md

@@ -0,0 +1,297 @@
+# Mission source
+
+Mission source is a mission written as text. `sltool dte decompile` writes a mission's source from
+its `.DTE` file, and `sltool dte build` compiles source into a standard `.DTE` that plays in the
+original too ([#1126](https://github.com/OpenReliant/openreliant/issues/1126)). The syntax is a
+subset of Luau, read as the mission is built: nothing in the file runs, in OpenReliant or anywhere
+else. **Not implemented yet:** `sltool dte build`
+([#1130](https://github.com/OpenReliant/openreliant/issues/1130) writes the text).
+
+```luau
+-- mission1.dte as source.
+
+reliant_group = flight_group("(FG)Reliant", { wing = "player" })
+
+player_ship = ship("Player_Ship", {
+    type = "grendel",
+    group = reliant_group,
+    position = { -11608147, -3, 11596510 },
+    yaw = 90,
+})
+convoy_nav_point = nav_point("Convoy Nav Point", { position = { 1234884, 0, 11312658 }, yaw = 175 })
+
+convoykilled = global("(GV)convoykilled", 0)
+
+on(player_ship, "player_ready_to_jump", { repeats = "always" }, function()
+    jumping_to_convoy()
+    InterruptTriggerCode()
+    if convoykilled == 1 then
+        go_home_total_loss()
+    else
+        jumping_to_sherman()
+    end
+    InterruptTriggerCode()
+    return 1
+end)
+
+jumping_to_convoy = part("(F)Jumping to CONVOY", function()
+    -- ...
+    return 1
+end)
+```
+
+## The file
+
+A mission's source is one file, named `<mission>.mission.luau`, such as `mission1.mission.luau`.
+At its top level it holds declarations and nothing else:
+
+- `name = constructor(...)` declares a record, such as a ship or a part, under `name`.
+- `on(...)` declares a trigger.
+- `mission(...)` gives the mission's own details.
+
+Comments go anywhere, and blank lines don't count. Every name is the file's: a declaration can use
+any name the file declares, before or after it. A name is a Luau identifier, `[A-Za-z_][A-Za-z0-9_]*`,
+that isn't a Luau keyword or one of the names this page gives the source itself.
+
+The order of the declarations of a kind is the order of the mission's records of that kind, since
+the script and the records refer to records by their place: the first ship declared is ship 0, the
+first part part 0, and so on.
+
+## The mission
+
+```luau
+mission({ name = "Patrol" })
+```
+
+`name` is OpenReliant's name for the mission ([OpenReliant's mission
+name](dte.md#openreliants-mission-name)). A mission without the declaration has none.
+
+## Records
+
+Each record has its display name, the name the original's tools and the game show, and fields in a
+table. A field left out takes its default.
+
+A value the game names is written by the name the scripting API gives it
+([Scripting reference](../guide/reference.md)): a ship type as `"grendel"`, a pilot as
+`"bandit_tigers_leader"`, an order as `"find_new_target"`, a condition as `"shot_at"`. The names come
+from the tables the game runs on, so mission source, mods' scripts and the editor share them. A
+value the game doesn't name, such as a ship type or an order a mod adds, stays a number.
+
+### Globals
+
+```luau
+convoykilled = global("(GV)convoykilled", 0)
+```
+
+The display name and the value the global starts with.
+
+### Flight groups
+
+```luau
+reliant_group = flight_group("(FG)Reliant", { wing = "player" })
+```
+
+| Field | Value | Default |
+|---|---|---|
+| `wing` | The wing the group's ships are listed in: `"player"`, `"second"`, `"third"` or `"none"` | `"none"` |
+
+### Ships
+
+```luau
+naginata = ship("(A1)Naginata", {
+    type = "naginata",
+    group = reliant_group,
+    pilot = "bandit_tigers_leader",
+    position = { -11616031, 187, 11597653 },
+    yaw = 90,
+})
+```
+
+`ship` places a ship, a capital ship or a station. `nav_point`, `waypoint`, `point` and `curve_point`
+place the mission's markers ([Ships](dte.md#ships)), which take the same fields but `type`.
+
+| Field | Value | Default |
+|---|---|---|
+| `type` | The ship's type | Required for `ship` |
+| `group` | Its flight group | None |
+| `pilot` | Its pilot, by its record of `pilotstats.bin` | None |
+| `position` | `{ x, y, z }`, in the game's coordinates, Y down | `{ 0, 0, 0 }` |
+| `yaw`, `pitch`, `roll` | Whole degrees; -1 where the mission leaves the angle unset | 0 |
+| `launch` | `{ from = "reliant", gate = 2 }`: the type of ship it launches from, the first of that type in the mission, and the gate it launches through | It doesn't launch |
+| `formation_point` | Its place in a formation: `sabres[2]` is the second point of the formation `sabres` | None |
+| `tier` | The loadout tier its missile racks are fitted by; 255 asks for the campaign's | 255 |
+| `marker` | For a `point` that marks a place on a curve: `{ curve = camera_curve, at = 0.5 }`, the share of the way along it | None |
+
+### Squads
+
+```luau
+sabre_targets = squad("(AI)sabretargets", { sabre_1, sabre_2, component(kurgan, 3), escort_group })
+```
+
+The display name and the members in their order: ships, a ship's component as `component(ship,
+index)`, flight groups and other squads.
+
+### Formations
+
+```luau
+sabres = formation("sabres", { { 0, 0, 0 }, { -400, 0, 300 }, { 400, 0, 300 } })
+```
+
+The display name and the formation's points, each where it stands from the formation's origin. A
+ship names a point as `sabres[1]` for the first.
+
+### Curves
+
+```luau
+camera_curve = curve({
+    start = convoy_nav_point, finish = escort_point,
+    from = { 1234884, 0, 11312658 }, to = { 1240000, 0, 11300000 },
+    leaving_handle = handle_1, arriving_handle = handle_2,
+    leaving = { 10, 0, 0 }, arriving = { 0, 0, -10 },
+})
+```
+
+A curve has no display name. `start` and `finish` are the ships it runs between, `from` and `to`
+their places, `leaving_handle` and `arriving_handle` the ships its tangents are drawn to, and
+`leaving` and `arriving` the tangents ([Curves](dte.md#curves)). A ship left out is none.
+
+### Parts
+
+```luau
+launchfunction = part("(F)launchfunction", function()
+    -- ...
+    return 1
+end, { start = true })
+
+go_home = part("(F)GO HOME (Total Loss)", function()
+    -- ...
+    return 1
+end)
+
+unused = part("(F)Not written yet")
+```
+
+The display name and the part's code, a function whose parameters are the part's arguments. A part
+without a function has no block. `start = true` runs it as the mission starts, before any trigger is
+armed.
+
+### Triggers
+
+```luau
+on(black_guard, "shot_at", { repeats = "always", attacker = any_ship },
+    function(attacker, shield_damage, hull_damage, victim, weapon_fired)
+        if hull_damage >= 50 then
+            -- ...
+        end
+        return 1
+    end)
+```
+
+`on` takes the trigger's subject, the ship, flight group or squad whose events it watches, its
+condition, its fields, which can be left out, and the code its events run. The function's parameters are the values the
+condition's events carry, named after their labels in the condition catalogue
+([Conditions](dte.md#conditions)).
+
+| Field | Value | Default |
+|---|---|---|
+| `repeats` | `"once"`, which disarms the trigger as it fires; `"always"`; or `{ counted = 3 }`, which fires that many times each time the script arms it, with `left = 1` where fewer firings are left as the mission starts | `"once"` |
+| `component` | The subject's component it watches, by its index | The subject itself |
+| `deferred` | `true` leaves the thread to the scheduler rather than running it inside the event | `false` |
+| A value's name | What the event's value must be, for the values the condition checks: a number, a ship, flight group or squad, or `any_ship` for any of the players' ships | Not checked |
+
+The triggers are in the order of their declarations, and a subject's triggers must follow one
+another. A trigger whose subject is `nil` belongs to no object, so it never fires. A trigger without
+a function runs nothing.
+
+Triggers that share their code name it once:
+
+```luau
+convoy_hit = handler(function(attacker, shield_damage, hull_damage, victim, weapon_fired)
+    -- ...
+    return 1
+end)
+on(convoy_1, "shot_at", { repeats = "always" }, convoy_hit)
+on(convoy_2, "shot_at", { repeats = "always" }, convoy_hit)
+```
+
+## The script
+
+The code of parts and triggers is the script's statements ([Statements](dte.md#statements)), in
+Luau's syntax:
+
+| Statement | Source |
+|---|---|
+| A command | `SetAI(sabre_targets, "find_new_target", 1, nil)`. A command the catalogue doesn't have is `command(250, ...)` |
+| A part called | `go_home()`, with its arguments |
+| A part started on a thread of its own | `spawn(go_home)`, with its arguments after the part |
+| An assignment | `convoykilled = 1`, `convoykilled += 1`, and `-=`, `*=` and `/=`. Floating-point stores are `add_assign_f(target, value)` and the like |
+| `if` | `if ... then ... elseif ... else ... end`. An `else` with nothing in it is kept |
+| A random choice | `local roll = random()`, then `if roll < 50 then ... elseif roll < 100 then ... else ... end`: one roll below 100, and the first arm whose threshold is above it, the `else` being the default |
+| `return` | `return 1` |
+
+| Expression | Source |
+|---|---|
+| A number | `50`. Where the parameter takes a named value, it is written by its name: `"find_new_target"` for an order, `"shot_at"` for a condition, and a part's name for a part. A value the game doesn't name stays a number |
+| Text | `"new_sim02.wav"`, as a Luau string |
+| A ship, flight group, squad, curve or global | Its name |
+| A ship's component | `component(ship, 3)` |
+| None | `nil` |
+| One of the game's variables | `game.mission_over`, by its name ([The game's variables](../engine/script-vm.md#the-games-variables)), or `game[37]` for one without |
+| A part's argument | Its parameter |
+| An event's value | Its parameter, in a trigger's function; `thread_value(1)` in a part, for the value of the event that started the thread |
+| A value an object keeps of its last event | `last_event(ship, "shot_at", "Hull Damage")` |
+| The last command's result | `result()` |
+| A comparison | `==`, `~=`, `<`, `<=`, `>`, `>=`; in floating point `greater_f(a, b)` and the like |
+| `and`, `or` | `and`, `or` |
+| Arithmetic | `+`, `-`, `*`, `/`; in floating point `add_f(a, b)` and the like |
+| Membership | `in_flight_group(ship, group)`, `not_in_flight_group`, `in_squad(object, squad)`, `not_in_squad` |
+
+The script has no loops, since a branch only goes forward. A thread waits with commands such as
+`Wait`, and runs again on a timer or a trigger.
+
+## What the source leaves out
+
+The source holds what the game reads of a mission. It leaves out:
+
+- what the game works out again as it binds the mission or starts its script: the ships' runtime
+  places and angles, which equal the authored ones in every shipped mission, the objects' trigger
+  slices, a flight group's ships, the parts' and triggers' places in the script, the string pool,
+  and the directory;
+- the script's flags, which the editor link sets ([The editor link](../engine/editor-link.md)).
+
+The rest is parked as unknown and unused: fields and sections that nothing in the game reads. The
+compiler writes them as the shipped missions most often hold them. OpenReliant could keep data of
+its own there, as it keeps the mission's name in section 21, since the original ignores them.
+**Unverified:** that the original reads none of them; OpenReliant's engine, which follows the
+original's disassembly, reads none.
+
+| Where | What |
+|---|---|
+| Sections 1, 22 and 26 | The operand tables. `mission_build_part_tables` (`0x00452F50`) hands an entry of one, by the part's `kind`, to `mission_fill_part` (`0x00452FD0`), which pops it unread |
+| Sections 9, 11 and 23 | Unread ([Directory](dte.md#directory)) |
+| A ship's bytes `0x06`, `0x16`, `0x1A`, `0x2A`, `0x37`, `0x3C`, `0x3E` and `0x42` | Unread |
+| A trigger's bytes `0x04` to `0x13`, `0x17`, `0x18` and `0x1B`, and its armed flag | Unread; the game arms every trigger an object's slice holds as the mission starts |
+| A trigger's firings left, but for `{ counted = n }` | Read only for a counted trigger |
+| A part's bytes but its name, flags and argument count, and its `kind` | Unread, but for the operand tables above |
+| A curve's last word, a squad member's last bytes, the globals' and the flight groups' unused words | Unread |
+| The objects' tool flags | For the original's tools ([Objects](dte.md#objects)) |
+| A random choice's arms' last byte | Unread |
+
+## References past the records
+
+A reference to a record that the mission doesn't have, such as a curve's ship in some of the
+leftover missions, is written by its index: `ship_index(59031)`, and likewise `flight_group_index`,
+`squad_index`, `curve_index`, `global_index`, `part_index`, `object_index`, `formation_index` and
+`formation_point_index`.
+
+## The round trip
+
+`sltool dte decompile` names each record after its display name: the name in lower case, a leading
+tag such as `(F)` or `(GV)` left out, each run of other characters an underscore, and an underscore
+where a small letter meets a capital, with a number after it where two would clash. The name is the record's from then on, and renaming the display
+name doesn't change it.
+
+Built again, a decompiled mission is the same mission: the same records and routines, decoded, with
+names compared as text and references by what they name. The string pool's layout, its stale
+strings, the sections' layout and the parked bytes may differ. To keep a shipped mission byte for
+byte, edit its `.DTE` in place instead.

ghidra/names/LANCER.EXE.tsv

@@ -69,8 +69,8 @@ address	kind	name	type	comment
 00452a20	function	mission_bind_section		Binds the next directory entry to a section's count and pointer, noting its format flags
 0045cb40	function	mission_alloc_part_tables	void __fastcall (void)	Allocates the part tables, 256 entries each
 0045cb80	function	mission_alloc_table		Frees and reallocates a table of count x size bytes
-00452f50	function	mission_build_part_tables	void __fastcall (void)	Fills the runtime part tables from sections 8 and 17
-00452fd0	function	mission_fill_part	void __fastcall (MissionPart *part, VmFunction *entry)	block = script + start * 2, arguments from the descriptor
+00452f50	function	mission_build_part_tables	void __fastcall (void)	Fills the runtime part tables from sections 8 and 17. For each part of section 8 it hands mission_fill_part the operand mission_operand gives at index 0 of the bank the part's kind names, and 0 for those of section 17
+00452fd0	function	mission_fill_part	void __fastcall (MissionPart *part, VmFunction *entry, void *operand)	block = script + start * 2, arguments from the descriptor. The operand, a third argument on the stack, it pops unread (RET 4), so the game reads nothing of the operand tables
 00453020	function	part_in_table_b	uint __fastcall (MissionPart *part)	Whether a part descriptor lies outside section 8, in section 17
 00525f3c	data	mission_operands_a		Section 1
 005294f8	data	mission_globals	MissionGlobal *	Section 2
@@ -99,7 +99,7 @@ address	kind	name	type	comment
 00525fb4	data	mission_script_b	byte *	Section 18
 00529510	data	mission_script_b_halfwords	ushort	
 00525f9c	data	mission_command_flags	ushort *	Section 24, one per command
-004529d0	function	mission_operand	void * __fastcall (ushort index, char bank)	Operand index of the table bank names: 0 section 1, 1 section 22, 2 section 26
+004529d0	function	mission_operand	void * __fastcall (ushort index, char bank)	Operand index of the table bank names: 0 section 1, 1 section 22, 2 section 26. Its one caller, mission_build_part_tables, hands the operand to mission_fill_part, which pops it unread
 00525278	data	mission_operands_b		Section 22, the second operand table
 0052570c	data	mission_operands_c		Section 26, the third operand table
 00525f18	data	mission_formations	Formation *	Section 14, the formations