Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,14 +83,14 @@ internal/
├── types.go Enums, structs (`Modal`, `ListingRow`, `SubRowKind`, `UndoEntry`, `TextInput`)
├── listing.go `BuildListingRows`: Sub-Row Code View builder, 16-byte `dc.b` coalescer, 3-tier EOL comment priority
├── app.go `App` (`tea.Model`) + segment/status management
├── actions.go Interactive type marking (`c`/`b`/`w`/`l`/`p`/`s`/`u`), carve/uncarve & undo
├── actions.go Interactive type marking (`c`/`b`/`w`/`l`/`p`/`s`/`u`), virtual segment carve/uncarve (`K`), bitplane/music ripper actions (`B`/`M`), & transactional undo (`Ctrl-z`)
├── nav.go Cursor, byte-offset (`←`/`→`), & automatic cross-hunk / jump-vector reference navigation
├── popups.go XREFs (`x`), Functions (`f`), Search (`/`), Structs (`t`), and Segment/Space popups
├── keys.go Keyboard/mouse event handling
├── bitplane.go Interactive Amiga Bitplane Image Viewer (`B`), `BPLCON0`/modulo geometry, planar/EHB/HAM6 decoder, `FORM ILBM` encoder, & `XTWINOPS` (`\x1b[14t`) pixel sizing
├── bitplane.go Interactive Amiga Bitplane Image Viewer (`B`), `BPLCON0`/modulo geometry, planar/EHB/HAM6 decoder, `FORM ILBM` encoder, `XTWINOPS` (`\x1b[14t`) pixel sizing, & block comment round-trip
├── pixelgfx.go Terminal pixel graphics encoders (`Kitty`, `iTerm2`, `Sixel`, Unicode `Half-Block`)
├── theme.go Configurable truecolor `Theme`, `LoadTheme`, and per-token operand/comment styling
└── render.go Fixed-grid 2D `screenBuffer` compositor, scrollbars, floating modals
└── render.go Fixed-grid 2D `screenBuffer` compositor, scrollbars, floating modals (`ModalBitplane`, `ModalMusicRipper`, etc.)
```

## Data Model
Expand Down Expand Up @@ -217,9 +217,9 @@ preserved.
register-context-resolved `d16(An)` hardware accesses are named
(`INTREQR+1` etc.).

## Music Module Recognizer Engine (`internal/modules/`)
## Music Module Recognizer Engine (`internal/modules/`, Experimental)

A self-contained recognizer subsystem (reverse-engineered from ExoticRipper
A self-contained experimental recognizer subsystem (reverse-engineered from ExoticRipper
V2.31) that operates directly on `[]byte` buffers without SQLite, `Project`, or
I/O dependencies:

Expand Down
26 changes: 22 additions & 4 deletions MANUAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ cmp -l assets/disk.1 build/disk_1.bin # 0 differences
| `-m <module> map load FILE` | Bulk-load `<addr> <name> [type]` label mappings from a text file |
| `-m <module> status` | Show module byte coverage statistics |
| `-m <module> tui` | Launch the interactive terminal UI |
| `rip FILE` | Scan a raw binary/memory dump for embedded Amiga music/tracker modules (26 ExoticRipper recognizers) and report detected formats, offsets, and region bounds |
| `rip FILE` | *(Experimental)* Scan a raw binary/memory dump for embedded Amiga music/tracker modules (26 ExoticRipper recognizers) and report detected formats, offsets, and region bounds |

---

Expand Down Expand Up @@ -478,7 +478,7 @@ Instructions that jump/call across hunks or through a register jump-vector table
| `p` | Mark as `POINTER` (4-byte address, resolves target label) |
| `P` | Mark pointer table (prompts for entry count) |
| `s` | Mark as null-terminated ASCII `STRING` |
| `K` | Carve v-selection into an `incbin` block (prompts for a name); with no selection, uncarve the carved range at the cursor. Refuses interior relocations in relocatable segments |
| `K` | Carve v-selection into an `incbin` block (prompts for a label name and jumps cursor to the carve start); with no selection, uncarve the carved range at the cursor (undoable via `Ctrl-z`). Refuses interior relocations in relocatable segments |
| `u` | Undefine cell(s) (revert to raw unmapped bytes) |

### Annotation Keys
Expand All @@ -502,8 +502,9 @@ Instructions that jump/call across hunks or through a register jump-vector table
| `m` | Memory map region editor (`d` to delete region) |
| `t` | Struct editor (`n` = new struct, `Enter` = edit struct fields, `a` = add field, `d` = delete, `Esc` = back) |
| `a` | Register alias editor (maps a base register to a struct for automatic `d16(An)` field substitution) |
| `Ctrl-z` | Undo last type-marking action |
| `Ctrl-z` | Undo last edit (type-marking, carve/uncarve, rename, inline/block comment) |
| `B` | Open interactive Bitplane Image Viewer at cursor address |
| `M` | *(Experimental)* Open interactive Music Module Ripper (scan, browse, jump, rip, carve) |
| `?` | Help popup |
| `q` | Quit |

Expand Down Expand Up @@ -550,9 +551,26 @@ The viewer queries the terminal text area pixel dimensions at startup via the cr
| `v` | Cycle terminal graphics protocol (`Kitty` $\rightarrow$ `iTerm2` $\rightarrow$ `Sixel` $\rightarrow$ `Half-Block`) |
| `w` | Export unscaled bitmap as standard PNG to `assets/<seg>_<addr>_<WxHxD>.png` |
| `W` | Export bitmap as standard Commodore Amiga `FORM ILBM` file to `assets/<seg>_<addr>_<WxHxD>.ilbm` |
| `K` | Carve bitplane byte range in database as Data (`DAT`) with label `gfx_<W>x<H>x<D>` (undoable via `Ctrl-z`) |
| `K` | Carve bitplane byte range as `incbin` Virtual Segment (prompts for `gfx_<W>x<H>x<BPU>[i][h][ham]` label, writes `; Bitmap:` / `; Span:` / `; Regs:` / `; Planes:` / `; Pal:` block comment above the label; round-trips geometry, modulo, `VIS`/`SYNC` masks, plane pointers, and palette/invert/bias on `B`; flat binaries only; undoable via `Ctrl-z`) |
| `Esc` / `q` | Close viewer and return immediately to disassembly listing |

### Interactive Music Module Ripper (`M`) — Experimental

> **Note:** The Music Module Ripper (`M` in TUI and `hunks rip`) is currently **experimental**.

Pressing `M` in the Disassembly view scans the active segment using the zero-dependency recognizer engine (`modules.Scan`) for embedded tracker and sound driver modules across 26 ExoticRipper-derived formats (including ProTracker, SoundTracker 2.6, Old SoundTracker, Future Composer, David Whittaker, Delta Music 1.0, Final Music System, ThePlayer, TFMX, OctaMED, and Jochen Hippel).

If modules are detected, `ModalMusicRipper` displays all found songs and companion regions (e.g. StarTrekker sample banks) with format, variant, offset, and size:

| Key | Action |
|-----|--------|
| `j` / `k` or `↓` / `↑` | Select detected module region |
| `PgUp` / `PgDn` / `Home` / `End` | Page / jump through detected module regions |
| `Enter` | Jump disassembly cursor directly to the module's start address |
| `w` | Rip / export the selected module directly to `assets/<slug>_<offset>.mod` (or `.bin`, keyed on format ID) |
| `K` | Carve module byte range as `incbin` Virtual Segment (prompts for `mod_<slug>_<offset>` label, writes `; Music:` / `; Span:` block comment above the label; flat binaries only; undoable via `Ctrl-z`) |
| `Esc` / `q` | Close ripper and return to disassembly listing |

### Theme Customization (`~/.config/hunks/theme.json`)

The TUI uses a built-in dark truecolor palette out of the box. You can override any subset of token styles in `~/.config/hunks/theme.json` (or via `$HUNKS_THEME`) using `"<fg> [bg:<bg>] [bold] [italic] [reverse]"` specifications:
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ Amiga M68k Reverse Engineering System — terminal-native, keyboard-first disass
- **Relocation-exact symbol resolution** (`HUNK_RELOC32` / `HUNK_DREL32`), automatic cross-hunk `XDEF` exports, and collision-free `LAB_<hunk>_<offset>` auto-labels (`LAB_5_24BE8`)
- **Source-linkable `; XREF:` comments** (`; XREF: hunk5:LAB_5_24E84 (jsr 498(a5))`) and automatic cross-hunk / jump-vector target hints (`; -> hunk3:LAB_3_26432`)
- **Static Call Graph CLI (`hunks -m <module> cg`)** with automatic `d16(a5)` jump-vector resolution, bottom-up `--ready` worklists, `--leaves`, `--anchors` (strings & `$DFFxxx` hardware), `--suspect` audits, and `--code` caller/callee inspection
- **Standalone Amiga music module scanner (`hunks rip <file>`)** with 26 ExoticRipper recognizers (including sandboxed self-executing M68000 driver probes for David Whittaker and Final Music System)
- **Interactive TUI (`hunks -m <module> tui`)** with full-width Sub-Row Code View, inline 16-byte `dc.b` hex/ASCII grid, 3-tier EOL comment priority, exact MC68000 instruction cycle & R/W bus timing, interactive **Amiga Bitplane Image Viewer & Ripper (`B`)** (Kitty/iTerm2/Sixel/Half-Block pixel graphics, `BPLCON0` 1–7 planes, EHB/HAM6, Copper/`$0RGB` palette extraction, PNG & `FORM ILBM` export), configurable truecolor themes (`~/.config/hunks/theme.json`), vertical scrollbar, floating modal overlays, byte-exact cursor splitting, cross-hunk `Enter` navigation, `x` XREFs modal, `f` Functions filter, and struct/alias editors
- **Standalone Amiga music module scanner (`hunks rip <file>`, experimental)** with 26 ExoticRipper recognizers (including sandboxed self-executing M68000 driver probes for David Whittaker and Final Music System)
- **Interactive TUI (`hunks -m <module> tui`)** with full-width Sub-Row Code View, inline 16-byte `dc.b` hex/ASCII grid, 3-tier EOL comment priority, exact MC68000 instruction cycle & R/W bus timing, interactive **Amiga Bitplane Image Viewer & Ripper (`B`)** (Kitty/iTerm2/Sixel/Half-Block pixel graphics, `BPLCON0` 1–7 planes, EHB/HAM6, Copper/`$0RGB` palette extraction, PNG & `FORM ILBM` export), experimental **TUI Music Ripper (`M`)** (scan, jump, rip, & carve), virtual segment `incbin` carves (`v` + `K` in editor, `K` in asset viewers with metadata block comments and round-trip on `B`), configurable truecolor themes (`~/.config/hunks/theme.json`), vertical scrollbar, floating modal overlays, byte-exact cursor splitting, cross-hunk `Enter` navigation, `x` XREFs modal, `f` Functions filter, and struct/alias editors
- **100% bit-for-bit reassembly** verified with `vasmm68k_mot`

## Quick Start
Expand Down
3 changes: 2 additions & 1 deletion TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,7 @@ Verify:
- `/` search; `f` Functions; `x` XREFs (`Enter` jumps to caller).
- `Enter` follows branches, cross-hunk refs (`⇢`), and jump-vector hints; `-`/`Bksp` returns.
- `←`/`→` moves the byte cursor on `DATA` rows or cycles caller badges on `; XREF` rows; `s`/`b`/`w`/`l`/`p`/`c`/`u` mark cells; `K` carve/uncarve; `Ctrl-z` undo.
- `B` opens the Bitplane Image Viewer (`K` carves as `incbin` with metadata block comment and round-trips on `B`); `M` opens the experimental Music Module Ripper (`Enter` jumps, `w` rips to `assets/`, `K` carves as `incbin`).
- `?` Help; `q` quit.

---
Expand Down Expand Up @@ -158,7 +159,7 @@ Expected:
- (b) carve over an interior reloc in a relocatable hunk **exits non-zero** with
a clean refusal (no panic).
- (c) carve on the flat part succeeds (guard does not apply to non-relocatable parts).
- (d) `uncarve` empties `project.json` `carves`.
- (d) `uncarve` removes the `INCBIN` cell, label, and block comment from `project.db`.

### Carve byte-neutrality on relocatable hunks
Carve a **reloc-free DATA range**, emit, re-assemble, confirm load-image parity:
Expand Down
60 changes: 24 additions & 36 deletions cmd/hunks/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ import (
"github.com/spf13/cobra"
)

var version = "dev"
var version = "v1.1.0"

func parseHex(val string) (uint32, error) {
return project.ParseUint32(val)
Expand Down Expand Up @@ -1788,17 +1788,21 @@ func main() {
segBytes := seg.LoadBinaryData()
off := int(startVal - seg.Base)
endOff := int(endVal - seg.Base)
if endOff > len(segBytes) {
if off < 0 || endOff > len(segBytes) {
return fmt.Errorf("carve range exceeds segment payload")
}
raw := slices.Clone(segBytes[off:endOff])

label := carveName
isAuto := false
if label == "" {
label = proj.AutoLabel(startVal)
isAuto = true
}
sidecar := carveFile
if sidecar == "" {
var opStr *string
if sidecar != "" {
opStr = &sidecar
} else {
sidecar = fmt.Sprintf("assets/%s.bin", label)
}

Expand All @@ -1815,27 +1819,15 @@ func main() {
Size: endVal - startVal,
Type: db.CellIncbin,
Confidence: 255,
RawBytes: raw,
OpStr: opStr,
}); err != nil {
return err
}
return ctrl.SetLabel(startVal, label, db.LabelDat, false)
return ctrl.SetLabel(startVal, label, db.LabelDat, isAuto)
}); err != nil {
return err
}

proj.Data.Carves = slices.DeleteFunc(proj.Data.Carves, func(c project.Carve) bool {
return c.Start == startVal
})
proj.Data.Carves = append(proj.Data.Carves, project.Carve{
Start: startVal,
End: endVal,
File: sidecar,
Name: label,
})
if err := proj.Save(); err != nil {
return err
}
fmt.Printf("Carved [%#X..%#X] (%d bytes) in '%s' -> %s (label %s)\n",
startVal, endVal, endVal-startVal, seg.Name, sidecar, label)
return nil
Expand All @@ -1862,17 +1854,6 @@ func main() {
if err != nil {
return err
}
var foundCarve *project.Carve
for i := range proj.Data.Carves {
if proj.Data.Carves[i].Start == startVal {
foundCarve = &proj.Data.Carves[i]
break
}
}
if foundCarve == nil {
return fmt.Errorf("no carve starts at $%X", startVal)
}
carve := *foundCarve
seg := proj.SegmentAtAddr(startVal)
if seg == nil {
return fmt.Errorf("no segment owns address $%X", startVal)
Expand All @@ -1884,17 +1865,24 @@ func main() {
defer ctrl.Close()
proj.ActiveSegmentName = seg.Name
proj.SyncDBScope(ctrl)
if err := ctrl.DeleteCellsInRange(carve.Start, carve.End); err != nil {
return err
cell, err := ctrl.GetCell(startVal)
if err != nil || cell == nil || cell.Type != db.CellIncbin || cell.Size == 0 {
return fmt.Errorf("no carve starts at $%X", startVal)
}
proj.Data.Carves = slices.DeleteFunc(proj.Data.Carves, func(c project.Carve) bool {
return c.Start == startVal
})
if err := proj.Save(); err != nil {
carveEnd := cell.Addr + cell.Size
if err := ctrl.WithTransaction(func() error {
if err := ctrl.DeleteCellsInRange(cell.Addr, carveEnd); err != nil {
return err
}
if err := ctrl.DeleteLabelAt(cell.Addr); err != nil {
return err
}
return ctrl.UpdateBlockComment(cell.Addr, "")
}); err != nil {
return err
}
fmt.Printf("Uncarved [%#X..%#X] in '%s'. Run `hunks analyze` to reclassify the range.\n",
carve.Start, carve.End, seg.Name)
cell.Addr, carveEnd, seg.Name)
return nil
},
}
Expand Down
24 changes: 14 additions & 10 deletions internal/db/controller.go
Original file line number Diff line number Diff line change
Expand Up @@ -1861,22 +1861,23 @@ func (c *Controller) ClearOverlaps(addr, size uint32) error {
end := addr + size

type partialCell struct {
addr uint32
size uint32
raw []byte
conf uint8
addr uint32
size uint32
typeStr string
raw []byte
conf uint8
}
var partials []partialCell
rows, err := c.conn.Query(
"SELECT addr, size, raw_bytes, confidence FROM cells WHERE seg = ?1 AND addr < ?3 AND addr + size > ?2",
"SELECT addr, size, type, raw_bytes, confidence FROM cells WHERE seg = ?1 AND addr < ?3 AND addr + size > ?2",
seg, addr, end,
)
if err != nil {
return err
}
for rows.Next() {
var p partialCell
if err := rows.Scan(&p.addr, &p.size, &p.raw, &p.conf); err != nil {
if err := rows.Scan(&p.addr, &p.size, &p.typeStr, &p.raw, &p.conf); err != nil {
_ = rows.Close()
return err
}
Expand All @@ -1901,11 +1902,14 @@ func (c *Controller) ClearOverlaps(addr, size uint32) error {
return err
}

// Remnant head/tail bytes are reinserted as individual DATA_B cells (one per byte) to
// preserve exact emission/label boundaries. ClearOverlaps is called both standalone
// (TUI) and inside an outer transaction (stage2, CLI pointer hints), so it must not open
// its own transaction here — doing so would nest BEGIN on the single SQLite connection.
// For small cells (<= 16 bytes, non-INCBIN), remnant head/tail bytes are reinserted as
// individual DATA_B cells to preserve exact emission/label boundaries. Large cells
// (INCBIN, multi-KB DATA_ZERO, long STRING) are never shattered into thousands of 1-byte
// rows; their unmapped remnants fall through naturally to BinaryData.
for _, p := range partials {
if p.typeStr == CellIncbin.AsStr() || p.size > 16 {
continue
}
cEnd := p.addr + p.size
if p.addr < addr {
for bAddr := p.addr; bAddr < addr; bAddr++ {
Expand Down
11 changes: 6 additions & 5 deletions internal/emitter/emitter.go
Original file line number Diff line number Diff line change
Expand Up @@ -751,11 +751,12 @@ func (e *Emitter) PopulateCellsItems() error {

switch cell.Type {
case db.CellIncbin:
file := fmt.Sprintf("assets/incbin_%08X.bin", addr)
for _, c := range e.Proj.Data.Carves {
if c.Start == addr {
file = c.File
break
file := opStr
if file == "" {
if lbl := getLabelStr(addr); lbl != "" {
file = fmt.Sprintf("assets/%s.bin", lbl)
} else {
file = fmt.Sprintf("assets/incbin_%08X.bin", addr)
}
}
outPath := file
Expand Down
13 changes: 13 additions & 0 deletions internal/modules/modules.go
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,19 @@ func (m *Match) TotalLen() int {
return total
}

// FileExtForFormatID returns the canonical file extension ("mod" or "bin") for a FormatID and RegionRole.
func FileExtForFormatID(formatID uint32, role RegionRole) string {
if role != RegionPrimary {
return "bin"
}
switch formatID {
case 22, 23, 42: // SoundTracker 2.6, ProTracker, Old SoundTracker
return "mod"
default:
return "bin"
}
}

// Recognizer is a single-format detector. Pure: inspects data at/around offset and returns
// a Match iff a valid module of this format is present.
type Recognizer interface {
Expand Down
52 changes: 0 additions & 52 deletions internal/project/project.go
Original file line number Diff line number Diff line change
Expand Up @@ -140,53 +140,6 @@ func (m *MemoryRegion) UnmarshalJSON(data []byte) error {
return nil
}

type Carve struct {
Start uint32 `json:"start"`
End uint32 `json:"end"`
File string `json:"file"`
Name string `json:"name"`
}

func (c Carve) MarshalJSON() ([]byte, error) {
type wire struct {
Start string `json:"start"`
End string `json:"end"`
File string `json:"file"`
Name string `json:"name"`
}
return json.Marshal(wire{
Start: formatHex(c.Start),
End: formatHex(c.End),
File: c.File,
Name: c.Name,
})
}

func (c *Carve) UnmarshalJSON(data []byte) error {
var wire struct {
Start json.RawMessage `json:"start"`
End json.RawMessage `json:"end"`
File string `json:"file"`
Name string `json:"name"`
}
if err := json.Unmarshal(data, &wire); err != nil {
return err
}
start, err := parseHexJSON(wire.Start)
if err != nil {
return err
}
end, err := parseHexJSON(wire.End)
if err != nil {
return err
}
c.Start = start
c.End = end
c.File = wire.File
c.Name = wire.Name
return nil
}

type Segment struct {
Name string `json:"name"`
HunkIndex int `json:"hunk_index"`
Expand Down Expand Up @@ -354,7 +307,6 @@ type ProjectData struct {
MemoryRegions []MemoryRegion `json:"memory_regions"`
Structs map[string]map[string]string `json:"structs"`
RegisterAliases map[string]string `json:"register_aliases"`
Carves []Carve `json:"carves"`
Analysis AnalysisData `json:"_analysis"`
}

Expand All @@ -376,7 +328,6 @@ func New(filepath string) *Project {
MemoryRegions: []MemoryRegion{},
Structs: make(map[string]map[string]string),
RegisterAliases: make(map[string]string),
Carves: []Carve{},
Analysis: AnalysisData{
IndirectAddressing: []string{},
},
Expand Down Expand Up @@ -445,9 +396,6 @@ func (p *Project) Load() error {
if p.Data.MemoryRegions == nil {
p.Data.MemoryRegions = []MemoryRegion{}
}
if p.Data.Carves == nil {
p.Data.Carves = []Carve{}
}
if p.Data.Analysis.IndirectAddressing == nil {
p.Data.Analysis.IndirectAddressing = []string{}
}
Expand Down
Loading
Loading