diff --git a/projects/TFMCCore/README.md b/projects/TFMCCore/README.md index ba7f758..a2ccb11 100644 --- a/projects/TFMCCore/README.md +++ b/projects/TFMCCore/README.md @@ -8,7 +8,7 @@ TFMC runs Minecraft **1.21.10**. See the [shared platform and build baseline](.. - [Overview](overview.md) - [Shared feature ownership](ownership.md) -- [Three-part resource-pack delivery](resource-pack.md) +- [Multipart resource-pack delivery](resource-pack.md) - [Statistics architecture](src/main/java/net/tfminecraft/tfmccore/stats/ARCHITECTURE.md) - [Statistics configuration and commands](src/main/java/net/tfminecraft/tfmccore/stats/STATS.md) - [Player commands](src/main/java/net/tfminecraft/tfmccore/tfmc/TFMC.md) diff --git a/projects/TFMCCore/resource-pack.md b/projects/TFMCCore/resource-pack.md index 2a93fd6..798be62 100644 --- a/projects/TFMCCore/resource-pack.md +++ b/projects/TFMCCore/resource-pack.md @@ -1,13 +1,47 @@ -# Three-part resource-pack delivery +# Multipart resource-pack delivery [TFMCCore](README.md) · [All projects](../../README.md) -TFMC's pack spends substantial client reload time scanning ZIP entries. The opt-in -multipart path partitions the completed ItemsAdder archive by whole namespaces: -`core`, `models` (ModelEngine), and `sounds` (namespaces containing only -`sounds.json`). Resource identifiers, bytes, overlay order and protected ZIP -records remain unchanged. Overlay compaction (`resource-pack.compact-overlays`) -still runs during `/iazip`. +Most of a join's pack time is the client rebuilding its resources, not the download. +The opt-in multipart path partitions the completed ItemsAdder archive by whole +namespaces into five packs: `core`, `core2` and `core3`, `models` (ModelEngine), and +`sounds` (namespaces containing only `sounds.json`). Resource identifiers, overlay +order and protected ZIP records remain unchanged. Overlay compaction +(`resource-pack.compact-overlays`) still runs during `/iazip`. + +## Why the pack is split this way + +A 1.21 client reads every entry of a pack ZIP once per namespace that pack holds +each time it lists resources, and only one thread can read a ZIP at a time. Main's +core namespaces hold 19,795 entries across 33 namespaces. The publisher sorts the core namespaces +by entry count and splits them into the three runs with the lowest total of +namespaces × entries, so the large namespaces sit in parts of their own and the many +small ones share a small part. That cuts listing work about five times. + +The client also checks every provider of a font for every code point it knows, +including unifont's ~60,000. ItemsAdder writes one bitmap provider per font image, +616 in `minecraft:default` and again in `minecraft:uniform`, enough to block the +render thread for over 2 seconds on each reload. The publisher merges one-glyph providers +that share an image size and every other setting (height, ascent, filter) into grid +sheets under `textures/tfmc_font_grid/` in the font's namespace. Each glyph keeps its +exact pixels, so the client computes the same width. Glyphs are left as they are +when their texture is not in the pack (the `mcemojis` vanilla item emojis use +vanilla textures), an overlay replaces their texture, their code point is defined +twice, their PNG is not 8-bit RGB, RGBA or palette, or the font contains providers +whose code points cannot be listed (references, TrueType, unihex). Glyphs are also +left alone when stb_image, the client's decoder, would reject their PNG, when the +font file inflates past 4 MiB or the texture past 8 MiB, when the image exceeds +512 × 512 pixels, or once that font's merged glyphs reach 4 million pixels. Sheets +are at most 2048 × 2048 pixels. +Only plainly laid-out PNGs are merged (palette chunks on palette images before the +data, one run of image data, the zlib trailer present). Rewritten fonts and their +sheets may total 16 MiB per build; a font that would take the total past that stays +as ItemsAdder wrote it. +On Main this takes 616 providers to 309. + +Merged fonts and their sheets are written as ordinary ZIP records; every other +record is copied as ItemsAdder wrote it. The output is deterministic, so a restart +republishes the same hashes and `models` and `sounds` keep theirs. The implementation is in [`net.tfminecraft.tfmccore.resourcepack`](https://github.com/TF-Minecraft/TFMCCore/tree/main/src/main/java/net/tfminecraft/tfmccore/resourcepack), @@ -17,8 +51,8 @@ and the defaults are in ## Build and delivery The background publisher checks the completed `generated.zip` against ItemsAdder's -cached SHA-1. It copies a snapshot, verifies the hash, writes all three parts into -a temporary directory and atomically publishes the directory. Requests use three +cached SHA-1. It copies a snapshot, verifies the hash, writes every part into a +temporary directory and atomically publishes the directory. Requests use the parts' URLs and hashes from that single generation. An unfinished or unsupported build uses ItemsAdder's single-pack delivery. Packs containing `filter` metadata, ZIP64, or missing the expected namespace groups are not split. @@ -38,8 +72,8 @@ without `/iazip`. It detects later rebuilds automatically. It does not itself rebuild ItemsAdder's content on restart. Configuration changes below require a restart; `/tcore reload config` does not restart the HTTP listener. -After `/iazip`, ItemsAdder's automatic send is replaced with the matching three -parts once publication completes. No `/tfmc pack` or reconnect is needed. +After `/iazip`, ItemsAdder's automatic send is replaced with every matching part +once publication completes. No `/tfmc pack` or reconnect is needed. ItemsAdder still selects the recipients, so `--apply-to self` and `--apply-to none` retain their meaning. `/iatexture` uses the same bridge. Only the configured ItemsAdder pack UUID in the play phase is intercepted; other packs and login @@ -104,7 +138,7 @@ Activate and verify on Dev before Main. activation if either setting is incompatible. ItemsAdder 4.0.18 applies its equipment-hiding loading effect once for each - accepted pack, but removes it only once per player. With three packs, leftover + accepted pack, but removes it only once per player. With several packs, leftover entries repeatedly hide the selected hotbar/held item after loading finishes. Disconnecting removes only one entry, so reconnecting may not clear it. Disabling `lock-player` avoids that visual/movement-lock effect; the separate @@ -138,11 +172,13 @@ Activate and verify on Dev before Main. 5. Restart the server and verify the published-generation log, HTTP hashes, real joining, reconnecting, and `/iazip` regeneration. -Parts are kept under `plugins/TFMCCore/resource-pack/parts-v1//`. +Parts are kept under `plugins/TFMCCore/resource-pack/parts-v2//`. +Each layout of the split has its own folder, so an older layout of the same +ItemsAdder build (`parts-v1` held three parts) never blocks publishing. Published generations are immutable and retained so in-flight clients can finish an older download. They occupy approximately one full pack per rebuild; remove -old generations during maintenance when no clients are downloading them. The -original ItemsAdder archive is never overwritten. +old generations, and old layout folders, during maintenance when no clients are +downloading them. The original ItemsAdder archive is never overwritten. ## Rollback @@ -167,3 +203,10 @@ The offline prototype applied in 11–12 seconds versus 21–23 seconds for the compacted single ZIP on the tested CachyOS 1.21.10 client. This is application time, not a promise about download or total join time. Production Java output also passed exact resource-map comparison at 24 format boundaries. + +On a vanilla 1.21.10 client with Main's pack (CachyOS, 16 threads), a warm reload +took 8.0 seconds with one core part and 616-provider fonts, of which 2.7 seconds +blocked the render thread. The five-part layout with merged fonts took 3.0 seconds, +with 1.1 seconds blocked; the client logged the same warnings, and every glyph's +cell, height, ascent and filter matched. Most of the remaining block is the 247 +vanilla item emoji glyphs. Without the pack the same client reloads in 0.3 seconds.