From da73c1015d861db5059d2ed1df372e10a6594f10 Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 20:06:10 -0700 Subject: [PATCH 1/8] Rename Why Filepack? to Features --- book/src/introduction.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/book/src/introduction.md b/book/src/introduction.md index 4c88951a..75e133a0 100644 --- a/book/src/introduction.md +++ b/book/src/introduction.md @@ -8,11 +8,11 @@ started, [Packages](packages.md) for an overview of Filepack packages, and [Why is metadata important?](why-metadata.md) for some of the higher-level ideas behind Filepack. -Why Filepack? -------------- +Features +-------- -If you package content for distribution, Filepack offers a number of benefits, -even if you already use [SFV +If you package content for distribution, Filepack offers a number of useful +features, even if you already use [SFV files](https://en.wikipedia.org/wiki/Simple_file_verification): - Filepack can detect both accidental corruption and intentional modification, From 0e04e2ce2bdcfff86a115e3a81a7b9eb65d19e73 Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 20:09:49 -0700 Subject: [PATCH 2/8] Add why chapter --- README.md | 3 +-- book/src/SUMMARY.md | 2 +- book/src/introduction.md | 5 ++--- book/src/{why-metadata.md => why.md} | 5 ++++- crates/generate/templates/summary.md | 2 +- 5 files changed, 9 insertions(+), 8 deletions(-) rename book/src/{why-metadata.md => why.md} (98%) diff --git a/README.md b/README.md index c63e642b..131f8912 100644 --- a/README.md +++ b/README.md @@ -43,8 +43,7 @@ Documentation is available at [filepack.com/book/](https://filepack.com/book/). See [Installation](https://filepack.com/book/installation.html) and [Quickstart](https://filepack.com/book/quickstart.html) to get started, [Packages](https://filepack.com/book/packages.html) for an overview of Filepack -packages, and [Why is metadata -important?](https://filepack.com/book/why-metadata.html) for some of the +packages, and [Why?](https://filepack.com/book/why.html) for some of the higher-level ideas behind Filepack. Lints diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index c051a507..a4bd0eb5 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -14,7 +14,7 @@ Summary - [Server](server.md) # Background -- [Why is metadata important?](why-metadata.md) +- [Why?](why.md) - [Alternatives and Prior Art](alternatives.md) # Reference diff --git a/book/src/introduction.md b/book/src/introduction.md index 75e133a0..942cc5de 100644 --- a/book/src/introduction.md +++ b/book/src/introduction.md @@ -4,9 +4,8 @@ Introduction {{#include ../../README.md:introduction}} See [Installation](installation.md) and [Quickstart](quickstart.md) to get -started, [Packages](packages.md) for an overview of Filepack packages, and [Why -is metadata important?](why-metadata.md) for some of the higher-level ideas -behind Filepack. +started, [Packages](packages.md) for an overview of Filepack packages, and +[Why?](why.md) for some of the higher-level ideas behind Filepack. Features -------- diff --git a/book/src/why-metadata.md b/book/src/why.md similarity index 98% rename from book/src/why-metadata.md rename to book/src/why.md index c0d03927..25d9fb6c 100644 --- a/book/src/why-metadata.md +++ b/book/src/why.md @@ -1,5 +1,8 @@ +Why? +==== + Why is metadata important? -========================== +-------------------------- Metadata is the difference between a good user experience and a bad user experience. diff --git a/crates/generate/templates/summary.md b/crates/generate/templates/summary.md index 2ca5a827..c8d849fc 100644 --- a/crates/generate/templates/summary.md +++ b/crates/generate/templates/summary.md @@ -14,7 +14,7 @@ Summary - [Server](server.md) # Background -- [Why is metadata important?](why-metadata.md) +- [Why?](why.md) - [Alternatives and Prior Art](alternatives.md) # Reference From 837aaa07964ec3dbf648d5036d05101cf59dcfb6 Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 20:10:30 -0700 Subject: [PATCH 3/8] Combine background and guide --- book/src/SUMMARY.md | 4 +--- crates/generate/templates/summary.md | 4 +--- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index a4bd0eb5..15559267 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -9,12 +9,10 @@ Summary - [Quickstart](quickstart.md) # Guide +- [Why?](why.md) - [Packages](packages.md) - [Configuration](configuration.md) - [Server](server.md) - -# Background -- [Why?](why.md) - [Alternatives and Prior Art](alternatives.md) # Reference diff --git a/crates/generate/templates/summary.md b/crates/generate/templates/summary.md index c8d849fc..b454ff19 100644 --- a/crates/generate/templates/summary.md +++ b/crates/generate/templates/summary.md @@ -9,12 +9,10 @@ Summary - [Quickstart](quickstart.md) # Guide +- [Why?](why.md) - [Packages](packages.md) - [Configuration](configuration.md) - [Server](server.md) - -# Background -- [Why?](why.md) - [Alternatives and Prior Art](alternatives.md) # Reference From 8fedee5437ae0a7421c4229d27013d3bf056ddbb Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 20:12:43 -0700 Subject: [PATCH 4/8] Rename alternatives and prior art to comparisons --- book/src/SUMMARY.md | 2 +- book/src/{alternatives.md => comparisons.md} | 4 ++-- crates/generate/templates/summary.md | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) rename book/src/{alternatives.md => comparisons.md} (97%) diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 15559267..162fcc98 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -13,7 +13,7 @@ Summary - [Packages](packages.md) - [Configuration](configuration.md) - [Server](server.md) -- [Alternatives and Prior Art](alternatives.md) +- [Comparisons](comparisons.md) # Reference - [API](api.md) diff --git a/book/src/alternatives.md b/book/src/comparisons.md similarity index 97% rename from book/src/alternatives.md rename to book/src/comparisons.md index 2c5317d9..75fc1cb1 100644 --- a/book/src/alternatives.md +++ b/book/src/comparisons.md @@ -1,5 +1,5 @@ -Alternatives and Prior Art -========================== +Comparisons +=========== `filepack` serves the same purpose as programs like `shasum`, which hash files and output a text file containing file hashes and paths, which can later be diff --git a/crates/generate/templates/summary.md b/crates/generate/templates/summary.md index b454ff19..046a5842 100644 --- a/crates/generate/templates/summary.md +++ b/crates/generate/templates/summary.md @@ -13,7 +13,7 @@ Summary - [Packages](packages.md) - [Configuration](configuration.md) - [Server](server.md) -- [Alternatives and Prior Art](alternatives.md) +- [Comparisons](comparisons.md) # Reference - [API](api.md) From 70aa4d6f05d1456b297dcebb514807d714bdeed4 Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 21:17:15 -0700 Subject: [PATCH 5/8] Add content addressed distribution to storage --- book/src/comparisons.md | 71 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/book/src/comparisons.md b/book/src/comparisons.md index 75fc1cb1..8a230fc3 100644 --- a/book/src/comparisons.md +++ b/book/src/comparisons.md @@ -1,6 +1,9 @@ Comparisons =========== +Hashing +------- + `filepack` serves the same purpose as programs like `shasum`, which hash files and output a text file containing file hashes and paths, which can later be used with the same program to verify that the files have not changed. @@ -27,6 +30,9 @@ hash function, but is now known to be insecure. `filepack` and `b3sum` both use BLAKE3, a fast, general-purpose cryptographic hash function. +Signing +------- + `filepack` can also create and verify signatures. Other signing and verification utilities include: @@ -39,3 +45,68 @@ verification utilities include: | [`SignTool`](https://learn.microsoft.com/en-us/windows/win32/seccrypto/signtool) | Windows code signing | | [`codesign`](https://developer.apple.com/library/archive/documentation/Security/Conceptual/CodeSigningGuide/Procedures/Procedures.html) | macOS code signing | | [`jarsigner`](https://docs.oracle.com/javase/8/docs/technotes/tools/windows/jarsigner.html) | JDK code signing | + +Content-addressed Distribution +------------------------------ + +`filepack` packages are Merkle trees over their contents, which allows them to +be used for content-addressed storage and distribution. + +### BitTorrent + +Filepack packages are similar to +[BitTorrent](https://en.wikipedia.org/wiki/BitTorrent) torrents. + +[Version 1 torrents](https://www.bittorrent.org/beps/bep_0003.html) use the +[SHA-1](https://en.wikipedia.org/wiki/SHA-1) hash function, now known to be +cryptographically broken. Files do not have consistent hashes between torrents. + +[Version 2 torrents](https://www.bittorrent.org/beps/bep_0052.html) use the +[SHA-2](https://en.wikipedia.org/wiki/SHA-2) hash function with a Merkle tree +construction that does give files consistent hashes between torrents. + +Version 1 torrents are currently far more widely adopted than version 2 +torrents. + +Torrents are not naively composable. A torrent may contain `.torrrent` files, +but may not naively contain other torrents. + +Compared to BitTorrent, Filepack uses BLAKE3, a fast, modern cryptographic hash +function with built-in Merkle tree construction. Files are consistently hashed +regardless of the package they appear in. Packages are composable. Package +fingerprints identify the root directory of the package, which may itself +contain packages in subdirectories. + +Filepack uses a more consistent content-addressing scheme. All objects, +including files and directories, are identified by BLAKE3 hash, which is itself +a Merkle tree, allowing any object to be incrementally streamed and verified. +By contrast, a BitTorrent infohash is not a tree hash, so the info dictionary +object it identifies cannot be downloaded using the same piece bartering +algorithm as the files in the torrent. + +Filepack packages are directories of files, so packages can be transported +using BitTorrent. Additionally, Filepack metadata identifies assets by relative +path, so Filepack metadata can be used in torrents independently of Filepack +hashing or manifests. + +### IPFS + +Filepack and IPFS use a similar data model, where directories files and files +are both identified by hash. + +IPFS is notoriously complex, and supports a wide variety of hash functions, +public-key cryptography algorithms, and peer to peer transport and addressing +schemes. + +Compared to IPFS, Filepack is far simpler. Filepack uses a single hash +function, BLAKE3, and a single public-key cryptography implementation, Ed25519. +Additionally, Filepack can be used for hashing, signing, and verification +without depending on the network. + +### Iroh + +Iroh is a peer-to-peer networking stack that uses BLAKE3 for content +addressing. + +Filepack and Iroh are largely orthogonal. Iroh is used for transport of flat +blobs, and does not define a directory encoding. From a2271bb0a33eb46ad9e0223208be5e8d57e84d51 Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 21:21:45 -0700 Subject: [PATCH 6/8] Amend --- book/src/comparisons.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/book/src/comparisons.md b/book/src/comparisons.md index 8a230fc3..18068a70 100644 --- a/book/src/comparisons.md +++ b/book/src/comparisons.md @@ -58,18 +58,18 @@ Filepack packages are similar to [BitTorrent](https://en.wikipedia.org/wiki/BitTorrent) torrents. [Version 1 torrents](https://www.bittorrent.org/beps/bep_0003.html) use the -[SHA-1](https://en.wikipedia.org/wiki/SHA-1) hash function, now known to be -cryptographically broken. Files do not have consistent hashes between torrents. +[SHA-1](https://en.wikipedia.org/wiki/SHA-1) hash function, which is vulnerable +to collisions. Files do not have consistent hashes between torrents. [Version 2 torrents](https://www.bittorrent.org/beps/bep_0052.html) use the -[SHA-2](https://en.wikipedia.org/wiki/SHA-2) hash function with a Merkle tree +[SHA-256](https://en.wikipedia.org/wiki/SHA-2) hash function with a Merkle tree construction that does give files consistent hashes between torrents. Version 1 torrents are currently far more widely adopted than version 2 torrents. -Torrents are not naively composable. A torrent may contain `.torrrent` files, -but may not naively contain other torrents. +Torrents are not composable. A torrent may contain `.torrent` files, but may +not natively contain other torrents. Compared to BitTorrent, Filepack uses BLAKE3, a fast, modern cryptographic hash function with built-in Merkle tree construction. Files are consistently hashed @@ -91,10 +91,10 @@ hashing or manifests. ### IPFS -Filepack and IPFS use a similar data model, where directories files and files -are both identified by hash. +Filepack and IPFS use a similar data model, where directories and files are +both identified by hash. -IPFS is notoriously complex, and supports a wide variety of hash functions, +IPFS is notoriously complex, supports a wide variety of hash functions, public-key cryptography algorithms, and peer to peer transport and addressing schemes. From d642727378f0b113944d38263eedf60192f63c77 Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 21:24:02 -0700 Subject: [PATCH 7/8] Adjust --- book/src/comparisons.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/book/src/comparisons.md b/book/src/comparisons.md index 18068a70..4098175c 100644 --- a/book/src/comparisons.md +++ b/book/src/comparisons.md @@ -50,7 +50,9 @@ Content-addressed Distribution ------------------------------ `filepack` packages are Merkle trees over their contents, which allows them to -be used for content-addressed storage and distribution. +be used for content-addressed storage and distribution. Compared to BitTorrent +and IPFS, Filepack does not yet have a peer-to-peer distribution network, +although the data model is compatible with one. ### BitTorrent From a74d9f04f129e112156f1e7e9c2b7f74de2d7a18 Mon Sep 17 00:00:00 2001 From: Casey Rodarmor Date: Mon, 26 Oct 2026 21:27:49 -0700 Subject: [PATCH 8/8] Revise --- book/src/comparisons.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/book/src/comparisons.md b/book/src/comparisons.md index 4098175c..8ca15d85 100644 --- a/book/src/comparisons.md +++ b/book/src/comparisons.md @@ -111,4 +111,4 @@ Iroh is a peer-to-peer networking stack that uses BLAKE3 for content addressing. Filepack and Iroh are largely orthogonal. Iroh is used for transport of flat -blobs, and does not define a directory encoding. +blobs and does not define a directory encoding.