Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
133 changes: 12 additions & 121 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,108 +40,12 @@ may change at any time.

Documentation is available at [filepack.com/book/](https://filepack.com/book/).

See [installation](https://filepack.com/book/installation.html) and
[quickstart](https://filepack.com/book/quickstart.html) to get started, and
[Why is metadata important?](https://filepack.com/book/why-metadata.html) for
some of the higher-level ideas behind filepack.

Manifest
--------

`filepack` manifests are named `manifest.filepack` and are placed alongside the
files they reference.

Manifests are encoded with deco, a compact binary serialization format, and may
be converted to JSON for inspection or manipulation with `filepack manifest`.

Manifests, when converted to JSON, are an object with three mandatory keys,
`embedded`, `package`, and `signatures`.

### `embedded`

The value of the mandatory `embedded` key is an object mapping BLAKE3 hashes to
hex-encoded file contents. The `package` object only includes file names,
hashes, and sizes, but files may be embedded in the manifest archive in the
`embedded` map.

This is currently only used to embed `metadata.filemeta`.

### `package`

The value of the mandatory `package` key is an object mapping path components
to directory entries. Directory entries may be subdirectories or files. Files
are objects with two fields: `hash`, the tagged hex BLAKE3 hash of the file,
and `size`, the length of the file in bytes.

Path components are UTF-8 and may not be `.` or `..`, contain the path
separators `/` or `\`, contain control characters, or begin with a Windows
drive prefix, such as `C:`.

### `signatures`

The value of the mandatory `signatures` key is an array of signatures.
Signatures are tagged hex strings that include an Ed25519 key, the package
fingerprint the signature is made over, an optional timestamp, and the
signature itself.

Public keys are Curve25519 points and signatures are Ed25519 signatures made
over the hash of a serialized deco byte string containing the package
fingerprint, which commits to the content of `package`, and an optional
timestamp.

### Example

A manifest converted to JSON over a directory containing the files `README.md`
and `src/main.rs`, with a signature:

```json
{
"embedded": {},
"package": {
"README.md": {
"hash": "hash1fc253b84551ce6b00e820a826ac18054dc7f63a318ce62f3175315f5c467a62a",
"size": 11883
},
"src": {
"main.rs": {
"hash": "hash11fa48b95ed335369d45b91af8138bdccd1413364bcdbfa6e9034e8a2cfd6e17f",
"size": 33
}
}
},
"signatures": ["…"]
}
```

The signature is elided for brevity. Signatures are tagged hex strings
containing a public key and signature.

Metadata
--------

Filepack packages may contain a file named `metadata.filemeta` describing the
package and its contents. If present, `metadata.filemeta` is also embedded in
the `manifest.filepack` archive.

Metadata is authored by creating a file named `metadata.yaml` in the root of a
new package. `filepack create` then loads `metadata.yaml` if present, checks
for validity and unknown fields, and writes the deco serialization to
`metadata.filemeta` in the package root.

`metadata.yaml` is retained as a human-readable reference and for amending
metadata, but `metadata.filemeta` is the authoritative source of metadata. For
consumption by scripts and tools, `filepack metadata` prints the contents of
`metadata.filemeta` as JSON.

Filepack metadata is intended to be a broadly useful machine and human readable
description of the contents of a package, covering personal, distribution, and
archival use-cases.

Metadata follows a fixed schema and is not user-extensible. Future versions of
`filepack` may define new metadata fields, causing verification errors if those
fields are present and invalid according to the new schema.

Please feel free to open an issue with ideas for new metadata fields.
See [Installation](https://filepack.com/book/installation.html) and
[Quickstart](https://filepack.com/book/quickstart.html) to get started,
[Packages](https://filepack.com/book/packages.html) for an overview of Filepack
packages, and [Why is metadata
important?](https://filepack.com/book/why-metadata.html) for some of the
higher-level ideas behind Filepack.

Lints
-----
Expand Down Expand Up @@ -186,7 +90,7 @@ filepack keygen
```

Which creates `master.public` and `master.private` files in the `keychain`
subdirectory of the filepack data directory.
subdirectory of the Filepack data directory.

### Public Key Printing

Expand Down Expand Up @@ -220,19 +124,6 @@ filepack verify --key PUBLIC_KEY
Which will fail if a valid signature for `PUBLIC_KEY` over the manifest
contents is not present.

Fingerprints
------------

Filepack signatures are made over the package fingerprint, which is the hash of
a deco object which commits to the files and directories contained in the
manifest.

Fingerprints are BLAKE3 hashes, constructed such that it is impossible to
produce packages which are different but which have the same fingerprint.

Fingerprints may be used as a globally unique identifier. If two packages have
the same fingerprint, they have the same content.

Guide
-----

Expand All @@ -241,7 +132,7 @@ defaults to the current directory.

### Creating a Package

To create a filepack manifest:
To create a Filepack manifest:

```shell
filepack create [DIRECTORY]
Expand All @@ -267,7 +158,7 @@ Please feel free to open issues requesting new lints.
### Creating a Package with Metadata

To create a package with metadata describing the package, create a file named
`metadata.yaml` in the package directory, for example:
`metadata.yaml` in the package root, for example:

```yaml
title: The Necronomicon
Expand Down Expand Up @@ -299,9 +190,9 @@ filepack keygen
```

This creates a new public key, `master.public`, with corresponding private key,
`master.private`, in the filepack keychain directory.
`master.private`, in the Filepack keychain directory.

The `filepack info` command will print the path to the filepack data directory,
The `filepack info` command will print the path to the Filepack data directory,
keychain directory, and any public keys in the keychain directory.

Your public key can be printed with:
Expand Down Expand Up @@ -384,7 +275,7 @@ keys.

### Detecting Accidental Corruption

Create a filepack manifest with:
Create a Filepack manifest with:

```shell
filepack create <PACKAGE>
Expand Down
2 changes: 1 addition & 1 deletion bin/package
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ cp -r \
metadata.yaml \
dist

echo "Creating filepack manifest..."
echo "Creating Filepack manifest..."
cargo run create --deny distribution dist

echo "Creating release archive..."
Expand Down
2 changes: 1 addition & 1 deletion book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@ Summary
- [Quickstart](quickstart.md)

# Guide
- [Packages](packages.md)
- [Configuration](configuration.md)
- [Authoring Metadata](authoring-metadata.md)
- [Server](server.md)

# Background
- [Why is metadata important?](why-metadata.md)
- [Alternatives and Prior Art](alternatives.md)
- [Design](design.md)

# Reference
- [Commands](commands.md)
Expand Down
6 changes: 3 additions & 3 deletions book/src/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ Configuration
Data Directory
--------------

Filepack stores local data, including private keys, in the filepack data
Filepack stores local data, including private keys, in the Filepack data
directory, located in `~/.filepack`, or `$XDG_DATA_HOME/filepack` if
`$XDG_DATA_HOME` is set and non-empty.

Filepack stores private keys in the `keychain` subdirectory of the filepack
Filepack stores private keys in the `keychain` subdirectory of the Filepack
data directory, by default in `~/.filepack/keychain`.

The location of the filepack data directory used by a command can be overridden
The location of the Filepack data directory used by a command can be overridden
with the `--data-dir` option or the `FILEPACK_DATA_DIR` environment variable.

Logging
Expand Down
51 changes: 0 additions & 51 deletions book/src/design.md

This file was deleted.

9 changes: 5 additions & 4 deletions book/src/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ Introduction

{{#include ../../README.md:introduction}}

See [installation](installation.md) to get started and [Why is metadata
important?](why-metadata.md) for some of the higher-level ideas behind
filepack.
See [Installation](installation.md) and [Quickstart](quickstart.md) to get
started, [Packages](packages.md) for an overview of Filepack packages, and [Why
is metadata important?](why-metadata.md) for some of the higher-level ideas
behind Filepack.

Why Filepack?
-------------
Expand Down Expand Up @@ -41,6 +42,6 @@ files](https://en.wikipedia.org/wiki/Simple_file_verification):
Additionally, errors can be detected and recovered from by transmitting
only those parts of the file which are corrupted.

- Packages may contain machine readable metadata following a filepack-defined
- Packages may contain machine readable metadata following a Filepack-defined
schema, allowing packages to be searched, indexed, and exposed through rich,
featureful interfaces.
Loading
Loading