Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ jobs:
if: ${{ matrix.node != '24' }}
run: npm run test:unit
# Node 24 runs the same tests with coverage; the run fails if line coverage of src/
# drops below the threshold in package.json (measured at 86.8% when it was added)
# drops below the threshold in package.json
- name: Run tests with coverage
if: ${{ matrix.node == '24' }}
run: npm run test:coverage
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ Test layers:
- MIME fidelity and golden BODYSTRUCTURE/ENVELOPE wire forms (checked against Dovecot) in `test/mime-fidelity.test.ts` with fixtures in `test/fixtures/mime/`
- parser level tests of the MIME code in `test/mime.test.ts`

Every transcript from `ctx.run` and `openSession` goes through `test/helpers/validate-responses.ts` before the test sees it: CRLF framing and literals, the RFC 3501 section 9 shape of tagged, untagged and `+` responses (status text is required, so every OK/NO/BAD/BYE response, including untagged ones with only a response code, must carry human readable text; no 8-bit outside literals, nz-numbers for FETCH/EXPUNGE, FETCH lists in pairs), and ImapFlow's response parser. A failure there means ImapKit sent something a compliant client can not parse, so fix the server rather than the check. `test/fuzz.test.ts` replays mutated commands under the same guardrail; on failure it prints `FUZZ_SEED`, the iteration and the input, and `FUZZ_SEED=<n> FUZZ_ITERATIONS=<n> node --test test/fuzz.js` reproduces or widens a run.
Every transcript from `ctx.run` and `openSession` goes through `test/helpers/validate-responses.ts` before the test sees it: CRLF framing and literals, the RFC 3501 section 9 shape of tagged, untagged and `+` responses (status text is required, so every OK/NO/BAD/BYE response, including untagged ones with only a response code, must carry human readable text; no 8-bit outside literals, nz-numbers for FETCH/EXPUNGE, FETCH lists in pairs), and ImapFlow's response parser. A failure there means ImapKit sent something a compliant client can not parse, so fix the server rather than the check. `test/fuzz.test.ts` replays mutated commands under the same guardrail; on failure it prints `FUZZ_SEED`, the iteration and the input, and `FUZZ_SEED=<n> FUZZ_ITERATIONS=<n> node --import tsx --test test/fuzz.test.ts` reproduces or widens a run.

## Comparing with Dovecot

Expand Down
19 changes: 10 additions & 9 deletions README.md

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions bin/help.txt
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Configuration file takes the following structure
}
},
"id":{
"name": "hoodecrow",
"name": "imapkit",
"version": "__VERSION__",
"key": "value"
}
Expand All @@ -82,7 +82,8 @@ Scripted faults
---------------

A script file holds a list of rules. Each rule watches one event
("greeting", "command", "input", "response" or "continuation") and
("greeting", "command", "input", "response", "continuation" or
"quiet", no input and output for "quietFor" milliseconds) and
changes what the server does when its matchers fit. For example, this
script answers the first FETCH with a literal that is too short and
then drops the connection:
Expand Down
20 changes: 15 additions & 5 deletions bin/imapkit.js
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,9 @@ if (debug) {
config.debug = true;
}

const port = argv.port || process.env.IMAPKIT_PORT || config.port || (secure ? 993 : 143);
// secureConnection can also come from the --config file
const isSecure = secure || config.secureConnection === true;
const port = argv.port || process.env.IMAPKIT_PORT || config.port || (isSecure ? 993 : 143);

if (argv.help) {
const help = fs.readFileSync(path.join(__dirname, 'help.txt'), 'utf-8');
Expand Down Expand Up @@ -120,13 +122,21 @@ if (argv.help) {
// smtp-server is an optional dependency, start() loads it only when SMTP is enabled
config.smtp = { port: Number(smtpPort) };
}
const server = imapkit(config);
let server;
try {
server = imapkit(config);
} catch (err) {
// an unknown plugin or quirk name, invalid storage ...
console.error('Failed to start ImapKit: %s', err.message);
process.exit(1);
}
console.log('Starting ImapKit ...');
server.start(Number(port)).then(
() => {
console.log('ImapKit successfully%s listening on port %s', secure ? ' and securely' : '', port);
listening => {
// the actual port, also for -p 0
console.log('ImapKit successfully%s listening on port %s', isSecure ? ' and securely' : '', listening);
if (server.restServer) {
console.log('REST API listening on %s:%s', restHost || '127.0.0.1', server.restServer.address().port);
console.log('REST API listening on %s:%s', (config.rest && config.rest.host) || '127.0.0.1', server.restServer.address().port);
}
if (server.smtpServer) {
console.log('Incoming SMTP server up and running on port %s', server.smtpServer.server.address().port);
Expand Down
2 changes: 1 addition & 1 deletion compare/compare.ts
Original file line number Diff line number Diff line change
Expand Up @@ -722,7 +722,7 @@ Replays IMAP commands against imapkit and Dovecot (start it with
-v, --verbose also show greetings, login and seeding notes
--json print machine readable results

Scenario syntax: see the comment on parseScenario() in compare/compare.js.`);
Scenario syntax: see the comment on parseScenario() in compare/compare.ts.`);
return;
}

Expand Down
126 changes: 126 additions & 0 deletions docs/docs/contributing/comparing-with-dovecot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
title: Comparing with Dovecot
sidebar_position: 2
description: Replay the same IMAP commands against ImapKit and a real Dovecot server and see where the responses differ.
---

# Comparing with Dovecot

`compare/` holds a development aid, not a test suite. It replays the same IMAP commands against ImapKit and a real Dovecot 2.4 server and shows where the responses differ. Use it when building or fixing a feature, to see what RFC compliant input and output look like in practice.

Dovecot is the most spec compliant server around, but it has its own bugs, quirks and extensions. Treat a difference as a hint to check the RFC (from `https://www.rfc-editor.org/rfc/rfcNNNN.txt`), not as proof that ImapKit is wrong, and do not copy Dovecot behavior that the RFC does not require. ImapKit is [strict by design](../guides/strict-by-design.md), so it refuses input Dovecot accepts in many places.

## Starting Dovecot

The tool needs Docker. Dovecot runs as a long-running container named `imapkit-dovecot` (image `dovecot/dovecot:2.4.4`) with plain IMAP on `127.0.0.1:32143`:

```bash
npm run dovecot:start
npm run dovecot:stop
bash compare/dovecot.sh status # also: restart, logs
```

`compare/dovecot.conf` is mounted as a drop-in. It allows cleartext login, turns off FTS so SEARCH is plain substring matching, stops auto-creating special-use mailboxes, and turns on QUOTA (count driver, 10M and 1000 messages), APPENDLIMIT (5M), ACL (vfile) and METADATA for comparison.

| Variable | What it changes |
| -------------------------- | -------------------------------------------------------------------------------------- |
| `IMAPKIT_DOVECOT_IMAGE` | the image to run |
| `IMAPKIT_DOVECOT_PLATFORM` | the platform, e.g. `linux/amd64`. Forcing `linux/amd64` on Apple Silicon does not work |
| `IMAPKIT_DOVECOT_PORT` | the host port, read by both `dovecot.sh` and the compare tool (default `32143`) |
| `IMAPKIT_DOVECOT_HOST` | the host the compare tool connects to (default `127.0.0.1`) |

## Running a comparison

```bash
npm run compare -- compare/scenarios/fetch.txt
node --import tsx compare/compare.ts -c 'SELECT INBOX' -c 'FETCH 1 BODYSTRUCTURE'
node --import tsx compare/compare.ts --plugin IDLE,MOVE compare/scenarios/basic.txt
```

Every run starts an in-process ImapKit on a random port, and logs into Dovecot as a brand-new user (static passdb, password `pass`), so runs never see each other's mail. The new user is seeded from the same storage JSON through a hidden setup connection: CREATE, SUBSCRIBE, and APPEND with flags and internal date. Only personal namespaces with `/` as separator are seeded and `\Recent` is dropped. A warning is shown when Dovecot assigns other UIDs than the storage specifies.

For each step the output shows `= same` (with the ImapKit output dimmed), or both outputs one after the other. The last line counts the steps that differ (or says that all steps match). The exit code is 0 even when steps differ, it is 1 only when the run fails (for example when Dovecot is not reachable).

### Options

| Option | What it does |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `-c`, `--command <cmd>` | A command to run, repeatable, instead of a scenario file. |
| `--storage <file>` | The ImapKit storage JSON, also seeded into Dovecot. Default `compare/storage.json`. |
| `--plugin <names>` | ImapKit plugins, comma separated, repeatable. |
| `--target <name>` | `both` (default), `imapkit` or `dovecot`. `--target imapkit` needs no Docker. |
| `--manual-login` | Do not log in automatically, the scenario logs in with `$USER` and `$PASS`. |
| `--keep-text` | Also compare the human readable text of OK, NO, BAD and BYE responses and `+` continuations. |
| `--exact` | Do not sort LIST responses and flag lists, and do not unquote mailbox names, before comparing. |
| `--timeout <ms>` | How long to wait for a tagged response. Default 3000. |
| `--settle <ms>` | How long to wait after each step for extra output. Default 100. |
| `-v`, `--verbose` | Also show greetings, login and seeding notes. |
| `--json` | Machine readable results: the warnings per target, and for every step the line, session, what was sent, whether both sides matched (`same`) and the responses of each target. |

`node --import tsx compare/compare.ts --help` prints the same list.

## Scenario files

Scenario files in `compare/scenarios/*.txt` hold one step per line:

| Line | What it does |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `SELECT INBOX` | A command, sent with an automatic tag (`A1`, `A2`, ...). |
| `2: SELECT INBOX` | A command on session 2. Sessions open and log in on first use, the default is session 1. |
| `> DONE` | Sent verbatim with no tag (also `2:> DONE`). Waits for the tagged response of the command a continuation left open (IDLE, AUTHENTICATE). |
| `!wait 500` | Pauses, then collects whatever the sessions received. |
| `# comment` | Ignored, as are blank lines. |

Inside commands and verbatim lines:

- the four characters `\r\n` become CRLF
- `{file:path}` and `{file+:path}` become a synchronizing or non-synchronizing literal with the contents of the file, line endings converted to CRLF. The path is relative to the scenario file, sample messages are in `compare/messages/`
- `~{file:path}` and `~{file+:path}` become a literal8 ([RFC 3516](https://www.rfc-editor.org/rfc/rfc3516)) with the file's octets as they are, binary samples are in `compare/messages/literal8/`
- `$USER` and `$PASS` expand to the credentials of each server, for `--manual-login`
- `$UIDVALIDITY` expands to the last UIDVALIDITY the server sent, for `SELECT INBOX (QRESYNC ($UIDVALIDITY 1))`

Synchronizing literals wait for the `+` continuation. After a tagged OK to `COMPRESS DEFLATE` a session compresses, and its output is compared decompressed (`compare/scenarios/compress.txt`).

A scenario saved in `compare/scenarios/`, run with `--plugin IDLE`:

```text title="compare/scenarios/my-idle.txt"
# a second session adds a message while the first one idles
SELECT INBOX
IDLE
2: APPEND INBOX {file:../messages/simple.eml}
!wait 1000
> DONE
```

Some scenarios need their own storage or plugins, the comment at the top of the file says how to run them. For example `compare/scenarios/sort-thread.txt` runs against `compare/storage-sort-thread.json`:

```bash
npm run compare -- --storage compare/storage-sort-thread.json \
--plugin SORT,SORT=DISPLAY,THREAD=REFERENCES,THREAD=ORDEREDSUBJECT \
compare/scenarios/sort-thread.txt
```

## Normalization

Both outputs are normalized before they are compared, so only meaningful differences show:

- Dovecot's `(0.001 + 0.000 secs)` timings are removed
- the human readable text of OK, NO, BAD and BYE responses and of `+` continuations is dropped, unless `--keep-text` is set
- UIDVALIDITY values are masked, also inside COPYUID and APPENDUID
- LIST and LSUB responses and flag lists are sorted, and quoted mailbox names that are valid atoms are unquoted, unless `--exact` is set

## Known differences

Many differences are expected and are not ImapKit bugs. The main groups:

- **Capabilities and extensions.** Dovecot advertises many more extensions, and some (IMAP4rev2, CONDSTORE) change its output once enabled. Dovecot does not support OBJECTID, PARTIAL or MULTISEARCH. Dovecot advertises COMPRESS=DEFLATE only after login.
- **Values that differ by design.** Lowercase BODYSTRUCTURE strings, INTERNALDATE in UTC, its own mod-sequences, UIDs when seeding, `\Recent` for the first session that selects a seeded mailbox, and quota limits from its own config.
- **Timing.** IDLE and NOTIFY notifications can arrive late in Dovecot, so put a `!wait 1000` after the step that triggers them.
- **Places where Dovecot is lenient.** Dovecot accepts input that ImapKit refuses with `BAD` or `NO` because the RFC grammar or a MUST forbids it: for example relative CATENATE URLs, `REVERSE REVERSE` in SORT, items RFC 9051 removed after `ENABLE IMAP4rev2`, and 8-bit headers in APPEND before `ENABLE UTF8=ACCEPT`.
- **Places where Dovecot differs from an RFC.** For example `* OK [CLOSED]` on every mailbox switch, SEARCH answered without ESEARCH after `ENABLE IMAP4rev2`, missing UID in unsolicited NOTIFY flag updates, and `NO` instead of `BAD` for a second COMPRESS.

The full list, with the RFC sections behind each difference, is in the "Comparing with Dovecot" section of [CLAUDE.md](https://github.com/postalsys/imapkit/blob/master/CLAUDE.md). Add to it when you find a new difference that is not an ImapKit bug.

## Tests of the tool

`test/compare.test.ts` covers the tool's scenario parsing, normalizing and seeding against ImapKit only, so `npm test` stays Docker-free.
Loading