ramify uploads a batch of files to many FTP, FTPS, and SFTP endpoints in parallel, verifies each transfer, retries failures, and exits non-zero if anything failed. Run it from cron, CI, or a folder-watcher: one static binary, one YAML file.
$ ramify ./batch --config stocks.yml
[shutterstock] photo1.jpg: uploaded (verified: size) in 1.21s
[dreamstime] photo1.jpg: uploaded (verified: size) in 0.98s
[shutterstock] photo2.jpg: uploaded (verified: size) in 1.15s
[dreamstime] photo2.jpg: uploaded (verified: size) in 1.04s
[shutterstock] done: 2 succeeded, 0 failed in 3.10s
[dreamstime] done: 2 succeeded, 0 failed in 2.87s- Endpoints live in YAML; add or remove one without touching code.
ftp,ftps(explicit AUTH TLS), andsftp(password or private key).- Size verification after each upload, on by default.
- Retries with a fresh connection, configurable globally and per endpoint.
--dry-runpreflight that proves each target directory is writable.- Quiet, default, verbose, or newline-delimited JSON output, with meaningful exit codes.
${ENV_VAR}interpolation for secrets.
brew install alexeyu/tap/ramifygo install github.com/alexeyu/ramify/cmd/ramify@latestDownload the archive for your OS/architecture from the
latest release,
unpack it, and put ramify on your PATH. Each release ships Linux, macOS,
and Windows builds for amd64 and arm64, listed in checksums.txt.
-
Write a config (
stocks.yml) listing your endpoints. Start fromconfig.yml.example, or see Configuration below. -
Preflight it before you trust it to a cron job:
$ export SHUTTERSTOCK_FTP_PASSWORD=... $ ramify ./batch --config stocks.yml --dry-run [shutterstock] dry-run ok: reachable and writable, would upload 12 files [dreamstime] dry-run ok: reachable and writable, would upload 12 files
-
Upload for real:
$ ramify ./batch photo-extra.jpg --config stocks.yml
Positional arguments may be files or directories, mixed. Directories expand non-recursively: ramify takes every regular file and skips subdirectories and dotfiles. It rejects two inputs that share a basename, since they would land under the same remote name.
Config is a YAML file with a list of endpoints plus optional global policy
defaults. Any endpoint can override any policy key.
endpoints:
- name: shutterstock
protocol: ftps
host: ftp.shutterstock.com
username: myuser
password: ${SHUTTERSTOCK_FTP_PASSWORD}
- name: dreamstime
protocol: sftp
host: sftp.dreamstime.com
username: myuser
private_key: ~/.ssh/id_ed25519
- name: legacy-agency
protocol: ftp
host: ftp.legacy-agency.example
port: 2121
username: myuser
password: hunter2 # literal secrets work too
overwrite: direct
attempts: 5 # overrides the global default below
attempts: 3
retry_delay: 2s
connect_timeout: 30s
stall_timeout: 5m
max_consecutive_connect_failures: 3| Field | Applies to | Notes |
|---|---|---|
name |
all | Required, unique. Shown in all output. |
protocol |
all | ftp, ftps, or sftp. Required. |
host |
all | Required. |
port |
all | Optional; defaults to the protocol's standard port. |
username |
all | Required. |
password |
ftp, ftps, sftp | Required for ftp/ftps. For sftp, an alternative or complement to private_key. |
private_key |
sftp | Path to an SSH private key (~ expanded). Set both this and password and the key wins; password then serves as the passphrase for an encrypted key. |
overwrite |
all | delete-first (default) deletes any existing remote file first; direct uploads straight over it. |
insecure_skip_verify |
ftps | Disables TLS certificate verification. For self-signed or test servers only. |
| Field | Default | Meaning |
|---|---|---|
attempts |
3 |
Total tries per file before ramify counts it as failed. |
retry_delay |
2s |
Fixed wait between attempts (Go duration string). |
connect_timeout |
30s |
Bounds the entire connect + authenticate sequence, TCP dial included. |
stall_timeout |
5m |
Fails a transfer that makes no forward progress for this long. 0 disables. |
max_consecutive_connect_failures |
3 |
After this many connect failures in a row, ramify skips the endpoint's remaining files as unreachable instead of retrying each one. |
${ENV_VAR} interpolation works in any string field when the config loads. An
unset variable fails the load, so you catch a typo before any upload starts.
Literal secrets work but invite leaks. ramify warns when the config file is
readable by group or others; chmod 600 stocks.yml and keep it out of version
control.
ramify <path>... --config <file> [flags]
| Flag | Effect |
|---|---|
--config <file> |
Path to the YAML config. Required. |
--dry-run |
Connect, authenticate, and write and delete a probe file on each endpoint, then report how many files would upload. Leaves your real files alone. |
--no-verify |
Skip post-upload size verification. |
--quiet |
Suppress non-error stdout. |
--verbose |
Print the full event stream, including byte-level progress. |
--json |
Emit newline-delimited JSON instead of text (honors the verbosity level). |
--version |
Print version and exit. |
--help |
Print usage and exit. |
Errors go to stderr in every mode. --quiet and --verbose are mutually
exclusive. Flags and paths may appear in any order. ramify treats everything
after a bare -- as a path, so you can upload a file whose name starts with
-:
$ ramify --config stocks.yml -- -weird-name.jpg--json prints one JSON object per line for whatever the current verbosity
level would show. Each line carries a type discriminator (file_start,
progress, file_success, file_error, endpoint_unreachable,
endpoint_given_up, endpoint_done, dry_run):
$ ramify ./batch --config stocks.yml --json
{"type":"file_success","endpoint":"shutterstock","file":"batch/photo1.jpg","verifyMethod":"size","durationSec":1.21}
{"type":"endpoint_done","endpoint":"shutterstock","succeeded":1,"failed":0,"durationSec":1.34}| Code | Meaning |
|---|---|
0 |
Every file uploaded (and verified) on every endpoint. |
1 |
Partial failure: at least one file failed after exhausting retries on at least one endpoint. |
2 |
Configuration error (bad or invalid YAML, validation failures). |
3 |
Usage error (bad flags, an input path that does not exist). |
- Concurrency. One goroutine per endpoint. Within an endpoint, files upload in sequence over one connection, which amortizes the login and stays inside server connection limits.
- Retries. Each retry reconnects from scratch and re-runs verification, so a size mismatch triggers a retry the same way a failed upload does.
- Verification. ramify compares the remote size (
SIZE/Stat) against the local file to catch truncation on all three protocols. SFTP has no standard hash command, so hash verification waits for a later release. - SSH host keys. ramify checks SFTP hosts against
~/.ssh/known_hostsand refuses unknown ones. Record a key first withsshorssh-keyscan. - Cancellation. On Ctrl-C or SIGTERM, workers stop starting new transfers and retries, let an in-flight transfer finish, and report their results. A second Ctrl-C exits at once with code 130.
The CLI wraps the ramify package. Upload returns a channel of typed events:
package main
import (
"context"
"fmt"
"github.com/alexeyu/ramify"
)
func main() {
endpoints, err := ramify.LoadConfig("stocks.yml")
if err != nil {
panic(err)
}
files := []string{"photo1.jpg", "photo2.jpg"}
events := ramify.Upload(context.Background(), files, endpoints, ramify.Options{})
// Drain the channel fully: workers send unbuffered and deadlock otherwise.
for ev := range events {
switch e := ev.(type) {
case ramify.FileSuccessEvent:
fmt.Printf("%s -> %s ok\n", e.File, e.Endpoint)
case ramify.FileErrorEvent:
fmt.Printf("%s -> %s failed: %s\n", e.File, e.Endpoint, e.Reason)
}
}
}You can also build []ramify.Endpoint yourself instead of loading YAML. See
the package reference for the
full event vocabulary and Options.
Requires Go (see go.mod for the version).
make build # build ./ramify with the version stamped in
make test # unit tests, race detector on
make test-integration # real FTP/SFTP servers in Docker; skips without Docker
make lint # golangci-lintMaintainers: see RELEASING.md.
Pre-1.0. The config schema and the Go library API may change between minor
versions; v1.0.0 will freeze both.