Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@

All notable changes to this project are documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [Unreleased]

### Changed
- The progress spinner (yaspin) is replaced by a progress bar (tqdm): uploads and downloads show bytes transferred with rate and ETA plus the `done/total` file count; deletes, and downloads whose sizes the listing did not report, count files.

## [2.0.0] 2026-09-20

First release: a native Python3 port of git-ftp 1.6.0. See [COMPATIBILITY.md](COMPATIBILITY.md) for what is unchanged, what was fixed and what is new.
Expand Down
6 changes: 3 additions & 3 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,9 @@ fixed rather than reproduced.
- `--password` as an alias of `--passwd`; `--no-post-hooks`.
- Ctrl-C stops the transfers promptly and exits with 130.
- `version -v` prints the libcurl and paramiko versions in use.
- An interactive terminal shows a progress spinner with a `done/total` count
while files upload, delete or download; it renders on stderr and is silent
when output is not a terminal or under `-n`.
- An interactive terminal shows a progress bar (bytes with rate and ETA, plus a
`done/total` file count) while files upload, delete or download; it renders
on stderr and is silent when output is not a terminal or under `-n`.
- `--worktree` / `git-ftp.worktree`: `init` and `push` read the upload from a
temporary Git worktree checked out at the deployed commit, so edits to the
working tree during the upload cannot leak in. Untracked files added by
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,8 @@ happen first, then deletes, and the commit log is written last, only when every
upload succeeded, so an interrupted deploy never claims a commit it did not
finish. Ctrl-C stops promptly.

In an interactive terminal a spinner shows a `done/total` count with the current
file on stderr. It is off when output is piped, in CI, or under `-n`, so scripts
In an interactive terminal a progress bar on stderr shows the bytes transferred
with rate and ETA, plus a `done/total` file count and the current file. It is off when output is piped, in CI, or under `-n`, so scripts
see the plain lines unchanged.

### Consistent uploads while editing
Expand Down
7 changes: 2 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ dependencies = [
"click>=8.2,<9",
"pycurl>=7.46,<8",
"paramiko>=3.4,<6",
"yaspin>=3.5,<4",
"tqdm>=4.66,<5",
]

[project.urls]
Expand All @@ -59,6 +59,7 @@ dev = [
"mypy>=1.17",
"types-paramiko>=3.4",
"types-pycurl>=7.45",
"types-tqdm>=4.66",
]

[tool.uv]
Expand Down Expand Up @@ -112,10 +113,6 @@ mypy_path = ["src"]
module = ["pyftpdlib", "pyftpdlib.*"]
ignore_missing_imports = true

[[tool.mypy.overrides]]
module = ["yaspin", "yaspin.*"]
ignore_missing_imports = true

[[tool.mypy.overrides]]
module = ["tests.helpers.ftpserver"]
disallow_subclassing_any = false
Expand Down
5 changes: 3 additions & 2 deletions src/gitftp/deploy.py
Original file line number Diff line number Diff line change
Expand Up @@ -298,8 +298,9 @@ def _sync(self, cs: csmod.ChangeSet) -> None:
if uploads and not self.opts.dry_run:
out.info("Uploading ...")
try:
with Progress(out, "Uploading", len(uploads)) as p:
pool.upload(uploads, on_done=p.advance)
size = sum(u.size for u in uploads)
with Progress(out, "Uploading", len(uploads), size=size) as p:
pool.upload(uploads, on_done=p.advance, on_bytes=p.transferred)
except TransferError as e:
raise UploadError(f"Could not upload files. {e}") from e
for path in cs.deletes:
Expand Down
7 changes: 5 additions & 2 deletions src/gitftp/mirror.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,8 +205,11 @@ def apply_plan(
d.mkdir(parents=True, exist_ok=True)
if plan.downloads:
try:
with Progress(out, "Downloading", len(plan.downloads)) as prog:
pool.download(plan.downloads, on_done=prog.advance)
sizes = [d.size for d in plan.downloads]
# A byte bar only when the listing reported every size; else count files.
size = None if None in sizes else sum(n or 0 for n in sizes)
with Progress(out, "Downloading", len(plan.downloads), size=size) as prog:
pool.download(plan.downloads, on_done=prog.advance, on_bytes=prog.transferred)
except TransferError as e:
raise DownloadError(f"Could not download files. {e}") from e
out.info(f"Downloaded {len(plan.downloads)} file(s), deleted {deleted} local file(s).")
Expand Down
94 changes: 72 additions & 22 deletions src/gitftp/progress.py
Original file line number Diff line number Diff line change
@@ -1,55 +1,105 @@
"""An interactive transfer spinner (yaspin), rendered on stderr.
"""An interactive transfer progress bar (tqdm), rendered on stderr.

The spinner shows a running ``<verb> <done>/<total> <current file>`` count while
files transfer. It is active only on an interactive terminal at normal
verbosity, so it never touches the stdout that scripts and the test suite parse,
and it stays silent when output is piped, under ``-n``, or under ``-v``/``-vv``
(which print their own per-file lines).
When the byte size of a batch is known (uploads, and downloads whose listing
reported every size) the bar tracks bytes with rate and ETA, followed by a
``<done>/<total> <current file>`` count; otherwise (deletes) it counts files.
It is active only on an interactive terminal at normal verbosity, so it never
touches the stdout that scripts and the test suite parse, and it stays silent
when output is piped, under ``-n``, or under ``-v``/``-vv`` (which print their
own per-file lines).
"""

from __future__ import annotations

import os
import threading
from types import TracebackType
from typing import Any
from typing import Any, TextIO

from gitftp.output import Level, Output


def _screen_size(stream: TextIO) -> tuple[int, int]:
"""Columns and rows of the terminal, 80x24 when it reports none.

tqdm hides its bar entirely on a terminal that reports 0 rows (bare ptys,
serial consoles), so the size is always passed explicitly.
"""
try:
cols, rows = os.get_terminal_size(stream.fileno())
except (AttributeError, OSError, ValueError):
cols = rows = 0
return cols or 80, rows or 24


class Progress:
"""Context manager wrapping a yaspin spinner; a no-op when not on a TTY."""
"""Context manager wrapping a tqdm bar; a no-op when not on a TTY."""

def __init__(self, out: Output, verb: str, total: int) -> None:
def __init__(self, out: Output, verb: str, total: int, *, size: int | None = None) -> None:
self.out = out
self.verb = verb
self.total = total
self.size = size or None # total bytes; None (or 0) counts files instead
self.done = 0
self._spinner: Any = None
self._bar: Any = None
self._seen: dict[str, int] = {} # bytes reported so far per in-flight file
self._lock = threading.Lock() # transfer threads report bytes concurrently
self.enabled = out.level == Level.NORMAL and out.stderr.isatty()

def _text(self, label: str = "") -> str:
base = f"{self.verb} {self.done}/{self.total}"
def _postfix(self, label: str = "") -> str:
if self.size is None:
return label # the bar itself shows the file count
base = f"{self.done}/{self.total}"
return f"{base} {label}" if label else base

def __enter__(self) -> Progress:
if self.enabled:
from yaspin import yaspin
from tqdm import tqdm

self._spinner = yaspin(text=self._text(), stream=self.out.stderr)
self._spinner.start()
if self.size is None:
kw: dict[str, Any] = {"total": self.total, "unit": "file"}
else:
kw = {"total": self.size, "unit": "B", "unit_scale": True, "unit_divisor": 1024}
ncols, nrows = _screen_size(self.out.stderr)
self._bar = tqdm(
desc=self.verb,
file=self.out.stderr,
leave=False,
ncols=ncols,
nrows=nrows,
postfix=self._postfix() or None,
**kw,
)
return self

def transferred(self, label: str, nbytes: int) -> None:
"""Record ``nbytes`` done so far of one file; safe to call from transfer threads."""
if self._bar is None or self.size is None:
return
with self._lock:
delta = nbytes - self._seen.get(label, 0) # negative when a transfer restarts
self._seen[label] = nbytes
if delta:
self._bar.update(delta)

def advance(self, label: str = "") -> None:
"""Count one finished transfer and refresh the spinner text."""
self.done += 1
if self._spinner is not None:
self._spinner.text = self._text(label)
"""Count one finished transfer and refresh the bar."""
with self._lock:
self.done += 1
self._seen.pop(label, None)
if self._bar is not None:
if self.size is None:
self._bar.set_postfix_str(self._postfix(label), refresh=False)
self._bar.update(1)
else:
self._bar.set_postfix_str(self._postfix(label))

def __exit__(
self,
exc_type: type[BaseException] | None,
exc: BaseException | None,
tb: TracebackType | None,
) -> None:
if self._spinner is not None:
self._spinner.stop() # clears its line and restores the cursor
self._spinner = None
if self._bar is not None:
self._bar.close() # leave=False clears its line
self._bar = None
31 changes: 26 additions & 5 deletions src/gitftp/transfer.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,20 @@
from typing import Generic, TypeVar

from gitftp.output import Output
from gitftp.transport.base import Connector, TransferCancelled, Transport
from gitftp.transport.base import Connector, ProgressFn, TransferCancelled, Transport

T = TypeVar("T")
R = TypeVar("R")

BytesFn = Callable[[str, int], None]
"""Called as ``(label, bytes_done)`` while a file transfers, from the worker thread."""


def _bytes_fn(on_bytes: BytesFn | None, label: str) -> ProgressFn | None:
if on_bytes is None:
return None
return lambda done, _total: on_bytes(label, done)


@dataclass(frozen=True)
class UploadTask:
Expand Down Expand Up @@ -247,10 +256,16 @@ def _map_parallel(

# -- typed helpers -----------------------------------------------------
def upload(
self, tasks: Sequence[UploadTask], *, on_done: Callable[[str], None] | None = None
self,
tasks: Sequence[UploadTask],
*,
on_done: Callable[[str], None] | None = None,
on_bytes: BytesFn | None = None,
) -> None:
def do(t: Transport, task: UploadTask) -> None:
t.put(task.local, task.remote, task.size)
t.put(task.local, task.remote, task.size, progress=_bytes_fn(on_bytes, task.label))
if on_bytes is not None:
on_bytes(task.label, task.size) # transports may not report the last chunk
self.out.debug(f"Uploaded '{task.label}'.")

self.map(do, tasks, fail_fast=True, label=lambda task: task.label, on_done=on_done)
Expand All @@ -268,16 +283,22 @@ def do(t: Transport, task: DeleteTask) -> None:
return [r for r in results if isinstance(r, TransferError)]

def download(
self, tasks: Sequence[DownloadTask], *, on_done: Callable[[str], None] | None = None
self,
tasks: Sequence[DownloadTask],
*,
on_done: Callable[[str], None] | None = None,
on_bytes: BytesFn | None = None,
) -> None:
def do(t: Transport, task: DownloadTask) -> None:
part = task.local.with_name(f".{task.local.name}.git-ftp-part")
task.local.parent.mkdir(parents=True, exist_ok=True)
try:
t.get_file(task.remote, part)
t.get_file(task.remote, part, progress=_bytes_fn(on_bytes, task.label))
if task.mtime is not None:
os.utime(part, (task.mtime, task.mtime))
os.replace(part, task.local)
if on_bytes is not None and task.size is not None:
on_bytes(task.label, task.size)
except BaseException:
with contextlib.suppress(OSError):
part.unlink()
Expand Down
9 changes: 8 additions & 1 deletion tests/fakes/memory_transport.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ def get(self, path: str) -> bytes:
def get_file(self, path: str, local: Path, *, progress: ProgressFn | None = None) -> None:
data = self.get(path)
local.write_bytes(data)
if progress is not None:
progress(len(data), len(data))

def _enter(self) -> None:
with self.store.lock:
Expand All @@ -79,7 +81,12 @@ def _leave(self) -> None:
def put(
self, local: Path, remote: str, size: int, *, progress: ProgressFn | None = None
) -> None:
self.put_bytes(local.read_bytes(), remote)
data = local.read_bytes()
if progress is not None:
progress(len(data) // 2, size)
self.put_bytes(data, remote)
if progress is not None:
progress(len(data), size)

def put_bytes(self, data: bytes, remote: str) -> None:
self._enter()
Expand Down
12 changes: 6 additions & 6 deletions tests/integration/test_entrypoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,16 +94,16 @@ def test_ctrl_c_stops_promptly_and_leaves_log_untouched(


@pytest.mark.skipif(sys.platform == "win32", reason="pty is POSIX-only")
def test_progress_spinner_on_a_tty_leaves_stdout_intact(
def test_progress_bar_on_a_tty_leaves_stdout_intact(
repo: Repo, ftp_server: FtpServer, cli_bin: list[str]
) -> None:
"""With stderr on a real terminal the spinner is active, yet stdout still
carries the exact, parseable messages (the spinner draws only on stderr)."""
"""With stderr on a real terminal the progress bar is active, yet stdout still
carries the exact, parseable messages (the bar draws only on stderr)."""
import pty
import threading

s = ftp_server
master, slave = pty.openpty() # child stderr is a TTY -> spinner enabled
master, slave = pty.openpty() # child stderr is a TTY -> progress bar enabled
drained: list[bytes] = []

def drain() -> None:
Expand All @@ -130,10 +130,10 @@ def drain() -> None:
stderr = b"".join(drained).decode("utf-8", "replace")

assert proc.returncode == 0, stderr
# stdout is unpolluted by the spinner: the upstream lines are exact.
# stdout is unpolluted by the progress bar: the upstream lines are exact.
assert "Uploading ..." in proc.stdout
assert f"Last deployment changed from to {repo.head()}." in proc.stdout
assert "Uploading 0/" not in proc.stdout # progress text never reaches stdout
assert s.remote().log() == repo.head()
# The spinner rendered its running count on the terminal (stderr).
# The bar rendered its progress on the terminal (stderr).
assert "Uploading" in stderr
Loading
Loading