Remote context with ssh:// URL not working

[focdanisch]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

We are facing a problem when trying to use a remote context with a ssh URL. When we clone a github repository to the local machine and use that as build context, we can successfully build the image. When we do the same with the same repository using a remote context URL, the build process returns an error saying it is not able to find a Dockerfile.

As a sidenote: In our internal use case, where we originally faced this problem, we are not using github, but an internal git repository on a non-standard ssh-port. So, we have to use the ssh:// URL format to be able to provide the correct port. That is the reason why we write the github-URL in this example in such a "complicated" way. We just used this public github repository as a way to showcase the problem. But internally, we need to set a different port for ssh access (and there is no https).

Expected behaviour

It should make no difference if we use a local git repository cloned manually or a remote URL as build context.

Actual behaviour

Using a remote URL as context leads to an error saying that the Dockerfile can not be read,

Buildx version

github.com/docker/buildx v0.31.1 a267595

Docker info

Client: Docker Engine - Community
 Version:    29.3.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.31.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.1.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 5
  Running: 5
  Paused: 0
  Stopped: 0
 Images: 16
 Server Version: 29.3.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 301b2dac98f15c27117da5c8af12118a041a31d9
 runc version: v1.3.4-0-gd6d73eb8
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.6.87.2-microsoft-standard-WSL2
 Operating System: Ubuntu 24.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 19.53GiB
 Name: LT-01-MDa3
 ID: 22b2e86d-059f-4c6f-9805-5fba1762191f
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.16.0.0/12, Size: 20
 Firewall Backend: iptables

Builders list

NAME/NODE     DRIVER/ENDPOINT   STATUS    BUILDKIT   PLATFORMS
default*      docker
 \_ default    \_ default       running   v0.28.0    linux/amd64 (+4), linux/386

Configuration

services:
  buildtestlocal:
    image: buildtestlocal:latest
    build:
      context: welcome-to-docker
      dockerfile: Dockerfile
  buildtestremote:
    image: buildtestremote:latest
    build:
      context: ssh://git@github.com:22/docker/welcome-to-docker.git
      dockerfile: Dockerfile
      ssh:
        - default=~/.ssh/id_ed25519 # there is no agent running locally, so I hardcoded my ssh key here

git clone ssh://git@github.com:22/docker/welcome-to-docker.git
docker compose build buildtestlocal
docker compose build buildtestremote

Build logs

# buildtestlocal
[+] Building 1.0s (13/13) FINISHED
 => [internal] load local bake definitions                                                                                                                                                                                                                                         0.0s
 => => reading from stdin 587B                                                                                                                                                                                                                                                     0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                               0.0s
 => => transferring dockerfile: 654B                                                                                                                                                                                                                                               0.0s
 => [internal] load metadata for docker.io/library/node:22-alpine                                                                                                                                                                                                                  0.8s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                  0.0s
 => => transferring context: 52B                                                                                                                                                                                                                                                   0.0s
 => [1/6] FROM docker.io/library/node:22-alpine@sha256:8094c002d08262dba12645a3b4a15cd6cd627d30bc782f53229a2ec13ee22a00                                                                                                                                                            0.0s
 => [internal] load build context                                                                                                                                                                                                                                                  0.0s
 => => transferring context: 393B                                                                                                                                                                                                                                                  0.0s
 => CACHED [2/6] WORKDIR /app                                                                                                                                                                                                                                                      0.0s
 => CACHED [3/6] COPY package*.json ./                                                                                                                                                                                                                                             0.0s
 => CACHED [4/6] COPY ./src ./src                                                                                                                                                                                                                                                  0.0s
 => CACHED [5/6] COPY ./public ./public                                                                                                                                                                                                                                            0.0s
 => CACHED [6/6] RUN npm install     && npm install -g serve@latest     && npm run build     && rm -fr node_modules                                                                                                                                                                0.0s
 => exporting to image                                                                                                                                                                                                                                                             0.0s
 => => exporting layers                                                                                                                                                                                                                                                            0.0s
 => => writing image sha256:cab5d044e2d25a91c77b93ee878a27ba7794d780b855ef2ab33b9839f63453ea                                                                                                                                                                                       0.0s
 => => naming to docker.io/library/buildtestlocal:latest                                                                                                                                                                                                                           0.0s
 => resolving provenance for metadata file                                                                                                                                                                                                                                         0.0s
[+] build 1/1
 ✔ Image buildtestlocal:latest Built

# buildtestremote
[+] Building 3.2s (2/2) FINISHED
 => [internal] load local bake definitions                                                                                                                                                                                                                                         0.0s
 => => reading from stdin 664B                                                                                                                                                                                                                                                     0.0s
 => CACHED [internal] load git source ssh://git@github.com:22/docker/welcome-to-docker.git                                                                                                                                                                                         2.8s
[+] build 0/1
 ⠙ Image buildtestremote:latest Building                                                                                                                                                                                                                                            3.2s
failed to solve: failed to read dockerfile: open ssh:/git@github.com:22/docker/welcome-to-docker.git/Dockerfile: no such file or directory

Additional info

I noticed that there is only one slash in the output printed to console (ssh:/). I do not know if this has anything to do with the problem, though.

[tonistiigi]

If you are seeing issue with invoking docker compose then please open issue there.

[jsternberg]

This section makes me think it's worth at least an investigation to see if it's related to the way we handle SSH urls.

 => CACHED [internal] load git source ssh://git@github.com:22/docker/welcome-to-docker.git                                                                                                                                                                                         2.8s
[+] build 0/1
 ⠙ Image buildtestremote:latest Building                                                                                                                                                                                                                                            3.2s
failed to solve: failed to read dockerfile: open ssh:/git@github.com:22/docker/welcome-to-docker.git/Dockerfile: no such file or directory

Notice how the load git source line is correct but the failed to read dockerfile has open ssh:/git. That looks like the URL somehow is getting treated as a filepath and was sent through path.Clean or something since why would it remove the extra slash?

[focdanisch]

Hello everybody,

We also had the impression that the problem might be located in buildx rather than in compose. If you think that it is more of a compose problem, I gladly open another issue in the compose project.

When I activate the debug mode, the bake build config also looks kind of strange to me:

$ docker -D compose build buildtestremote
DEBU[0000] bake build config:
{
  "group": {
    "default": {
      "targets": [
        "buildtestremote"
      ]
    }
  },
  "target": {
    "buildtestremote": {
      "context": "ssh://git@github.com:22/docker/welcome-to-docker.git",
      "dockerfile": "ssh:/git@github.com:22/docker/welcome-to-docker.git/Dockerfile",
      "labels": {
        "com.docker.compose.project": "buildtest",
        "com.docker.compose.service": "buildtestremote",
        "com.docker.compose.version": "5.1.1"
      },
      "tags": [
        "buildtestremote:latest"
      ],
      "ssh": [
        "default=/home/danisch/.ssh/id_ed25519"
      ],
      "output": [
        "type=docker"
      ]
    }
  }
}

I do not know the code, but I would theorize that this git repository most likely is cloned into a local, temporary directory. So for the "dockerfile" property, should the path being prepended not be the one of the temporary directory?

[focdanisch]

OK, after some more testing, it indeed seems to be a compose problem:

git clone ssh://git@github.com:22/docker/welcome-to-docker.git
docker image build -t buildtestlocal:latest welcome-to-docker/
docker image build -t buildtestremote:latest --ssh default=~/.ssh/id_ed25519 ssh://git@github.com:22/docker/welcome-to-docker.git

Both builds work. I will open an issue in compose with that problem.