Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
* [Import into an existing dataset](data-organization/import/import/existing-dataset.md)
* [Import using Team Files](data-organization/import/import/Import-Team-Files.md)
* [Import from Cloud](data-organization/import/import/Import-from-Cloud.md)
* [Import using CLI](data-organization/import/import/import-cli.md)
* [Import using API & SDK](data-organization/import/import/import-sdk-api.md)
* [Import using agent](data-organization/import/import/import-using-agent/import-using-agent.md)
* [Migrations](data-organization/import/migration/migrations.md)
Expand Down
121 changes: 121 additions & 0 deletions data-organization/import/import/import-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Import using CLI

Use `supervisely import` when your data is stored on a local machine and you want to start Auto Import from a terminal.

The command runs the Auto Import Docker image locally, mounts your source data into the container as read-only, detects the annotation format, and uploads data to an existing Supervisely project.

{% hint style="info" %}
This method is useful for local datasets that are too large or inconvenient to upload through the browser. Docker has to be installed and available in your terminal.
{% endhint %}

## Prerequisites

Install or update the Supervisely Python package:

```bash
pip3 install --upgrade supervisely
```

Create `~/supervisely.env` with your Supervisely server address and API token:

```text
SERVER_ADDRESS=<server-address>
API_TOKEN=<api-token>
```

You also need an existing destination project. Copy its ID from the project page in Supervisely.

## Import a local directory

```bash
supervisely import <local-source> --project-id <project-id>
```

In the following **required** arguments, replace:

* `<local-source>` with the local directory or file you want to import.
* `<project-id>` with the ID of the destination Supervisely project. Prefix: `--project-id`

In the following **optional** arguments, replace:

* `<dataset-id>` with the ID of an existing dataset in the destination project. Prefix: `--dataset-id`
* `<dataset-name>` with the name of the dataset that will be created if `--dataset-id` is not provided. Prefix: `--dataset-name`
* `<docker-image>` with a custom Auto Import CLI Docker image. Prefix: `--image`
* `<env-file>` with a custom path to the Supervisely credentials file. By default, `~/supervisely.env` is used. Prefix: `--env-file`
* Add the `--import-as-links` flag to import supported link-based datasets without uploading binary files.
* Add the `--dry-run` flag to print the Docker command without running the import.

For example:

```bash
supervisely import ./dataset --project-id 6911 --dataset-name "my dataset"
```

The source path is mounted into the Docker container as read-only. If you pass a directory, it is mounted as `/input`. If you pass a file, its parent directory is mounted and the file is passed to Auto Import inside `/input`.

{% hint style="info" %}
`--import-as-links` is intended for link-based formats, for example CSV, TXT, or TSV files with URLs. It is not a replacement for importing arbitrary local image files without uploading them.
{% endhint %}

## Advanced mode

For most imports, the short command above is enough. Use advanced options when you need to inspect the generated Docker command, use a custom Auto Import image, provide another credentials file, or control where temporary files are stored.

### Check the Docker command

Add `--dry-run` to print the `docker run` command without starting the import:

```bash
supervisely import ./dataset --project-id 6911 --dry-run
```

This is useful before running a large import, because you can check which local path is mounted and which environment variables are passed to Docker.

### Use a custom image or env file

```bash
supervisely import ./dataset \
--project-id 6911 \
--dataset-name "my dataset" \
--env-file ~/supervisely.env \
--image supervisely/main-import-cli:latest
```

Use `--image` if you want to run a specific Auto Import CLI image tag. By default, the command uses the latest published CLI image.

### Use a custom Docker work directory

Auto Import prepares data before uploading it. For example, it may unpack archives, remove temporary junk files, convert some files, or create intermediate files. By default, these files are stored in a temporary directory inside the Docker container.

For large archives or project structures with many files, Docker may run out of temporary disk space. In this case, run Docker manually and mount a work directory:

```bash
mkdir -p .sly-import-work

docker run --rm \
--env-file ~/supervisely.env \
-e PROJECT_ID=6911 \
-e DATASET_NAME="my dataset" \
-e SLY_APP_DATA_DIR=/work \
-v "$PWD/dataset:/input:ro" \
-v "$PWD/.sly-import-work:/work" \
supervisely/main-import-cli:latest \
--input /input \
--work-dir /work
```

Keep the source dataset mounted as read-only (`/input:ro`). All writable temporary data should go to the work directory.

For a single archive or file, mount the parent directory and pass the file path inside `/input`:

```bash
docker run --rm \
--env-file ~/supervisely.env \
-e PROJECT_ID=6911 \
-e SLY_APP_DATA_DIR=/work \
-v "$PWD:/input:ro" \
-v "$PWD/.sly-import-work:/work" \
supervisely/main-import-cli:latest \
--input /input/dataset.zip \
--work-dir /work
```
4 changes: 2 additions & 2 deletions data-organization/import/import/import.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ We don't want you to convert anything yourself, so, to deal with that, here at S

Using Supervisely Apps or API, you can turn your images, videos and annotations into Supervisely projects and datasets: they will be stored in the [Supervisely Format](https://github.com/supervisely/docs/blob/master/data-organization/supervisely-format.md) and at any time you can [download](../export/export.md) them in this or another format.

Supervisely has three ways how to store your assets:
Supervisely has three ways to store your assets:

**Store files locally**

Expand All @@ -26,4 +26,4 @@ The hybrid approach that takes the best of both worlds. In this scenario, you do
Supervisely calculate file hashes when you upload your assets: because of that, we do not store duplicates.
{% endhint %}

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Import using Web UI</strong></td><td>The most simple and straightforward method of importing is uploading your data using one of our Supervisely Apps.</td><td><a href="Import-using-Web-UI.md">Import-using-Web-UI.md</a></td></tr><tr><td><strong>Import sample dataset</strong></td><td>Save valuable time by starting with already prepared datasets. We provide access to a variety of ready-made data to speed up your start.</td><td><a href="Import-sample-dataset.md">Import-sample-dataset.md</a></td></tr><tr><td><strong>Import into an existing dataset</strong></td><td>It is possible to add more assets such as images to the existing project or dataset.</td><td><a href="existing-dataset.md">existing-dataset.md</a></td></tr><tr><td><strong>Import using Team Files</strong></td><td>you can just select the appropriate Supervisely App from the context menu of a folder in your Team Files - and enjoy.</td><td><a href="Import-Team-Files.md">Import-Team-Files.md</a></td></tr><tr><td><strong>Import from Cloud</strong></td><td>Want to contribute to Supervisely? Start with our GitHub page here.</td><td><a href="Import-from-Cloud.md">Import-from-Cloud.md</a></td></tr><tr><td><strong>Import using API &#x26; SDK</strong></td><td>Save valuable time by starting with already prepared datasets.</td><td><a href="import-sdk-api.md">import-sdk-api.md</a></td></tr></tbody></table>
<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Import using Web UI</strong></td><td>The most simple and straightforward method of importing is uploading your data using one of our Supervisely Apps.</td><td><a href="Import-using-Web-UI.md">Import-using-Web-UI.md</a></td></tr><tr><td><strong>Import sample dataset</strong></td><td>Save valuable time by starting with already prepared datasets. We provide access to a variety of ready-made data to speed up your start.</td><td><a href="Import-sample-dataset.md">Import-sample-dataset.md</a></td></tr><tr><td><strong>Import into an existing dataset</strong></td><td>It is possible to add more assets such as images to the existing project or dataset.</td><td><a href="existing-dataset.md">existing-dataset.md</a></td></tr><tr><td><strong>Import using Team Files</strong></td><td>You can select the appropriate Supervisely App from the context menu of a folder in your Team Files.</td><td><a href="Import-Team-Files.md">Import-Team-Files.md</a></td></tr><tr><td><strong>Import from Cloud</strong></td><td>Import data from cloud providers and optionally keep files by link without copying them to Supervisely storage.</td><td><a href="Import-from-Cloud.md">Import-from-Cloud.md</a></td></tr><tr><td><strong>Import using CLI</strong></td><td>Run Auto Import from a terminal for local datasets using the Supervisely CLI and Docker.</td><td><a href="import-cli.md">import-cli.md</a></td></tr><tr><td><strong>Import using API &#x26; SDK</strong></td><td>Create custom import workflows with Supervisely SDK and API.</td><td><a href="import-sdk-api.md">import-sdk-api.md</a></td></tr></tbody></table>