From 6a41ea4ebc7a15f0cb07a537ab379b2df21e1e51 Mon Sep 17 00:00:00 2001 From: vorozhkog Date: Wed, 15 Jul 2026 20:03:12 +0100 Subject: [PATCH] Add import CLI doc --- SUMMARY.md | 1 + data-organization/import/import/import-cli.md | 121 ++++++++++++++++++ data-organization/import/import/import.md | 4 +- 3 files changed, 124 insertions(+), 2 deletions(-) create mode 100644 data-organization/import/import/import-cli.md diff --git a/SUMMARY.md b/SUMMARY.md index 3aaed73b..2f085d1b 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -64,6 +64,7 @@ * [Import into an existing dataset](data-organization/import/import/existing-dataset.md) * [Import using Team Files](data-organization/import/import/Import-Team-Files.md) * [Import from Cloud](data-organization/import/import/Import-from-Cloud.md) + * [Import using CLI](data-organization/import/import/import-cli.md) * [Import using API & SDK](data-organization/import/import/import-sdk-api.md) * [Import using agent](data-organization/import/import/import-using-agent/import-using-agent.md) * [Migrations](data-organization/import/migration/migrations.md) diff --git a/data-organization/import/import/import-cli.md b/data-organization/import/import/import-cli.md new file mode 100644 index 00000000..17ed288c --- /dev/null +++ b/data-organization/import/import/import-cli.md @@ -0,0 +1,121 @@ +# Import using CLI + +Use `supervisely import` when your data is stored on a local machine and you want to start Auto Import from a terminal. + +The command runs the Auto Import Docker image locally, mounts your source data into the container as read-only, detects the annotation format, and uploads data to an existing Supervisely project. + +{% hint style="info" %} +This method is useful for local datasets that are too large or inconvenient to upload through the browser. Docker has to be installed and available in your terminal. +{% endhint %} + +## Prerequisites + +Install or update the Supervisely Python package: + +```bash +pip3 install --upgrade supervisely +``` + +Create `~/supervisely.env` with your Supervisely server address and API token: + +```text +SERVER_ADDRESS= +API_TOKEN= +``` + +You also need an existing destination project. Copy its ID from the project page in Supervisely. + +## Import a local directory + +```bash +supervisely import --project-id +``` + +In the following **required** arguments, replace: + +* `` with the local directory or file you want to import. +* `` with the ID of the destination Supervisely project. Prefix: `--project-id` + +In the following **optional** arguments, replace: + +* `` with the ID of an existing dataset in the destination project. Prefix: `--dataset-id` +* `` with the name of the dataset that will be created if `--dataset-id` is not provided. Prefix: `--dataset-name` +* `` with a custom Auto Import CLI Docker image. Prefix: `--image` +* `` with a custom path to the Supervisely credentials file. By default, `~/supervisely.env` is used. Prefix: `--env-file` +* Add the `--import-as-links` flag to import supported link-based datasets without uploading binary files. +* Add the `--dry-run` flag to print the Docker command without running the import. + +For example: + +```bash +supervisely import ./dataset --project-id 6911 --dataset-name "my dataset" +``` + +The source path is mounted into the Docker container as read-only. If you pass a directory, it is mounted as `/input`. If you pass a file, its parent directory is mounted and the file is passed to Auto Import inside `/input`. + +{% hint style="info" %} +`--import-as-links` is intended for link-based formats, for example CSV, TXT, or TSV files with URLs. It is not a replacement for importing arbitrary local image files without uploading them. +{% endhint %} + +## Advanced mode + +For most imports, the short command above is enough. Use advanced options when you need to inspect the generated Docker command, use a custom Auto Import image, provide another credentials file, or control where temporary files are stored. + +### Check the Docker command + +Add `--dry-run` to print the `docker run` command without starting the import: + +```bash +supervisely import ./dataset --project-id 6911 --dry-run +``` + +This is useful before running a large import, because you can check which local path is mounted and which environment variables are passed to Docker. + +### Use a custom image or env file + +```bash +supervisely import ./dataset \ + --project-id 6911 \ + --dataset-name "my dataset" \ + --env-file ~/supervisely.env \ + --image supervisely/main-import-cli:latest +``` + +Use `--image` if you want to run a specific Auto Import CLI image tag. By default, the command uses the latest published CLI image. + +### Use a custom Docker work directory + +Auto Import prepares data before uploading it. For example, it may unpack archives, remove temporary junk files, convert some files, or create intermediate files. By default, these files are stored in a temporary directory inside the Docker container. + +For large archives or project structures with many files, Docker may run out of temporary disk space. In this case, run Docker manually and mount a work directory: + +```bash +mkdir -p .sly-import-work + +docker run --rm \ + --env-file ~/supervisely.env \ + -e PROJECT_ID=6911 \ + -e DATASET_NAME="my dataset" \ + -e SLY_APP_DATA_DIR=/work \ + -v "$PWD/dataset:/input:ro" \ + -v "$PWD/.sly-import-work:/work" \ + supervisely/main-import-cli:latest \ + --input /input \ + --work-dir /work +``` + +Keep the source dataset mounted as read-only (`/input:ro`). All writable temporary data should go to the work directory. + +For a single archive or file, mount the parent directory and pass the file path inside `/input`: + +```bash +docker run --rm \ + --env-file ~/supervisely.env \ + -e PROJECT_ID=6911 \ + -e SLY_APP_DATA_DIR=/work \ + -v "$PWD:/input:ro" \ + -v "$PWD/.sly-import-work:/work" \ + supervisely/main-import-cli:latest \ + --input /input/dataset.zip \ + --work-dir /work +``` diff --git a/data-organization/import/import/import.md b/data-organization/import/import/import.md index 349398ec..e9bcff91 100644 --- a/data-organization/import/import/import.md +++ b/data-organization/import/import/import.md @@ -8,7 +8,7 @@ We don't want you to convert anything yourself, so, to deal with that, here at S Using Supervisely Apps or API, you can turn your images, videos and annotations into Supervisely projects and datasets: they will be stored in the [Supervisely Format](https://github.com/supervisely/docs/blob/master/data-organization/supervisely-format.md) and at any time you can [download](../export/export.md) them in this or another format. -Supervisely has three ways how to store your assets: +Supervisely has three ways to store your assets: **Store files locally** @@ -26,4 +26,4 @@ The hybrid approach that takes the best of both worlds. In this scenario, you do Supervisely calculate file hashes when you upload your assets: because of that, we do not store duplicates. {% endhint %} -
Import using Web UIThe most simple and straightforward method of importing is uploading your data using one of our Supervisely Apps.Import-using-Web-UI.md
Import sample datasetSave valuable time by starting with already prepared datasets. We provide access to a variety of ready-made data to speed up your start.Import-sample-dataset.md
Import into an existing datasetIt is possible to add more assets such as images to the existing project or dataset.existing-dataset.md
Import using Team Filesyou can just select the appropriate Supervisely App from the context menu of a folder in your Team Files - and enjoy.Import-Team-Files.md
Import from CloudWant to contribute to Supervisely? Start with our GitHub page here.Import-from-Cloud.md
Import using API & SDKSave valuable time by starting with already prepared datasets.import-sdk-api.md
+
Import using Web UIThe most simple and straightforward method of importing is uploading your data using one of our Supervisely Apps.Import-using-Web-UI.md
Import sample datasetSave valuable time by starting with already prepared datasets. We provide access to a variety of ready-made data to speed up your start.Import-sample-dataset.md
Import into an existing datasetIt is possible to add more assets such as images to the existing project or dataset.existing-dataset.md
Import using Team FilesYou can select the appropriate Supervisely App from the context menu of a folder in your Team Files.Import-Team-Files.md
Import from CloudImport data from cloud providers and optionally keep files by link without copying them to Supervisely storage.Import-from-Cloud.md
Import using CLIRun Auto Import from a terminal for local datasets using the Supervisely CLI and Docker.import-cli.md
Import using API & SDKCreate custom import workflows with Supervisely SDK and API.import-sdk-api.md