Skip to content

Commit 9806d04

Browse files
Integer-CtrlInteger-Ctrl
committed
fix: coderabbitai
1 parent 9e0776a commit 9806d04

5 files changed

Lines changed: 34 additions & 31 deletions

File tree

‎README.md‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Options:
2323
2424
Commands:
2525
deploy
26-
downoad
26+
download
2727
```
2828

2929
## Docker Image Usage
@@ -43,7 +43,7 @@ Arguments:
4343
content variants of a distribution, fileExt and Compression can be set, if not they are inferred from the path [required]
4444
4545
Options:
46-
--versionid TEXT Target databus version/dataset identifier of the form <h
46+
--version-id TEXT Target databus version/dataset identifier of the form <h
4747
ttps://databus.dbpedia.org/$ACCOUNT/$GROUP/$ARTIFACT/$VE
4848
RSION> [required]
4949
--title TEXT Dataset title [required]
@@ -55,11 +55,11 @@ Options:
5555
```
5656
Examples of using deploy command
5757
```
58-
databusclient deploy --versionid https://databus.dbpedia.org/user1/group1/artifact1/2022-05-18 --title title1 --abstract abstract1 --description description1 --license http://dalicc.net/licenselibrary/AdaptivePublicLicense10 --apikey MYSTERIOUS 'https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml|type=swagger'
58+
databusclient deploy --version-id https://databus.dbpedia.org/user1/group1/artifact1/2022-05-18 --title title1 --abstract abstract1 --description description1 --license http://dalicc.net/licenselibrary/AdaptivePublicLicense10 --apikey MYSTERIOUS 'https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml|type=swagger'
5959
```
6060

6161
```
62-
databusclient deploy --versionid https://dev.databus.dbpedia.org/denis/group1/artifact1/2022-05-18 --title "Client Testing" --abstract "Testing the client...." --description "Testing the client...." --license http://dalicc.net/licenselibrary/AdaptivePublicLicense10 --apikey MYSTERIOUS 'https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml|type=swagger'
62+
databusclient deploy --version-id https://dev.databus.dbpedia.org/denis/group1/artifact1/2022-05-18 --title "Client Testing" --abstract "Testing the client...." --description "Testing the client...." --license http://dalicc.net/licenselibrary/AdaptivePublicLicense10 --apikey MYSTERIOUS 'https://raw.githubusercontent.com/dbpedia/databus/master/server/app/api/swagger.yml|type=swagger'
6363
```
6464

6565
A few more notes for CLI usage:
@@ -70,11 +70,11 @@ A few more notes for CLI usage:
7070

7171
### Download command
7272
```
73-
python3 -m databusclient downoad --help
73+
databusclient download --help
7474
```
7575

7676
```
77-
Usage: python3 -m databusclient download [OPTIONS] DATABUSURIS...
77+
Usage: databusclient download [OPTIONS] DATABUSURIS...
7878
7979
Arguments:
8080
DATABUSURIS... databus uris to download from https://databus.dbpedia.org,
@@ -102,34 +102,34 @@ Examples of using download command
102102

103103
**File**: download of a single file
104104
```
105-
python3 -m databusclient download https://databus.dbpedia.org/dbpedia/mappings/mappingbased-literals/2022.12.01/mappingbased-literals_lang=az.ttl.bz2
105+
databusclient download https://databus.dbpedia.org/dbpedia/mappings/mappingbased-literals/2022.12.01/mappingbased-literals_lang=az.ttl.bz2
106106
```
107107

108108
**Version**: download of all files of a specific version
109109
```
110-
python3 -m databusclient download https://databus.dbpedia.org/dbpedia/mappings/mappingbased-literals/2022.12.01
110+
databusclient download https://databus.dbpedia.org/dbpedia/mappings/mappingbased-literals/2022.12.01
111111
```
112112

113113
**Artifact**: download of all files with latest version of an artifact
114114
```
115-
python3 -m databusclient download https://databus.dbpedia.org/dbpedia/mappings/mappingbased-literals
115+
databusclient download https://databus.dbpedia.org/dbpedia/mappings/mappingbased-literals
116116
```
117117

118118
**Group**: download of all files with lates version of all artifacts of a group
119119
```
120-
python3 -m databusclient download https://databus.dbpedia.org/dbpedia/mappings
120+
databusclient download https://databus.dbpedia.org/dbpedia/mappings
121121
```
122122

123123
If no `--localdir` is provided, the current working directory is used as base directory. The downloaded files will be stored in the working directory in a folder structure according to the databus structure, i.e. `./$ACCOUNT/$GROUP/$ARTIFACT/$VERSION/`.
124124

125125
**Collcetion**: download of all files within a collection
126126
```
127-
python3 -m databusclient download https://databus.dbpedia.org/dbpedia/collections/dbpedia-snapshot-2022-12
127+
databusclient download https://databus.dbpedia.org/dbpedia/collections/dbpedia-snapshot-2022-12
128128
```
129129

130130
**Query**: download of all files returned by a query (sparql endpoint must be provided with `--databus`)
131131
```
132-
python3 -m databusclient download 'PREFIX dcat: <http://www.w3.org/ns/dcat#> SELECT ?x WHERE { ?sub dcat:downloadURL ?x . } LIMIT 10' --databus https://databus.dbpedia.org/sparql
132+
databusclient download 'PREFIX dcat: <http://www.w3.org/ns/dcat#> SELECT ?x WHERE { ?sub dcat:downloadURL ?x . } LIMIT 10' --databus https://databus.dbpedia.org/sparql
133133
```
134134

135135
#### Authentication with vault
@@ -138,7 +138,7 @@ For downloading files from the vault, you need to provide a vault token. See [ge
138138

139139
Example:
140140
```
141-
python3 -m databusclient download https://databus.dbpedia.org/dbpedia-enterprise/live-fusion-snapshots/fusion/2025-08-23 --token vault-token.dat
141+
databusclient download https://databus.dbpedia.org/dbpedia-enterprise/live-fusion-snapshots/fusion/2025-08-23 --token vault-token.dat
142142
```
143143

144144
If vault authentication is required for downloading a file, the client will use the token. If no vault authentication is required, the token will not be used.

‎databusclient/cli.py‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -12,27 +12,27 @@ def app():
1212

1313
@app.command()
1414
@click.option(
15-
"--versionid",
15+
"--version-id", "version_id",
1616
required=True,
1717
help="Target databus version/dataset identifier of the form "
1818
"<https://databus.dbpedia.org/$ACCOUNT/$GROUP/$ARTIFACT/$VERSION>",
1919
)
2020
@click.option("--title", required=True, help="Dataset title")
2121
@click.option("--abstract", required=True, help="Dataset abstract max 200 chars")
2222
@click.option("--description", required=True, help="Dataset description")
23-
@click.option("--license", required=True, help="License (see dalicc.net)")
23+
@click.option("--license", "license_url", required=True, help="License (see dalicc.net)")
2424
@click.option("--apikey", required=True, help="API key")
2525
@click.argument(
2626
"distributions",
2727
nargs=-1,
2828
required=True,
2929
)
30-
def deploy(version_id, title, abstract, description, license_uri, apikey, distributions: List[str]):
30+
def deploy(version_id, title, abstract, description, license_url, apikey, distributions: List[str]):
3131
"""
3232
Deploy a dataset version with the provided metadata and distributions.
3333
"""
3434
click.echo(f"Deploying dataset version: {version_id}")
35-
dataid = client.create_dataset(version_id, title, abstract, description, license_uri, distributions)
35+
dataid = client.create_dataset(version_id, title, abstract, description, license_url, distributions)
3636
client.deploy(dataid=dataid, api_key=apikey)
3737

3838

‎databusclient/client.py‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -410,12 +410,12 @@ def __download_file__(url, filename, vault_token_file=None, auth_url=None, clien
410410
then fetch Vault access token and retry with Authorization header.
411411
"""
412412

413-
print("Download file: "+url)
414-
os.makedirs(os.path.dirname(filename), exist_ok=True) # Create the necessary directories
415-
413+
print(f"Download file: {url}")
414+
dirpath = os.path.dirname(filename)
415+
if dirpath:
416+
os.makedirs(dirpath, exist_ok=True) # Create the necessary directories
416417
# --- 1. Get redirect URL by requesting HEAD ---
417418
response = requests.head(url, stream=True)
418-
419419
# Check for redirect and update URL if necessary
420420
if response.headers.get("Location") and response.status_code in [301, 302, 303, 307, 308]:
421421
url = response.headers.get("Location")
@@ -450,7 +450,7 @@ def __download_file__(url, filename, vault_token_file=None, auth_url=None, clien
450450
progress_bar.close()
451451

452452
if total_size_in_bytes != 0 and progress_bar.n != total_size_in_bytes:
453-
raise
453+
raise IOError("Downloaded size does not match Content-Length header")
454454

455455

456456
def __get_vault_access__(download_url: str,
@@ -666,7 +666,7 @@ def download(
666666
if "/collections/" in databusURI: # TODO "in" is not safe! there could be an artifact named collections, need to check for the correct part position in the URI
667667
query = __handle_databus_collection__(databusURI)
668668
res = __handle_databus_file_query__(endpoint, query)
669-
__download_list__(res, localDir)
669+
__download_list__(res, localDir, vault_token_file=token, auth_url=auth_url, client_id=client_id)
670670
# databus file
671671
elif file is not None:
672672
__download_list__([databusURI], localDir, vault_token_file=token, auth_url=auth_url, client_id=client_id)
@@ -711,4 +711,4 @@ def download(
711711
if endpoint is None: # endpoint is required for queries (--databus)
712712
raise ValueError("No endpoint given for query")
713713
res = __handle_databus_file_query__(endpoint, databusURI)
714-
__download_list__(res, localDir)
714+
__download_list__(res, localDir, vault_token_file=token, auth_url=auth_url, client_id=client_id)

‎test.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/usr/bin/env bash
22

33
databusclient deploy \
4-
--versionid "https://d8lr.tools.dbpedia.org/hopver/testGroup/testArtifact/1.0-alpha/" \
4+
--version-id "https://d8lr.tools.dbpedia.org/hopver/testGroup/testArtifact/1.0-alpha/" \
55
--title "Test Title" \
66
--abstract "Test Abstract" \
77
--description "Test Description" \

‎tests/test_download.py‎

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,19 @@
55
DEFAULT_ENDPOINT="https://databus.dbpedia.org/sparql"
66
TEST_QUERY="""
77
PREFIX dcat: <http://www.w3.org/ns/dcat#>
8-
SELECT ?x WHERE {
9-
?sub dcat:downloadURL ?x .
10-
} LIMIT 10
8+
SELECT ?file
9+
WHERE {
10+
?file dcat:downloadURL ?url ;
11+
dcat:byteSize ?size .
12+
FILTER(STRSTARTS(STR(?file), "https://databus.dbpedia.org/dbpedia/"))
13+
FILTER(xsd:integer(?size) < 104857600)
14+
}
15+
LIMIT 10
1116
"""
1217
TEST_COLLECTION="https://databus.dbpedia.org/dbpedia/collections/dbpedia-snapshot-2022-12"
1318

1419
def test_with_query():
15-
cl.download("tmp",DEFAULT_ENDPOINT,[TEST_QUERY]
16-
17-
)
20+
cl.download("tmp",DEFAULT_ENDPOINT,[TEST_QUERY])
1821

1922
def test_with_collection():
2023
cl.download("tmp",DEFAULT_ENDPOINT,[TEST_COLLECTION])

0 commit comments

Comments
 (0)