Skip to content

Za/data access#661

Closed
amrc-za wants to merge 96 commits into
mainfrom
za/data-access
Closed

Za/data access#661
amrc-za wants to merge 96 commits into
mainfrom
za/data-access

Conversation

@amrc-za

@amrc-za amrc-za commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

New Data Access Service. It supports HTTP WebApi and Notify API.

WebAPI supports endpoints to

  • GET /metadata - lists dataset uuids
  • GET /metadata/:uuid - returns dataset metadata
  • POST /data/:uuid - downloads dataset data in ZIP (csv file per device)
  • GET /structure - lists dataset uuids
  • POST /structure - creates new dataset
  • PUT /structure/:uuid - updates existing dataset
  • GET /delete/:uuid - deletes dataset

Notify API supports:
WATCH /metadata
WATCH /metadata/:uuid
SEARCH /metadata

WATCH /strcuture
WATCH /structure/:uuid
SEARCH /structure

@amrc-za amrc-za requested a review from djnewbould June 8, 2026 09:21
amrc-za added 29 commits June 8, 2026 10:23
@amrc-za
data access skeleton
f0e656b
@amrc-za
data-access service deployment files
a229957
@amrc-za
js-rx-client - new method to watch acl with permission
6739659
@amrc-za
dataflow start
02ac346
@amrc-za
gitignore
b530f68
@amrc-za
acs-config-db package.lock update
c904d49
@amrc-za
data access helm
248967f
@amrc-za
node modules dockerignored
4404885
@amrc-za
deploy data access
48dc1f1
@amrc-za
data access fixed keytabs/server not found
ea1c9dd
@amrc-za
data access - dataflow build metadata and code clean up
c25f2b3
@amrc-za
deploy - fixed git krbs keytabs not being created
9155114
@amrc-za
data access dataflow - keeping track of datasets with contents not ju…
4ad614a 
…st list of uuids
@amrc-za
data access get /metadata api ready
08a7eaa
@amrc-za
data access api created helper method to get list dataset uuids
c7d7e4d
@amrc-za
data access dataflow added method to get dataset content by uuid
112ebdd
@amrc-za
data access api structure_uuid
f38d591
@amrc-za
data-access space format and comments
efe3463
@amrc-za
data access api method name change
ce599bb
@amrc-za
data access api.js - passing configDB instance to apiV1
474afdc
@amrc-za
data access - constants - removed structure dataset type uuids
8f2d878
@amrc-za
data access - flattened dataset definitions map to keys to be dataset…
4c7af11 
… uuids
@amrc-za
data access api-v1 implemented put structure
fe7b367
@amrc-za
data access - package json added influx client lib
bf8d5fa
@amrc-za
data access - added dataset validity enum
3487323
@amrc-za
data access api and dataflow update to keep track of invalid datasets
7df7059
@amrc-za
data access - get data from influx partial implementation
7cc88b8
@amrc-za
deploy data access - added influx vars and secret
d2e6350
@amrc-za
data access updated flux query
40c89d9
amrc-za added 27 commits June 8, 2026 10:23
@amrc-za
data access installed deep-equal
d14bc96
@amrc-za
data access - fixed notify metadata SEARCH
d21230c
@amrc-za
data access structure uuid - removed from and to fields from returnin…
817c74c 
…g obj
@amrc-za
data access package-lock
c76c6ae
@amrc-za
data-access notify structure SEARCH done
e6017ba
@amrc-za
data access added test script for rx client
a46e60f
@amrc-za
data-access download data http checking if write is complete
94b058c
@amrc-za
data access added simple test http endpoints via service client
66ea3e6
@amrc-za
js-service-client updated data access interface
ef79e5e
@amrc-za
data access package lock
b375767
@amrc-za
data access download http endpoint returns zip with separate csv file…
dddb38d 
…s per device
@amrc-za
data access js service client download_data handles zip
e644060
@amrc-za
data access indlux query removed unnecessary cols
969e0cd
@amrc-za
data access removed unnecessary comment
51fa965
@amrc-za
data access GET /metadata vitest
b41d59c
@amrc-za
data access http GET /metadata tests - fixed fplus client cache bug
a705281
@amrc-za
data access tests GET /metadata added parent child invalid tests
259fcfd
@amrc-za
data access tests GET /metadata added invalid session child test
ed05520
@amrc-za
data access tests GET /metadata removed cache
990aeeb
@amrc-za
data access tests GET /structure
a375255
@amrc-za
data access webapi added config validation
ae4965d
@amrc-za
data-access tests structure creaate
4eb6a71
@amrc-za
data access tests temp data
8c5033c
@amrc-za
data access added delete endpoint
98a6f64
@amrc-za
data access test create added subclass relationship tests
a90ee08
@amrc-za
data access tests update
dbc238f
@amrc-za
data access test scripts
e2e10f2
@amrc-za amrc-za closed this Jun 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@djnewbould djnewbould Awaiting requested review from djnewbould

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@amrc-za