You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TODO: move this to a separate tutorial page focused on using curl
92
+
-->
93
+
94
+
The source files behind the data are available for downloading through the API. By executing a series
95
+
of requests against the API it's possible to determine the files of interest and then download them.
96
+
97
+
Each of the API URL's have the same beginning (https://terraref.ncsa.illinois.edu/clowder/api),
98
+
followed by the data needed for a specific request. As we step through the process you will be able
99
+
to see how then end of the URL changes depending upon the reuqest.
100
+
101
+
Below is what the API looks like as a URL. Try pasting it into your browser.
102
+
103
+
[https://terraref.ncsa.illinois.edu/clowder/api/geostreams/sensors?sensor_name=MAC Field Scanner Season 1 Field Plot 101 W](https://terraref.ncsa.illinois.edu/clowder/api/geostreams/sensors?sensor_name=MAC Field Scanner Season 1 Field Plot 101 W)
104
+
105
+
This will return data for the requested plot including its id. This id (or identifier) can then be used for
106
+
additional queries against the API.
76
107
77
-
The following method demonstrates the same approach using the Clowder API. This
78
-
approach is useful for understanding the data layout and when the Python
79
-
terrautils package is not available.
108
+
In the examples below we will be using **curl** on the command line to make our API calls. Since the
109
+
API is accessed through URLs, it's possible to use the URLs in software programs or with a programming language
110
+
to retrieve its data.
111
+
112
+
## A Word of Caution
113
+
114
+
We are no longer using the python terrautils package, which is a python library that provides helper functions that simplify interactions with the Clowder API. One of the ways it makes the interface easier is by using function names that make sense in the scope of the project. The API and the Clowder database have different names and _this is confusing_ since the same names are used for different parts of the database.
115
+
116
+
The names and meanings of variables in this section don't necessarily match the ones in the section
117
+
above and it may be easy to get them confused. The API queries the database directly and thereby reflects
118
+
the database structure. This is the main reason for the naming differences between the API and the terraref
119
+
client.
120
+
121
+
For example, the Clowder API's use of the term *SENSOR_NAME* is equivalent to *site_name* above.
80
122
81
123
## Finding plot ID
82
124
83
-
```
84
-
SENSOR_NAME = "MAC Field Scanner Season 1 Field Plot 101 W"
85
-
GET https://terraref.ncsa.illinois.edu/clowder/api/geostreams/sensors?sensor_name={SENSOR_NAME}
125
+
We can query the API to find the identifier associated with the name of a plot. For this example
126
+
we use the variable name of SENSOR_DATA to indicate the name of the plot.
127
+
128
+
```{sh eval=FALSE}
129
+
SENSOR_NAME="MAC Field Scanner Season 1 Field Plot 101 W"
130
+
curl -o plot.json -X GET "https://terraref.ncsa.illinois.edu/clowder/api/geostreams/sensors?sensor_name=${SENSOR_NAME}"
86
131
```
87
132
88
-
This returns a JSON object with an 'id' parameter. You can use this ID parameter to specify the right data stream.
133
+
This creates a file named *plot.json* containing the JSON object returned by the API. The JSON object has an
134
+
'id' parameter. This ID parameter can be used to specify the correct data stream.
89
135
90
136
## Finding stream ID within a plot
91
137
92
-
The names are formatted as "<SensorGroup> Datasets (<SensorID>)".
138
+
Using the sensor ID returned in the JSON from the previous call and the id of a sensor returned previously to get
139
+
the stream id. The names of streams are are formatted as "<SensorGroup> Datasets (<SensorID>)".
93
140
94
-
```
95
-
SENSOR_ID = 3355
96
-
STREAM_NAME = "Thermal IR GeoTIFFs Datasets ({SENSOR_ID})"
97
-
GET https://terraref.ncsa.illinois.edu/clowder/api/geostreams/streams?stream_name={STREAM_NAME}
141
+
```{sh eval=FALSE}
142
+
SENSOR_ID=3355
143
+
STREAM_NAME="Thermal IR GeoTIFFs Datasets (${SENSOR_ID})"
144
+
curl -o stream.json -X GET "https://terraref.ncsa.illinois.edu/clowder/api/geostreams/streams?stream_name=${STREAM_NAME}"
98
145
```
99
146
100
-
This returns a JSON object with an 'id' parameter. You can use this ID parameter to get the right datapoints.
147
+
A file named *stream.json* will be created containing the returned JSON object. This JSON object has an 'id' parameter that
148
+
contains the stream ID. You can use this ID parameter to get the datasets, and then datapoints, of interest.
101
149
102
-
## Listing Clowder file IDs for that plot & sensor stream
150
+
## Listing Clowder dataset IDs for that plot & sensor stream
103
151
104
-
```
105
-
STREAM_ID = "11586"
106
-
GET https://terraref.ncsa.illinois.edu/clowder/api/geostreams/datapoints?stream_id={STREAM_ID}
152
+
We now have a stream ID that we can use to list our datasets. The datasets in turn contain files of interest.
153
+
154
+
```{sh eval=FALSE}
155
+
STREAM_ID=11586
156
+
curl -o datasets.json -X GET "https://terraref.ncsa.illinois.edu/clowder/api/geostreams/datapoints?stream_id=${STREAM_ID}"
107
157
```
108
158
109
-
This returns a list of datapoint JSON objects, each with a 'properties' parameter that looks like:
159
+
After the call succeeds, a file named *datasets.json* is created containing the returned JSON onject. As part of the
160
+
JSON object there are one or more `properties` fields containing *source_dataset* parameters.
110
161
111
-
```{python}
162
+
```{javascript eval=FALSE}
112
163
properties: {
113
164
dataset_name: "Thermal IR GeoTIFFs - 2016-05-09__12-07-57-990",
curl -o "${FILE_NAME}" -X GET "https://terraref.ncsa.illinois.edu/clowder/api/files/${FILE_ID}"
223
+
```
224
+
225
+
This call will cause the server to return the contents of the file identified in the URL. This file is then stored locally in *ir_geotiff_L1_ua-mac_2016-05-09__12-07-57-990.tif*.
0 commit comments