Skip to content

Commit 73795c2

Browse files
committed
Add instructions for getting files paths as the preferred method.
1 parent 114cf39 commit 73795c2

1 file changed

Lines changed: 67 additions & 0 deletions

File tree

sensors/06-list-datasets-by-plot.md

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,72 @@
11
# Generating file lists by plot
22

3+
The terrautils python package has a new products module that aid in connecting
4+
plot boundaries stored within betydb with file-based data products available
5+
from the workbench or Globus.
6+
7+
## Getting started
8+
9+
After installing terrautils, you should be able to import the *prooduct* module.
10+
```
11+
from terrautils.products import get_sensor_list, unique_sensor_names
12+
from terrautils.products import get_file_listing, extract_file_paths
13+
```
14+
15+
The get\_sensor\_list and get\_file\_listing both require connection, url,
16+
and key parameters. *Connection* can be 'None', the *url* (called host in the
17+
code) should be something like https://terraref.ncsa.illinois.edu/clowder/.
18+
The *key* is a unique access key for the Clowder api.
19+
20+
## Getting the sensor list
21+
The first thing to get is the sensor name. This can be retreived using the
22+
get\_sensor\_list function. This function returns the full record which may
23+
be useful in some cases but also includes sensor names that include a plot id
24+
number. The utility function unique_sensor_names accpets the sensor list and
25+
provides a list of names suitable for use in the get_file_listing function.
26+
27+
```
28+
sensors = get_sensor_list(None, url, key)
29+
names = unique_sensor_names(sensors)
30+
```
31+
32+
Names will now contain a list of sensor names support by the Clowder geostreams
33+
API. The currently available sensors are:
34+
35+
* IR Surface Temperature
36+
* Thermal IR GeoTIFFs Datasets
37+
* flirIrCamera Datasets
38+
* (EL) sensor\_weather\_station
39+
* Irrigation Observations
40+
* Canopy Cover
41+
* Energy Farm Observations SE
42+
* (EL) sensor\_par
43+
* scanner3DTop Datasets
44+
* Weather Observations
45+
* Energy Farm Observations NE
46+
* RGB GeoTIFFs Datasets
47+
* (EL) sensor\_co2
48+
* stereoTop Datasets
49+
* Energy Farm Observations CEN
50+
51+
## Getting a list of files
52+
53+
The geostreams API can be used to get a list of datasets that overlap a
54+
specific plot boundary and, optionally, limited by a time range. Iterating
55+
over the datasets allows the paths to all the files to be extracted.
56+
57+
```
58+
sensor = 'Thermal IR GeoTIFFs Datasets'
59+
sitename = 'MAC Field Scanner Season 1 Field Plot 101 W'
60+
datasets = get_file_listing(None, url, key, sensor, sitename)
61+
files = extract_file_paths(datasets)
62+
```
63+
64+
65+
# Alternative method
66+
The following method demonstrates the same approach using the Clowder API. This
67+
approach is useful for understanding the data layout and when the Python
68+
terrautils package is not available.
69+
370
## Finding plot ID
471
```
572
SENSOR_NAME = "MAC Field Scanner Season 1 Field Plot 101 W"

0 commit comments

Comments
 (0)