Skip to content

Commit 026acff

Browse files
kimberlyh66kimberlyh66
committed
Merge branch 'traits_tutorials' of github.com:terraref/tutorials into traits_tutorials
2 parents 6c0a6ad + cff660f commit 026acff

2 files changed

Lines changed: 110 additions & 108 deletions

File tree

‎traits/02-betydb-api-access.Rmd‎

Lines changed: 106 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -1,106 +1,106 @@
1-
# Accessing Trait Data Via the BETYdb API
2-
3-
## Using URLs to construct Queries
4-
5-
The first step toward reproducible pipelines is to automate the process of searching the database and returning results. This is one of the key roles of an Application programming interface, or 'API'. You can learn to use the API in less than 20 minutes, starting now.
6-
7-
### What is an API?
8-
9-
An API is an 'Application Programming Interface'. An API is a way that you and your software can connect to and access data.
10-
11-
All of our databases have web interfaces for humans to browse as well as APIs that are constructed as URLs.
12-
13-
14-
### Using Your API key to Connect
15-
16-
An API key is like a password. It allows you to access data, and should be kept private.
17-
Therefore, we are not going to put it in code that we share. The one exception is the key 9999999999999999999999999999999999999999 that will allow you to access metadata tables (all tables except _traits_ and _yields_). It will also allow you to access all of the simulated data in the https://terraref.ncsa.illinois.edu/bety-test database.
18-
19-
A common way of handling private API keys is to place it in a text file in your current directory.
20-
Don't put it in a project directory where it might be inadvertently shared.
21-
22-
Here is how to find and save your API key:
23-
24-
* click file --> new --> text file
25-
* copy the api key that was sent when you registered into the file
26-
* file --> save as '.betykey'
27-
28-
For the public key, you can call this file `.betykey_public`.
29-
30-
### Ways to access API data
31-
32-
1. Through a URL query
33-
2. Using the bash shell
34-
3. Using the R jsonlite package
35-
36-
37-
### Accessing data using a URL query
38-
39-
40-
## Components of a URL query
41-
42-
* base url: `terraref.ncsa.illinois.edu/bety`
43-
* path to the api: `/api/v1`
44-
* api endpoint: `/search` or `traits` or `sites`. For BETYdb, these are the names of database tables.
45-
* Query parameters: `genus=Sorghum`
46-
* Authentication: `key=9999999999999999999999999999999999999999` is the public key for the TERRA REF traits database.
47-
48-
## Constructing a URL query
49-
50-
First, lets construct a query by putting together a URL.
51-
52-
1. start with the database url: `terraref.ncsa.illinois.edu/bety`
53-
* this url brings you to the home page
54-
2. Add the path to the API, `/api/v1`
55-
* now we have terraref.ncsa.illinois.edu/bety/api/v1, which points to the API documentation
56-
3. Add the name of the table you want to query. Lets start with `variables`
57-
* terraref.ncsa.illinois.edu/bety/api/v1/variables
58-
4. add query terms by appending a `?` and combining with `&`, for example:
59-
* `key=9999999999999999999999999999999999999999`
60-
* `type=trait` where the variable type is 'trait'
61-
* `name=~height` where the variable name contains 'height'
62-
5. This is your complete query:
63-
* `terraref.ncsa.illinois.edu/bety/api/v1/variables?type=trait&name=~height&key=9999999999999999999999999999999999999999`
64-
* it will query all variables that are type trait and have 'height' in the name
65-
* Does it return the expected values?
66-
67-
## Your Turn
68-
69-
> What will the URL https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=9999999999999999999999999999999999999999 return?
70-
71-
> Write a URL that will query the database for sites with "Field Scanner" in the name field. Hint: combine two terms with a `+` as in `Field+Scanner`
72-
73-
What do you see? Do you think that this is all of the records? What happens if you add `&limit=none`?
74-
75-
76-
77-
#### Accessing data using the Shell
78-
79-
Type the following command into a bash shell (the `-o` option names the output file):
80-
81-
```sh
82-
curl -o sorghum.json \
83-
"https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=9999999999999999999999999999999999999999"
84-
```
85-
86-
If you want to write the query without exposing the key in plain text, you can construct it like this:
87-
88-
```sh
89-
curl -o sorghum.json \
90-
"https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=`cat .betykey_public`"
91-
```
92-
93-
### Accessing API data using the R jsonlite package
94-
95-
```{r text-api, warning = FALSE}
96-
sorghum.json <- readLines(
97-
paste0("https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=",
98-
readLines('traits/.betykey')))
99-
100-
## print(sorghum.json)
101-
## not a particularly useful format
102-
## lets convert to a data frame
103-
sorghum <- jsonlite::fromJSON(sorghum.json)
104-
```
105-
106-
More on how to use the rOpenSci traits package coming up in the [next tutorial](03-access-r-traits.Rmd)
1+
# Accessing Trait Data Via the BETYdb API
2+
3+
## Using URLs to construct Queries
4+
5+
The first step toward reproducible pipelines is to automate the process of searching the database and returning results. This is one of the key roles of an Application programming interface, or 'API'. You can learn to use the API in less than 20 minutes, starting now.
6+
7+
### What is an API?
8+
9+
An API is an 'Application Programming Interface'. An API is a way that you and your software can connect to and access data.
10+
11+
All of our databases have web interfaces for humans to browse as well as APIs that are constructed as URLs.
12+
13+
14+
### Using Your API key to Connect
15+
16+
An API key is like a password. It allows you to access data, and should be kept private.
17+
Therefore, we are not going to put it in code that we share. The one exception is the key 9999999999999999999999999999999999999999 that will allow you to access metadata tables (all tables except _traits_ and _yields_). It will also allow you to access all of the simulated data in the https://terraref.ncsa.illinois.edu/bety-test database.
18+
19+
A common way of handling private API keys is to place it in a text file in your current directory.
20+
Don't put it in a project directory where it might be inadvertently shared.
21+
22+
Here is how to find and save your API key:
23+
24+
* click file --> new --> text file
25+
* copy the api key that was sent when you registered into the file
26+
* file --> save as '.betykey'
27+
28+
For the public key, you can call this file `.betykey_public`.
29+
30+
### Ways to access API data
31+
32+
1. Through a URL query
33+
2. Using the bash shell
34+
3. Using the R jsonlite package
35+
36+
37+
### Accessing data using a URL query
38+
39+
40+
## Components of a URL query
41+
42+
* base url: `terraref.ncsa.illinois.edu/bety`
43+
* path to the api: `/api/v1`
44+
* api endpoint: `/search` or `traits` or `sites`. For BETYdb, these are the names of database tables.
45+
* Query parameters: `genus=Sorghum`
46+
* Authentication: `key=9999999999999999999999999999999999999999` is the public key for the TERRA REF traits database.
47+
48+
## Constructing a URL query
49+
50+
First, lets construct a query by putting together a URL.
51+
52+
1. start with the database url: `terraref.ncsa.illinois.edu/bety`
53+
* this url brings you to the home page
54+
2. Add the path to the API, `/api/v1`
55+
* now we have terraref.ncsa.illinois.edu/bety/api/v1, which points to the API documentation
56+
3. Add the name of the table you want to query. Lets start with `variables`
57+
* terraref.ncsa.illinois.edu/bety/api/v1/variables
58+
4. add query terms by appending a `?` and combining with `&`, for example:
59+
* `key=9999999999999999999999999999999999999999`
60+
* `type=trait` where the variable type is 'trait'
61+
* `name=~height` where the variable name contains 'height'
62+
5. This is your complete query:
63+
* `terraref.ncsa.illinois.edu/bety/api/v1/variables?type=trait&name=~height&key=9999999999999999999999999999999999999999`
64+
* it will query all variables that are type trait and have 'height' in the name
65+
* Does it return the expected values?
66+
67+
## Your Turn
68+
69+
> What will the URL https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=9999999999999999999999999999999999999999 return?
70+
71+
> Write a URL that will query the database for sites with "Field Scanner" in the name field. Hint: combine two terms with a `+` as in `Field+Scanner`
72+
73+
What do you see? Do you think that this is all of the records? What happens if you add `&limit=none`?
74+
75+
76+
77+
#### Accessing data using the Shell
78+
79+
Type the following command into a bash shell (the `-o` option names the output file):
80+
81+
```sh
82+
curl -o sorghum.json \
83+
"https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=9999999999999999999999999999999999999999"
84+
```
85+
86+
If you want to write the query without exposing the key in plain text, you can construct it like this:
87+
88+
```sh
89+
curl -o sorghum.json \
90+
"https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=`cat .betykey_public`"
91+
```
92+
93+
### Accessing API data using the R jsonlite package
94+
95+
```{r text-api, warning = FALSE}
96+
sorghum.json <- readLines(
97+
paste0("https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=",
98+
readLines('traits/.betykey')))
99+
100+
## print(sorghum.json)
101+
## not a particularly useful format
102+
## lets convert to a data frame
103+
sorghum <- jsonlite::fromJSON(sorghum.json)
104+
```
105+
106+
More on how to use the rOpenSci traits package coming up in the [next tutorial](03-access-r-traits.Rmd)

‎traits/03-access-r-traits.Rmd‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@ The rOpenSci traits package makes it easier to query the TERRA REF trait databas
66

77
First, make sure we have the latest version from the terraref fork of the repository on github. (you can install using the standard `install.packages('traits')` but as of Jan 2018 this version times out on very large datasets).
88

9-
### Install the package
9+
### Install the traits package
10+
11+
This is for users working on their own computer - if you are using the TERRA REF Rstudio Docker container (including on workbench) you can skip this step.
1012

1113
```{r install_traits, echo = FALSE, include = FALSE}
1214
devtools::install_github('terraref/traits')
@@ -16,7 +18,6 @@ Now, we can load the packages that we will need to get started.
1618

1719
```{r 00-setup, message = FALSE}
1820
library(traits)
19-
knitr::opts_chunk$set(echo = FALSE, cache = TRUE)
2021
library(ggplot2)
2122
library(ggthemes)
2223
theme_set(theme_bw())
@@ -59,6 +60,7 @@ sorghum_info <- betydb_query(table = 'species',
5960
Notice all of the arguments that the `betydb_query` function requires? We can change this by setting the default connection options thus:
6061

6162

63+
6264
```{r set-up, echo = TRUE}
6365
options(betydb_key = readLines('traits/.betykey', warn = FALSE),
6466
betydb_url = "https://terraref.ncsa.illinois.edu/bety/",

0 commit comments

Comments
 (0)