fixed link to agronomic metadata tutorial (#35)

* fixed link to agronomic metadata tutorial

* Updated tutorial. Added a summary showing the 3 ways to access API data.

* Further updated the tutorial on how to access API using URL query and bash shell.

* Further updated tutorial. Added short comment on the use of the fromJSON function.

* Further updated R jsonlite portion of tutorial

* Update traits/01-web-access.Rmd

Co-Authored-By: kimberlyh66 <44081116+kimberlyh66@users.noreply.github.com>

* added comment. link to traitvis webapp not working.

* fixed links to terraref and pecan sites, betydb schemas, and doi reference. also updated comment on embedded shiny app (link to traitvis app does work).

* updated beta user program link, and fixed links to agronomic metadata tutorial and terraref bety home page

* made changes according to previous comments

* Update traits/04-danforth-indoor-phenotyping-facility.Rmd

Co-Authored-By: kimberlyh66 <44081116+kimberlyh66@users.noreply.github.com>

* made minor edits - assuming betykey in traits directory since ~ does not work on Windows

Author: kimberlyh66

index.Rmd

@@ -13,4 +13,6 @@ output:
 
 ```{r}
 knitr::opts_chunk$set(echo = FALSE, cache = TRUE)
-```
+options(warn = -1)
+```
+

traits/00-BETYdb-getting-started.Rmd

@@ -14,15 +14,15 @@ It contains trait (phenotype) data at the plot or plant level as well as meta da
 ### Introduction to BETYdb 
 
 The TERRA REF trait database (terraref.ncsa.illinois.edu/bety) uses the BETYdb data schema (structure) and web application.
-The BETYdb software is actively used and developed by the [TERRA Reference](terraref.org) program as well as by the [PEcAn project](pecanproject.org).
+The BETYdb software is actively used and developed by the [TERRA Reference](http://terraref.org) program as well as by the [PEcAn project](http://pecanproject.org).
 
 For more information about BETYdb, see the following:
 
 * BETYdb documentation (available via the web application under 'Docs')
   * _Data Access_: how to access data
   * _Data Entry Workflow:_ how to add data to the database
-  * _BETYdb Technical Documentation_ is written for advanced users and website and database administrators who may also be interested in the [full database schema](betydb.org/schemas)
-* BETYdb: A Yield, Trait and Ecosystem Service Database Applied to Second Generation Bioenergy Feedstocks. ([LeBauer et al, 2017](dx.doi.org/10.1111/gcbb.12420))
+  * _BETYdb Technical Documentation_ is written for advanced users and website and database administrators who may also be interested in the [full database schema](https://www.betydb.org/schemas)
+* BETYdb: A Yield, Trait and Ecosystem Service Database Applied to Second Generation Bioenergy Feedstocks. ([LeBauer et al, 2017](https://onlinelibrary.wiley.com/doi/abs/10.1111/gcbb.12420))
 
 Other than the TERRA REF trait database, there are a handful of other projects that use the BETYdb software, mostly with the PEcAn and TERRA programs. The content presented here is focused on the TERRA REF instance of BETYdb. Most of the information presented here is relevant to other databases, but the TERRA program has more emphasis on trait diversity among cultivars or genotypes within a crop whereas PEcAn focuses on the diversity of traits within ecosystems and plant functional types. In addition, the TERRA program is more focused on high throughput phenotyping - intensive monitoring of agricultural breeding trials whereas PEcAn focuses on assimilating heterogeneous data to forecast ecosystem functioning. Fortunately, both uses can use the shared ecosystem of software used for these tasks. For example, the PEcAn crop modeling infrastructure can be directly used to infer additional targets of breeding, and the diversity of traits observed in breeding trials can be a first step toward predicting the impacts of crop traits on productivity and ecosystem functioning.
 
@@ -40,5 +40,5 @@ The traitvis webapp provides an interface for exploring available data that is u
 * website: https://traitvis.workbench.terraref.org
 
 ```{r}
-knitr::include_app("https://traitvis.workbench.terraref.org/", height = "1400px")
+knitr::include_app("https://traitvis.workbench.terraref.org", height = "1400px") #not working; shiny app is not being displayed in html output
 ```

traits/01-web-access.Rmd

@@ -12,7 +12,7 @@ output: html_document
 ### Web interface
 
 * Sign up for an account at https://terraref.ncsa.illinois.edu/bety
-* Sign up for the TERRA REF [beta user program](https://docs.google.com/forms/d/e/1FAIpQLScBsD042RrRok70BCGCRwARTcm9etvVHqvQaz1c5X7c5y0H3w/viewform?c=0&w=1)
+* Sign up for the TERRA REF [beta user program](https://docs.google.com/forms/d/e/1FAIpQLScIUJL_OSL9BvBOdlczErds3aOg5Lwz4NIdNQnUiXdsLsYdhw/viewform)
 * Wait for database access to be granted
 * Your API key will be sent in the email. It can also be found - and regenerated - by navigating to the Users page (data --> [users](https://terraref.ncsa.illinois.edu/bety/users)) in the web interface.
 
@@ -25,11 +25,11 @@ On the Welcome page there is a search option for trait and yield data. This tool
 
 ### Download search results as as csv file from the web interface
 
-* Point your browser to terraref.ncsa.illinois.edu/bety
+* Point your browser to https://terraref.ncsa.illinois.edu/bety/
 * login 
 * enter "NDVI" in the search box
 * on the next page you will see the results of this search 
   * if you want all of the data, including data that has not gone through QA/QC, make sure to check the 'include unchecked records' option
 * in the upper right, you will see a button that will allow you to download the search results as a CSV file. Click it. Open the file in a text editor or spreadsheet program and review its contents.
 
-Note that the web interface only provides a core set of data and limited meta-data. To access all of the data within BETYdb, it is necessary to search and merge multiple tables. More complex queries, such as those in the [Agronomic metadata](../traits/04-agronomic-metadata.Rmd).
+Note that the web interface only provides a core set of data and limited meta-data. To access all of the data within BETYdb, it is necessary to search and merge multiple tables. More complex queries, such as those in the [Agronomic metadata](06-agronomic-metadata.Rmd).

traits/02-betydb-api-access.Rmd

@@ -1,107 +1,112 @@
----
-title: "Accessing Trait Data Via the BETYdb API"
-author: "David LeBauer"
-date: "11/7/2017"
-output: html_document
----
-
-
-## Using URLs to construct Queries
-
-The first step toward reproducible pipelines is to automate the process of searching the database and returning results. This is one of the key roles of an Application programming interface, or 'API'. You can learn to use the API in less than 20 minutes, starting now. 
-
-### What is an API?
-
-An API is an 'Application Programming Interface'. An API is a way that you and your software can connect to and access data. 
-
-All of our databases have web interfaces for humans to browse as well as APIs that are constructed as URLs. 
-
-
-### Using Your API key to Connect
-
-An API key is like a password. It allows you to access data, and should be kept private. 
-Therefore, we are not going to put it in code that we share. The one exception is the key 9999999999999999999999999999999999999999 that will allow you to access metadata tables (all tables except _traits_ and _yields_). It will also allow you to access all of the simulated data in the terraref.ncsa.illinois.edu/bety-test database.
-
-A common way of handling private API keys is to place it in a text file in your home directory. 
-Don't put it in a project directory where it might be inadvertently shared.
-
-Here is how to find and save your API key:
-
-* click file --> new --> text file
-* copy the api key that was sent when you registered into the file
-* file --> save as '~/.betykey'
-
-For the public key, you can call this file `~/.betykey_public`. 
-
-### Components of a URL query
-
-
-* base url: `terraref.ncsa.illinois.edu/bety`
-* path to the api: `/api/beta`
-* api endpoint: `/search` or `traits` or `sites`. For BETYdb, these are the names of database tables. 
-* Query parameters: `genus=Sorghum`
-* Authentication: `key=9999999999999999999999999999999999999999` is the public key for the TERRA REF traits database. 
-
-
-### Constructing a URL query
-
-First, lets construct a query by putting together a URL.
-
-1. start with the database url: `terraref.ncsa.illinois.edu/bety`
-  * this url brings you to the home page
-2. Add the path to the API, `/api/beta`
-  * now we have terraref.ncsa.illinois.edu/bety/api/beta, which points to the API documentation
-3. Add the name of the table you want to query. Lets start with `variables`
-  * terraref.ncsa.illinois.edu/bety/api/beta/variables
-4. add query terms by appending a `?` and combining with `&`, for example:
-  * `key=9999999999999999999999999999999999999999`
-  * `type=trait` where the variable type is 'trait'
-  * `name=~height` where the variable name contains 'height'
-5. This is your complete query:
-  * `terraref.ncsa.illinois.edu/bety/api/beta/variables?type=trait&name=~height&key=9999999999999999999999999999999999999999`
-  * it will query all variables that are type trait and have 'height' in the name
-  * Does it return the expected values?
-  
-
-#### Your Turn
-
-> What will the URL https://terraref.ncsa.illinois.edu/bety/api/beta/species?genus=Sorghum&key=9999999999999999999999999999999999999999 return?
-
-> write a URL that will query the database for sites with "Field Scanner" in the name field. Hint: combine two terms with a `+` as in `Field+Scanner`
-
-What do you see? Do you think that this is all of the records? What happens if you add `&limit=none`? 
-
-### Our first Query 
-
-#### Shell
-
-```sh
-wget -O sorghum.json \\ # -O names the output file 
-   "https://terraref.ncsa.illinois.edu/bety/api/beta/species?genus=Sorghum&key=999999999999999999999999999999999999
-9999"
-```
-
-If you want to write the query without exposing the key in plain text, you can construct it thus:
-
-```sh
-wget -O sorghum.json \\
-    "https://terraref.ncsa.illinois.edu/bety/api/beta/species?genus=Sorghum&key=`cat ~/.betykey_public`"
-```
-
-> What does `cat ~/.betykey_public` do?
-
-> How can you look at the files?
-
-
-#### R - using the jsonlite package
-
-```{r text-api}
-sorghum.json <- readLines(
-  paste0("https://terraref.ncsa.illinois.edu/bety/api/beta/species?genus=Sorghum&key=", 
-         readLines('~/.betykey')))
-
-## print(sorghum.json) 
-## not a particularly useful format
-## lets convert to a data frame
-sorghum <- jsonlite::fromJSON(sorghum.json)
-```
+---
+title: "Accessing Trait Data Via the BETYdb API"
+author: "David LeBauer"
+date: "11/7/2017"
+output: html_document
+---
+
+
+## Using URLs to construct Queries
+
+The first step toward reproducible pipelines is to automate the process of searching the database and returning results. This is one of the key roles of an Application programming interface, or 'API'. You can learn to use the API in less than 20 minutes, starting now. 
+
+### What is an API?
+
+An API is an 'Application Programming Interface'. An API is a way that you and your software can connect to and access data. 
+
+All of our databases have web interfaces for humans to browse as well as APIs that are constructed as URLs. 
+
+
+### Using Your API key to Connect
+
+An API key is like a password. It allows you to access data, and should be kept private. 
+Therefore, we are not going to put it in code that we share. The one exception is the key 9999999999999999999999999999999999999999 that will allow you to access metadata tables (all tables except _traits_ and _yields_). It will also allow you to access all of the simulated data in the https://terraref.ncsa.illinois.edu/bety-test database.
+
+A common way of handling private API keys is to place it in a text file in your current directory. 
+Don't put it in a project directory where it might be inadvertently shared.
+
+Here is how to find and save your API key:
+
+* click file --> new --> text file
+* copy the api key that was sent when you registered into the file
+* file --> save as '.betykey'
+
+For the public key, you can call this file `.betykey_public`. 
+
+### Ways to access API data
+
+1. Through a URL query
+2. Using the bash shell
+3. Using the R jsonlite package
+
+
+### Accessing data using a URL query
+
+
+## Components of a URL query
+
+* base url: `terraref.ncsa.illinois.edu/bety`
+* path to the api: `/api/v1`
+* api endpoint: `/search` or `traits` or `sites`. For BETYdb, these are the names of database tables. 
+* Query parameters: `genus=Sorghum`
+* Authentication: `key=9999999999999999999999999999999999999999` is the public key for the TERRA REF traits database. 
+
+## Constructing a URL query
+
+First, lets construct a query by putting together a URL.
+
+1. start with the database url: `terraref.ncsa.illinois.edu/bety`
+  * this url brings you to the home page
+2. Add the path to the API, `/api/v1`
+  * now we have terraref.ncsa.illinois.edu/bety/api/v1, which points to the API documentation
+3. Add the name of the table you want to query. Lets start with `variables`
+  * terraref.ncsa.illinois.edu/bety/api/v1/variables
+4. add query terms by appending a `?` and combining with `&`, for example:
+  * `key=9999999999999999999999999999999999999999`
+  * `type=trait` where the variable type is 'trait'
+  * `name=~height` where the variable name contains 'height'
+5. This is your complete query:
+  * `terraref.ncsa.illinois.edu/bety/api/v1/variables?type=trait&name=~height&key=9999999999999999999999999999999999999999`
+  * it will query all variables that are type trait and have 'height' in the name
+  * Does it return the expected values?
+  
+## Your Turn
+
+> What will the URL https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=9999999999999999999999999999999999999999 return?
+
+> Write a URL that will query the database for sites with "Field Scanner" in the name field. Hint: combine two terms with a `+` as in `Field+Scanner`
+
+What do you see? Do you think that this is all of the records? What happens if you add `&limit=none`? 
+
+
+
+#### Accessing data using the Shell
+
+Type the following command into a bash shell (the `-o` option names the output file): 
+
+```sh
+curl -o sorghum.json \
+   "https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=9999999999999999999999999999999999999999"
+```
+
+If you want to write the query without exposing the key in plain text, you can construct it like this:
+
+```sh
+curl -o sorghum.json \
+    "https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=`cat .betykey_public`"
+```
+
+### Accessing API data using the R jsonlite package
+
+```{r text-api}
+sorghum.json <- readLines(
+  paste0("https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=", 
+         readLines('.betykey')))
+
+## print(sorghum.json) 
+## not a particularly useful format
+## lets convert to a data frame
+sorghum <- jsonlite::fromJSON(sorghum.json)
+```
+
+More on how to use the rOpenSci traits package coming up in the [next tutorial](03-access-r-traits.Rmd)

traits/03-access-r-traits.Rmd

@@ -33,11 +33,11 @@ library(dplyr)
 ```{r writing-key}
 # This should be done once with the key sent to you in your email
 # writeLines('abcdefg_rest_of_key_sent_in_email', 
-#            con = '~/.betykey')
+#            con = '.betykey')
 
 # Example with the public key:
 writeLines('9999999999999999999999999999999999999999', 
-           con = '~/.betykey_public')
+           con = '.betykey_public')
 ```
 
 #### R - using the traits package
@@ -55,7 +55,7 @@ sorghum_info <- betydb_query(table = 'species',
                             api_version = 'beta',
                             limit = 'none',
                             betyurl = "https://terraref.ncsa.illinois.edu/bety/", 
-                            key = readLines('~/.betykey', warn = FALSE))
+                            key = readLines('.betykey', warn = FALSE))
 
 ```
 
@@ -65,7 +65,7 @@ Notice all of the arguments that the `betydb_query` function requires? We can ch
 
 
 ```{r}
-options(betydb_key = readLines('~/.betykey', warn = FALSE),
+options(betydb_key = readLines('.betykey', warn = FALSE),
         betydb_url = "https://terraref.ncsa.illinois.edu/bety/",
         betydb_api_version = 'beta')
 ```

traits/04-danforth-indoor-phenotyping-facility.Rmd

@@ -2,7 +2,8 @@
 title: "Phenotype Analysis"
 author: "David LeBauer, Craig Willis"
 date: "`r Sys.Date()`"
-output: md_document
+output:
+  html_document: default
 ---
 
 ```{r 02-setup, include=FALSE}
@@ -26,7 +27,7 @@ library(traits)
 Unlike the first two tutorials, now we will be querying real data from the public TERRA REF database. So we will use a new URL, https://terraref.ncsa.illinois.edu/bety/, and we will need to use our own private key.
 
 ```{r terraref-connect-options}
-options(betydb_key = readLines('~/.betykey', warn = FALSE),
+options(betydb_key = readLines('.betykey'),
         betydb_url = "https://terraref.ncsa.illinois.edu/bety/",
         betydb_api_version = 'beta')
 ```
@@ -114,4 +115,4 @@ ggplot(data = danforth_sorghum, aes(x = date, y = mean, color = cultivar)) +
 2. An 'entity' is a replicate. 
    * How many entities are there? 
    * How many entities per cultivar? 
-   * Did they all make it through the entire growing season?
+   * Did they all make it through the entire growing season?