updated file names

Merge branch 'traits_tutorials' of github.com:terraref/tutorials into traits_tutorials

# Conflicts:
#	_bookdown.yml

Author: kimberlyh66

.github/ISSUE_TEMPLATE/default_template.md

@@ -0,0 +1,20 @@
+*Fill out this issue by completing the items below. Not every item may be relevent to you, so feel free to make changes*
+
+#### The tutorial or vignette name
+Name:
+
+#### The current behavior or issue
+Description:
+
+#### The steps taken to reproduce the behavior or issue, or specify a location where the steps were recorded
+To reproduce:
+
+#### Describe the desired behavior or desired outcome
+Goal:
+
+#### Add other supporting information that may be useful
+Versions, OS information, other:
+
+#### Completion criteria
+This is done when:
+- [] (replace this text)

.github/ISSUE_TEMPLATE/new_feature_template.md

@@ -0,0 +1,12 @@
+#### Describe what you want to do
+
+(be as descriptive as you can)
+
+#### Let us know what your motivation for this new feature is
+
+I want this new feature because ...
+
+#### _(Optional) Please share any thoughts on how you might want this feature implemented_
+
+**Note:** we really appreciate your suggestions
+I think the best approach would be to ...

README.md

@@ -1,26 +1,40 @@
 
 # Tutorials
 
+
 ## An introduction to the use of TERRA REF data and software
 
-Many of these tutorials have complex dependencies. In addition, many datasets are very large. These tutorials use the National Data Service Labs Workbench (NDS Labs Workbench). The NDS Labs Workbench provides the following:
+This repository provides a set of tutorials that are divided by data types and use cases. 
+
+In the repository, you will find three folders that contain examples of how to access data:
+
+* traits
+* sensors
+* plantCV
+
+Within each folder there are both R markdown and Jupyter notebooks. These describe different approaches to accessing data. These are intended to cover diverse use cases, and you will find information about accessing data from web interfaces but the primary focus is on accessing data using R, Python, SQL, and REST APIs. These are intended to provide quick-start introductions to access data along with computing environments required for further exploration. They are not intended to teach analyses, although some illustrative visualizations and statistical models are provided.
+
+This is a work in progress, and an open source community that welcomes contributions in many forms. Please feel welcome to ask questions, provide suggestions or share analyses that may be of interest to others.
+
+
+## Getting Started
+
+### Requirements
 
-* Custom configured computing environments  
-  * In principle, it is possible to use any docker container on [Docker Hub](https://hub.docker.com/explore/).
-* To get started, see the [workbench/ndslabs_workbench_intro.html](https://htmlpreview.github.io/?https://github.com/terraref/tutorials/blob/master/workbench/ndslabs_workbench_intro.html) tutorial.
-* This will set you up to begin tutorials found in the traits, sensors, and plantCV folders.
+All of the tutorials have been designed to work in the cloud and can be accessed using a web browser. Therefore, the _only technical requirements_ are:
+* Web browser
+* Internet connection
 
-### Data Access
+In addition, you will need to:
+* Sign up as as a TERRA REF [Beta User by filling out this application](http://terraref.org/beta).
+* Sign up for an account on the [TERRA REF Workbench](https://www.workbench.terraref.org), and wait for approval.
 
-The first research-grade version of TERRA REF data products will be released in November 2019. 
-Before that, we will make evaluation releases available: the alpha version was released in November 2016 and the beta version will be released in 2018.
+### Using the Workbench
 
-To access data please fill out a quick [Beta User application](http://terraref.org/beta).
+The Workbench uses the National Data Service Labs Workbench (NDS Labs Workbench) software. The Workbench is a cloud analysis environment that minimizes and standardizes the software requirements so users can get started quickly. It also provides access to large files and databases that would otherwise be slow and impractical for users to download and store. 
 
-### Links
+Although we provide a few pre-configured computing environments, Workbench is designed to support any Docker container - that is, anything you can install on a linux computer can be used in the workbench - for an idea of what is possible, see the large number of options on [Docker Hub](https://hub.docker.com/explore/).
 
-TODO: add links to quick-start documentation, README's, code for learning and applied examples
+**To get started**, follow the [Workbench Quick Start](https://htmlpreview.github.io/?https://github.com/terraref/tutorials/blob/master/workbench/ndslabs_workbench_intro.html).
 
-* Data portal: [terraref.org/data](https://terraref.org/data)
-* Docker Images on Docker Hub: [hub.docker.com/terraref](https://hub.docker.com/terraref)
-* NDS Labs Workbench: [www.workbench.terraref.org](https://www.workbench.terraref.org)
+This will walk you through the process of getting started with the first tutorials on how to access data.

_bookdown.yml

@@ -4,5 +4,6 @@ language:
   ui:
     chapter_name: "Chapter "
 rmd_files: ["index.Rmd", "vignettes/00-introduction.Rmd", "vignettes/01-get-trait-data-R.Rmd", "vignettes/02-get-weather-data-R.Rmd",
-"vignettes/03-get-images-python.Rmd", "vignettes/04-synthesis-data.Rmd", "traits/03-access-r-traits.Rmd",
-"sensors/01-meteorological-data.Rmd", "sensors/06-list-datasets-by-plot.Rmd"]
+"vignettes/03-get-images-python.Rmd", "vignettes/04-synthesis-data.Rmd", "traits/03-access-r-traits.Rmd","sensors/01-meteorological-data.Rmd",
+"sensors/06-list-datasets-by-plot.Rmd"]
+

index.Rmd

@@ -42,6 +42,7 @@ At a minimum, you should have:
 We have tried to write these tutorials using open access sample data sets. However, access to much of the data will require you to 1) fill out the TERRA REF Beta user questionaire ([terraref.org/beta](terraref.org/beta)) and 2) request access to specific databases.
 
 <!-- Not sure where this goes, either in documentation or perhaps in an appendix. But I don't think this belongs in the introduction. Perhaps after the vignettes chaper 
+
 ## Ways of Acessing Data
 
 * Web Interfaces
@@ -55,14 +56,13 @@ We have tried to write these tutorials using open access sample data sets. Howev
 * API Clients
     + [rOpenSci traits package](https://pecan.gitbook.io/betydb-data-access/ropensci-traits-package)
 
--->
-
 ## Other Resources
 
 The TERRA REF website: [terraref.org](http://terraref.org/)
 
 The TERRA REF Technical Documentation: [docs.terraref.org](docs.terraref.org)
 
+
 ## Finding help
 
 - Slack at terra-ref.slack.com ([signup](https://terraref-slack-invite.herokuapp.com/))
@@ -73,6 +73,7 @@ The TERRA REF Technical Documentation: [docs.terraref.org](docs.terraref.org)
 
 ```{r, include = FALSE}
 knitr::opts_chunk$set(echo = FALSE)
+
 options(warn = -1)
 ```
 

traits/02-betydb-api-access.Rmd

@@ -26,6 +26,7 @@ The first step toward reproducible pipelines is to automate the process of searc
 ### Using Your API key to Connect
 
 An API key is like a password. It allows you to access data, and should be kept private. 
+
 Therefore, we are not going to put it in code that we share. 
 
 A common way of handling private API keys is to place it in a text file in your current directory. Don't put it in a project directory where it might be inadvertently shared.
@@ -39,6 +40,7 @@ Here is how to find and save your API key:
 An API key is not needed to access public data. This includes metadata tables and simulated data in the https://terraref.ncsa.illinois.edu/bety-test database.
 
 
+
 ## Accessing data using a URL query
 
 
@@ -48,8 +50,10 @@ An API key is not needed to access public data. This includes metadata tables an
 * path to the api: `/api/v1`
 * api endpoint: `/search` or `traits` or `sites`. For BETYdb, these are the names of database tables. 
 * Query parameters: `genus=Sorghum`
+
 * Authentication: `key=api_key` is your assigned API key. This will only be needed when querying trait data. No key is needed to access the public metadata tables.
 
+
 ### Constructing a URL query
 
 First, lets construct a query by putting together a URL.
@@ -72,6 +76,7 @@ First, lets construct a query by putting together a URL.
 
 > What will the URL https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum return?
 
+
 > Write a URL that will query the database for sites with "Field Scanner" in the name field. Hint: combine two terms with a `+` as in `Field+Scanner`
 
 What do you see? Do you think that this is all of the records? What happens if you add `&limit=none`? 
@@ -94,6 +99,7 @@ curl -o sorghum.json \
 
 ## Using the R jsonlite package to access the API with a URL query 
 
+
 ```{r 02-jsonlite-load, include = FALSE}
 
 library(jsonlite)
@@ -104,7 +110,7 @@ library(jsonlite)
 sorghum.json <- readLines(
   paste0("https://terraref.ncsa.illinois.edu/bety/api/v1/species?genus=Sorghum&key=", 
          readLines('.betykey')))
-
+         
 ## print(sorghum.json) 
 ## not a particularly useful format
 ## lets convert to a data frame

traits/03-access-r-traits.Rmd

@@ -27,9 +27,9 @@ library(ggthemes)
 theme_set(theme_bw())
 library(dplyr)
 ```
-
 Create a file that contains your API key. If you have signed up for access to the TERRA REF database, your API key will have been sent to you in an email. You will need this personal key _and_ permissions to access the trait data. If you receive empty (NULL) datasets, it is likely that you do not have permissions. 
 
+
 ```{r writing-key, echo = TRUE}
 # This should be done once with the key sent to you in your email
 
@@ -63,7 +63,6 @@ sorghum_info <- betydb_query(table = 'species',
 
 Notice all of the arguments that the `betydb_query` function requires? We can change this by setting the default connection options thus:
 
-
 ```{r 03-set-up, echo = TRUE}
 options(betydb_key = readLines('.betykey', warn = FALSE),
         betydb_url = "https://terraref.ncsa.illinois.edu/bety/",
@@ -73,6 +72,7 @@ options(betydb_key = readLines('.betykey', warn = FALSE),
 Now the same query can be reduced to:
 
 ```{r query-species-reduce, message = FALSE, echo = TRUE}
+
 sorghum_info <- betydb_query(table = 'species',
                              genus = "Sorghum",
                              limit = 'none')
@@ -81,6 +81,7 @@ sorghum_info <- betydb_query(table = 'species',
 ### Time series of height
 
 Now let's query some trait data.
+
 ```{r canopy_height, echo = TRUE, message = FALSE}
 canopy_height <- betydb_query(table     = 'search',
                                trait     = "canopy_height",
@@ -89,12 +90,14 @@ canopy_height <- betydb_query(table     = 'search',
 ```
 
 ```{r plot_height}
+
 ggplot(data = canopy_height,
        aes(x = lubridate::yday(lubridate::ymd_hms(raw_date)), y = mean)) +
   geom_point(size = 0.5, position = position_jitter(width = 0.1)) +
 # scale_x_datetime(date_breaks = '6 months') +
   xlab("Day of Year") + ylab("Plant Height") +
   guides(color = guide_legend(title = 'Genotype')) +
   theme_bw()
+  
 ```
 

traits/04-danforth-indoor-phenotyping-facility.Rmd

@@ -1,7 +1,8 @@
 # Danforth Indoor Phenotype Analysis
 
 ```{r 02-setup, include=FALSE}
-knitr::opts_chunk$set(echo = TRUE, cache = FALSE)
+
+knitr::opts_chunk$set(echo = TRUE, cache = TRUE)
 library(jsonlite)
 library(dplyr)
 library(ggplot2)
@@ -21,6 +22,7 @@ library(traits)
 Unlike the first two tutorials, now we will be querying real data from the public TERRA REF database. So we will use a new URL, https://terraref.ncsa.illinois.edu/bety/, and we will need to use our own private key.
 
 ```{r terraref-connect-options}
+
 options(betydb_key = readLines('.betykey', warn = FALSE),
         betydb_url = "https://terraref.ncsa.illinois.edu/bety/",
         betydb_api_version = 'v1')

traits/05-maricopa-field-scanner.Rmd

@@ -31,6 +31,7 @@ sites <- betydb_query(
 
 A more robust (but complicated way) would be to query the experiments and experiments_sites tables. But we will leave that as an exercise for the ambitious user.
 
+
 ### Plot Season 2 plots
 
 ```{r traits-05-map-mac-polygons, echo = TRUE}
@@ -39,6 +40,7 @@ site_bounds <- sites %>%
   rowwise() %>%
   do(boundaries = readWKT(text = .$geometry, id = .$id))
 
+
 site_bounds <- do.call('rbind', site_bounds$boundaries)
 #names(site_bounds) <- sites$sitename
 
@@ -78,7 +80,9 @@ leaflet() %>%
 ```{r}
 ## Time series of canopy cover, height, NDVI
 
-First look up variables by name. Let's look for measurements related to canopy size:
+#First look up variables by name. Let's look for measurements related to canopy size:
+
+```
 
 ```
 

traits/06-agronomic-metadata.Rmd

@@ -90,6 +90,7 @@ options(betydb_key = readLines('.betykey', warn = FALSE),
 
 
 ```{r 06_tibble, echo = TRUE, warning = FALSE}
+
 ## query and join tables
 species <- betydb_query(table = "species") %>%
   select(specie_id = id, scientificname, genus)
@@ -133,6 +134,7 @@ Let's do the manual equivalent of a cross-table join. BETY actually does contain
 The key idea here is that each treatment is associated with some (possibly many) managements, but the treatments table only reports the number of associated managements. To see the management IDs themselves, we need to query an individual treatment ID. So, we retrieve one table, then iterate over each row extracting the foreign keys for the other table. This requires an API call for every treatment, so beware that it is likely to be slow!
 
 ```{r 06_cross_join, echo = TRUE, results = 'hide'}
+
 treatments <- betydb_query(table = 'treatments') %>%
   select(treatment_id = id , name, definition, control)