Data provided at NASS Quick Stats is very useful in understanding the history and state of agricultural production at aggregate levels (county and state). Also, they have been used widely in various studies, including the estimation of climate change impacts on crop yield. In this blog post, I will show how to download NASS Quick Stats data from within R using the rnassqs package.
While their website is useful, they allows you to download \(50,000\) records (rows of data) at a time. This can be quite annoying when your target dataset is much larger, which means you have to do manual queries (lots of clicking) many many times. Now, rnassqs makes use of the Quick Stats API which puts the same limitation of the maximum of \(50,000\) records at a time. This of course means that rnassqs is also bound by the limitation. However, since it’s an R program, you can easily write a loop in a way that each download involves less than \(50,000\) records, obviating the need to manually implementing queries on the website repeatedly. But, before we get started, I need to mention that you can visit their ftp website to bulk-download the entire data available at the website. So, if you do not mind a large data sitting on your computer, you should stop reading and just go ahead and download the entire dataset.
At the time of writing this post, the package is not available on the CRANs. So, here is how you install the package: 1) load the devtools package first (install it first if you have not) and 2) install the rnassqs package as follows:
library(devtools)
install_github('potterzot/rnassqs')Let’s load all the libraries we use in this blog post:
#--- rnassqs package ---#
library(rnassqs)
#--- other packages ---#
library(magrittr) # for code piping
library(data.table)## No encoding supplied: defaulting to UTF-8.We first set the query parameters. Here I would like to have a data of corn yield (bu/acre) at the county level in 1995.
#--- set the query parameters ---#
params <- list(
'year__GE'=1995,
'agg_level_desc'='COUNTY',
'short_desc'='CORN, GRAIN - YIELD, MEASURED IN BU / ACRE'
)We then can use the nassqs() function to download the data we want. But, before doing that you need to obtain the API key from here.
#--- download the data ---#
corn_all <- nassqs(params=params, key='your key comes here') %>%
data.table()Here is a look at some of the variables of the dataset we just downloaded.
corn_all %>%
setnames(names(.),tolower(names(.))) %>%
.[,.(year,state_name,state_alpha,state_fips_code,county_name,agg_level_desc,short_desc,value,unit_desc)]## year state_name state_alpha state_fips_code
## 1: 2017 ALABAMA AL 01
## 2: 2016 ALABAMA AL 01
## 3: 2015 ALABAMA AL 01
## 4: 2014 ALABAMA AL 01
## 5: 2013 ALABAMA AL 01
## ---
## 45177: 1999 WYOMING WY 56
## 45178: 1998 WYOMING WY 56
## 45179: 1997 WYOMING WY 56
## 45180: 1996 WYOMING WY 56
## 45181: 1995 WYOMING WY 56
## county_name county_code agg_level_desc
## 1: COLBERT 033 COUNTY
## 2: COLBERT 033 COUNTY
## 3: COLBERT 033 COUNTY
## 4: COLBERT 033 COUNTY
## 5: COLBERT 033 COUNTY
## ---
## 45177: OTHER (COMBINED) COUNTIES 998 COUNTY
## 45178: OTHER (COMBINED) COUNTIES 998 COUNTY
## 45179: OTHER (COMBINED) COUNTIES 998 COUNTY
## 45180: OTHER (COMBINED) COUNTIES 998 COUNTY
## 45181: OTHER (COMBINED) COUNTIES 998 COUNTY
## short_desc value unit_desc
## 1: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 195.3 BU / ACRE
## 2: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 126.2 BU / ACRE
## 3: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 159.1 BU / ACRE
## 4: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 191.5 BU / ACRE
## 5: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 182.2 BU / ACRE
## ---
## 45177: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 0 BU / ACRE
## 45178: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 90 BU / ACRE
## 45179: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 110 BU / ACRE
## 45180: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 94 BU / ACRE
## 45181: CORN, GRAIN - YIELD, MEASURED IN BU / ACRE 95 BU / ACRESimple loop to get more years of corn data or other crops
It is very simple to get more years of county-level corn data by looping.
get_nass_data <- function(y){
params <- list(
'year__GE'=y,
'agg_level_desc'='COUNTY',
'short_desc'='CORN, GRAIN - YIELD, MEASURED IN BU / ACRE'
)
corn_all <- nassqs(params=params, key='your key comes here') %>%
data.table()
return(corn_all)
}
corn_data <- lapply(1995:2015,get_nass_data) %>%
rbindlist()It should be clear by now how you can loop over crop types if you want other crops than corn.
Session Information
## R version 3.5.1 (2018-07-02)
## Platform: x86_64-apple-darwin15.6.0 (64-bit)
## Running under: macOS 10.14.1
##
## Matrix products: default
## BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib
##
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] data.table_1.11.8 magrittr_1.5 rnassqs_0.2.2
##
## loaded via a namespace (and not attached):
## [1] Rcpp_1.0.0 bookdown_0.7 digest_0.6.18 rprojroot_1.3-2
## [5] R6_2.3.0 jsonlite_1.5 backports_1.1.2 evaluate_0.12
## [9] httr_1.3.1 blogdown_0.9 stringi_1.2.4 rmarkdown_1.10
## [13] tools_3.5.1 stringr_1.3.1 xfun_0.4 yaml_2.2.0
## [17] compiler_3.5.1 htmltools_0.3.6 knitr_1.20