EDS 240 - Course logistics, Intro, {ggplot2} review

The {ggplot2} hex sticker which features a connected scatterplot with data points in varying shades of blue atop a gray gridded background. The word 'ggplot2' sits beneath the data.

Pre-class Prep

Please be sure to complete the following before class:

Install required packages

install.packages("palmerpenguins") # a package containing the `penguins` data set, which we'll use for plotting practice
install.packages("tidyverse") # a collection of packages used for data wrangling / manipulation and visualization (including {ggplot2})

Create your `EDS-240-class-examples` repository

We’ll be coding together quite a bit throughout this course. To stay organized, we’ll complete all of our in-class examples in one repository (repo). Create and clone a GitHub repository named EDS-240-class-examples. For step-by-step instructions, unfold the following note (collapsed to save space):

Step-by-step instructions for creating and cloning a GitHub repo:

Create a remote (GitHub) repo:

Navigate to your GitHub profile and click on Repositories (top menu bar)
Click on the green New button in the upper right-hand corner
Give it the name EDS-240-class-examples, and optionally, a short description. Select the radio button for Public repository (it should be selected by default). Initialize this repository with a README by checking the box and Add a .gitignore (select R from the .gitignore template drop down menu) boxes. Click Create repository.
From your repo’s main page, click the green Code button and copy the URL to your clipboard

Clone your repo:

Open up RStudio
Click on the drop down menu next to the R project icon (top right corner)
Click New Project… > Version Control > Git > paste your repo’s URL in the Repository URL: box and press Tab to auto fill the Project directory name field (it should automatically be named the same as your GitHub repo). Browse to select where on your computer you’d like your repo to be saved.
Click Create Project

Lecture Materials

Week 1 instruction is broken down into three lessons:

Course logistics & syllabus

lecture 1.1 slides

Intro to data visualization

lecture 1.2 slides

{ggplot2} review

lecture 1.3 slides

Discussion Materials

Pre-discussion Prep

Before coming to section, you’ll need to install some packages and download data. For step-by-step instructions, unfold the following note (collapsed to save space):

Step-by-step pre-discussion prep instructions

Install the following packages:

install.packages("janitor") # tools for cleaning dirty data
install.packages("usdata") # demographic data on the United States at the county and state levels spanning multiple years.

Download the data (fracking.csv) from Google Drive and save it to your class repo in the following location: EDS-240-class-examples/week1/data/fracking.csv (NOTE: data were originally downloaded as a zip file from FracFocus in November 2023. Since then, FracFocus has published a cleaner version of these data, which defeats some of this purpose of this exercise. As such, we’ll be working with this saved version of the data, rather than downloading directly from FracFocus’s data download page.)
We don’t want to push any data in our repo to GitHub (GitHub enforces file size limits, and you’ll run into some serious headaches when you try to push a file that’s too big) – to avoid doing this, we can add our data files to our .gitignore file, which, as the name implies, is a collection of files that we want Git to ignore. Open up your .gitignore file and add the following line (anywhere is fine, but it makes sense to add it beneath the last line):

.gitignore

# ignore any folder named `data/` within folders named `week__` (* acts as a wildcard)
/week*/data/

Notice that when you Save your modified .gitignore file, untracked data files at the specified file path disappear from your Git tab in RStudio, and a modified .gitignore file appears in it’s place (this is what we want!). Stage/add, commit, and push .gitignore.

An example repo with three folders: week1/, week2/, and other-stuff/. Each of those folders has a subfolder named data/, and each data/ folder contains a .csv file. Adding the line, /week*/data/ to the .gitignore tells Git to ignore folders (and their contents) named data/ that are within a parent folder named week__ (the * acts as a wildcard). If we wanted to ignore all three data/ folders (including other-stuff/data/), we can instead include /*/data/ to our .gitignore.

Background

By now, you may have heard / read something like, “Data scientists spend 80% of their time preparing their data for analysis and / or visualization.” And while that may not be totally accurate for all data scientists or all projects, you will spend lots of time wrestling with data. You’ll spend this week’s discussion cleaning up a messy data set on hydraulic fracturing (aka fracking), with the goal of (re)familiarizing yourselves with some of commonly-used tidyverse functions.

This week’s data comes courtesy of Jeremy Singer-Vine’s Data is Plural weekly newsletter of useful / curious data sets (the 2023.09.27 edition). Singer-Vine’s description:

Since launching in 2011, FracFocus has become the largest registry of hydraulic fracturing chemical disclosures in the US. The database, available to explore online and download in bulk, contains 210,000+ such disclosures from fracking operators; it details the location, timing, and water volume of each fracking job, plus the names and amounts of chemicals used. The project is managed by the Ground Water Protection Council, “a nonprofit 501(c)6 organization whose members consist of state ground water regulatory agencies”. As seen in: The latest installment of the New York Times’ Uncharted Water series.

Interested in reading more about fracking? Check out this communications piece from USGS to start.

Week 1 discussion slides

Solution

You’ll get the most out of discussion section if you physically type out the code yourself (rather than copying / pasting)!

Note: Some of the wrangling in the solution (collapsed, below) may seem a bit superfluous – we’ve included lots of steps to (re)introduce as many of the common wrangling functions as possible, given our data set.

Complete code for week 1 discussion (fracking)

##~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##                                    setup                                 ----
##~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#..........................load packages.........................
library(tidyverse)
library(janitor)
library(usdata)

#......................import fracking data......................
fracking <- read_csv(here::here("week1", "data", "fracking.csv"))

##~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##                        clean/wrangle fracking data                       ----
##~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

fracking_clean <- fracking |> 
  
  # clean column names ----
  janitor::clean_names() |> 
  
  # clean up dates ----
  mutate(job_start_date = str_remove(job_start_date, " AM")) |> # remove 'AM' from string
  mutate(datetime_start = mdy_hms(job_start_date)) |> # convert from string to date (save to new col)
  mutate(year = year(datetime_start)) |> # create 'year' col from date

  # select relevant cols ----
  select(datetime_start, year, state_name, county_name, well_name, total_base_water_volume) |> 
  
  # filter out non-state names ----
  filter(!state_name %in% c("Beaver", "Beckham", "Harper", "Hemphill", "Midland", "Red River", "Roosevelt", "Rusk", "State", "WARD")) |> 
  
  # rename state_name to something shorter for typing out when using case_when (not necessary) ----
  rename(sn = state_name) |> 
  
  # make all words title case ----
  mutate(sn = str_to_title(sn)) |> 
  
  # fix misspelled state names ----
  mutate(sn = case_when(
    sn == "Colordao" ~ "Colorado",
    sn == "Loiusiana" ~ "Louisiana",
    sn == "Louisianna" ~ "Louisiana",
    sn == "Lousiana" ~ "Louisiana",
    sn == "New Mexcio" ~ "New Mexico",
    sn == "Norh Dakota" ~ "North Dakota",
    sn == "Norht Dakota" ~ "North Dakota",
    sn == "North  Dakota" ~ "North Dakota",
    sn == "North Dakata" ~ "North Dakota",
    sn == "North Dakotta" ~ "North Dakota",
    sn == "Noth Dakota" ~ "North Dakota",
    sn == "Pennslvania" ~ "Pennsylvania",
    sn == "Pennsylavania" ~ "Pennsylvania",
    sn == "Pennsylvanya" ~ "Pennsylvania",
    sn == "Penssylvania" ~ "Pennsylvania",
    sn == "Texasa" ~ "Texas",
    sn == "Texs" ~ "Texas", 
    sn == "West Viginia" ~ "West Virginia",
    sn == "Wyominng" ~ "Wyoming", 
    TRUE ~ sn # copy over rest of state names from as-is
  )) |> 
  
  # remove rows that have a '?' mark ----
  filter(!str_detect(string = sn, pattern = "\\?")) |> # `?` is a special chr; escape with `\\` prefix
  
  # make all uppercase (so that we can covert abbreviation to state names) ----
  mutate(sn = str_to_upper(sn)) |> 
  
  # mutate abbreviations to full state names ----
  mutate(sn = ifelse(test = str_length(sn) == 2, # if string in 'sn' col is 2 chrs long
                     yes = usdata::abbr2state(sn), # replace abbreviation with full state name 
                     no = sn)) |> # if string in 'sn' col is not 2 chrs long, keep state name as-is
  
  # make all words title case again ----
  mutate(sn = str_to_title(sn)) |> 
  
  # create a column of just state abbreviations ----
  mutate(state_abb = usdata::state2abbr(sn)) |> 
  
  # rename 'sn' to 'state_name' again for clarity ----
  rename(state_name = sn, total_base_water_volume_gal = total_base_water_volume) |> 
  
  # move 'state_abb' col after state_name col ----
  relocate(state_abb, .after = state_name) |> 
  
  # convert 'state_name' & 'state_abb' from string to factor ----
  mutate(state_name = as.factor(state_name),
         state_abb = as.factor(state_abb)) |> 
  
  # remove obs that don't have a measurement for 'total_base_water_volume' (NA) ----
  drop_na(total_base_water_volume_gal)

##~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
##          some exploratory data viz + a few plot mods for practice        ----
##~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

fracking_clean |> 
  filter(state_name %in% c("Texas", "Colorado", "North Dakota")) |> 
  filter(year == 2015) |> 
  group_by(state_name) |> # pipe directly into ggplot
  ggplot(aes(x = fct_rev(fct_infreq(state_name)), y = total_base_water_volume_gal)) + # need to reverse fct order for coord_flip() (plots lowest freq at top by default)
  geom_jitter(width = 0.3, alpha = 0.5, color = "gray15") +
  geom_violin(color = "red4", alpha = 0.3) +
  scale_y_continuous(labels = scales::label_comma()) +
  labs(y = "Total base water volumn (gal)") +
  coord_flip() +
  theme_minimal() +
  theme(
    axis.title.y = element_blank()
    )

Assignment Reminders

Assignment Type	Assignment Title	Date Assigned	Date Due
EOC	End-of-class survey (week 1)	Mon 01/08/2024	Mon 01/08/2024, 11:55pm PT
SR	Pre-course reflection (SR #1)	Mon 01/08/2024	Sat 01/13/2024, 11:59pm PT
HW	Homework Assignment #1	Mon 01/08/2024	Sat 01/20/2024, 11:59pm PT