MC³ LabCUHK Psychology

Lesson 5 · Working with data

Pick rows & columns

Keep the columns and rows you need, and sort them, with dplyr.

In this lesson

  • Chain steps with the pipe %>%
  • select() columns, filter() rows
  • Sort with arrange()

You need

Datastudents.csvStudent Life Survey: 300 students, with their study habits, sleep, stress, wellbeing and exam results.
Scriptlesson-05.RAll the code in this lesson, ready to run.

Set up

R code
library(tidyverse)
students <- read_csv("data/students.csv", show_col_types = FALSE)

Columns or rows?

select(ID, ExamScore)
IDGenderAgeExamScore
S001Male2073
S002Male2175
S003Female1952
keeps columns
filter(Gender == "Female")
IDGenderAgeExamScore
S001Male2073
S002Male2175
S003Female1952
keeps rows

select(): columns

R code
students %>% select(ID, Gender, ExamScore)
Output
# A tibble: 300 × 3
   ID    Gender ExamScore
   <chr> <chr>      <dbl>
 1 S001  Male          73
 2 S002  Male          75
 3 S003  Female        52
 4 S004  Male          43
 5 S005  Female        88
 6 S006  Male          67
 7 S007  Female        83
 8 S008  Female        39
 9 S009  Male          72
10 S010  Female        70
# ℹ 290 more rows
  • select(-ID) drops a column.
  • select(Gender:Living) keeps a range of columns.
  • select(starts_with("Wellbeing")) keeps columns whose names start that way.

The pipe %>%

%>% means "and then". It passes the data to the next step.

students300 rows and then→ filter( ) and then→ select( ) → result

Shortcut: Ctrl+Shift+M (Mac: Cmd+Shift+M). You may also see |>, which does the same.

filter(): rows

R code
students %>%
  filter(Gender == "Female") %>%
  select(ID, Gender, ExamScore)
Output
# A tibble: 184 × 3
   ID    Gender ExamScore
   <chr> <chr>      <dbl>
 1 S003  Female        52
 2 S005  Female        88
 3 S007  Female        83
 4 S008  Female        39
 5 S010  Female        70
 6 S011  Female        66
 7 S015  Female        78
 8 S017  Female        51
 9 S019  Female        63
10 S020  Female        63
# ℹ 174 more rows

The first line of the output shows how many rows are left. More ways to filter:

R code
students %>% filter(Age >= 25, ExamScore > 80)
students %>% filter(Grade %in% c("A", "B")) %>% nrow()
Output
# A tibble: 5 × 16
  ID    Gender   Age Living StudyHours Attendance Motivation
  <chr> <chr>  <dbl> <chr>       <dbl>      <dbl> <chr>     
1 S111  Male      25 Hall           31         85 High      
2 S158  Female    25 Hall           20         78 Medium    
3 S165  Female    25 Hall           27         89 Medium    
4 S183  Female    27 Home           27         84 Medium    
5 S275  Female    27 Home           18         97 Medium    
# ℹ 9 more variables: LearningStyle <dbl>, Sleep <dbl>, Stress <dbl>,
#   ExamScore <dbl>, Grade <chr>, Workshop <chr>, Wellbeing_T1 <dbl>,
#   Wellbeing_T2 <dbl>, Used_Support <chr>
[1] 179
  • A comma means and: both must be true.
  • | means or.
  • %in% matches any value in a list. nrow() counts the rows.

arrange(): sort

R code
students %>%
  arrange(desc(ExamScore)) %>%
  select(ID, StudyHours, ExamScore) %>%
  head(5)
Output
# A tibble: 5 × 3
  ID    StudyHours ExamScore
  <chr>      <dbl>     <dbl>
1 S102          29       100
2 S111          31       100
3 S165          27       100
4 S218          24       100
5 S275          18       100

arrange() sorts small to large; desc() reverses it.

Exercises

Exercise 5.1 Dataset: students.csv

Choose columns

Show only ID, Sleep and Stress.

Show answerHide answer
R code
students %>% select(ID, Sleep, Stress)
Output
# A tibble: 300 × 3
   ID    Sleep Stress
   <chr> <dbl>  <dbl>
 1 S001    6        5
 2 S002    6        6
 3 S003   NA        8
 4 S004    5.8      5
 5 S005    5.5      7
 6 S006    6.5      6
 7 S007    6.2      7
 8 S008    6.9      3
 9 S009    6.6      6
10 S010    6.6      6
# ℹ 290 more rows
Exercise 5.2 Dataset: students.csv

Choose rows

  1. How many students live in a hall (Living == "Hall")?
  2. How many of them took the wellbeing workshop (Workshop == "Yes")?
Show answerHide answer
R code
students %>% filter(Living == "Hall") %>% nrow()
students %>% filter(Living == "Hall", Workshop == "Yes") %>% nrow()
Output
[1] 125
[1] 51
Exercise 5.3 Dataset: students.csv

Top sleepers

Show the 5 students who sleep the most, with their ID, Sleep and Stress. What do you notice about their stress?

Show answerHide answer
R code
students %>%
  arrange(desc(Sleep)) %>%
  select(ID, Sleep, Stress) %>%
  head(5)
Output
# A tibble: 5 × 3
  ID    Sleep Stress
  <chr> <dbl>  <dbl>
1 S144    9        3
2 S251    8.2      5
3 S217    8.1      2
4 S145    8        7
5 S264    8        4

Most of them report low or average stress. Lesson 12 tests whether sleep and stress really go together.

Key points

  • %>% = "and then".
  • select() keeps columns; filter() keeps rows; arrange() sorts.
  • Questions use ==. A comma in filter() means "and".