Piping in R and the consequences of lazy-eval

With this post, I'd like to explore some interesting and potentially unexpected behavior observed when piping functions in R. It builds on this tweet by @MeganBeckett.

Problem

To better understand some implications of lazy evaluation, we are going to work through the following toy problem:
Let's say we want three functions, first(), second(), third(), which we expect to be able to pipe together, and print "first", "second" and "third" to the console, respectively...

... or in pseudo-code:

first() %>% second() %>% third()
#> "first"
#> "second"
#> "third"

Failed attempts

Here I'm going to show a couple of attempts at achieving the desired functionality, using some functions that might, at first glance, seem like they should work.

1. Functions with no return() calls

The tidyverse style-guide, suggests to "only use return() for early returns...". Following these guidelines, we can write a couple of functions that achieve the desired behavior when used individually.

However, notice what happens when we pipe them together:

library(magrittr)

first <- function(){
  print("First")
}

second <- function(x){
  print("Second")
}

third <- function(x){
  print("Third")
}

first() %>% 
  second() %>% 
  third()
#> [1] "Third"

First of all, we see that the second two functions in the dplyr chain require the unused argument x, to allow for piping. We also see that only the last function, third() actually prints its output to the console.

2. Empty return() calls

Maybe adding an empty return() call to each function will help:

library(magrittr)

first <- function(){
  print("First")
  return()
}

second <- function(x){
  print("Second")
  return()
}

third <- function(x){
  print("Third")
  return()
}

first() %>% 
  second() %>% 
  third()
#> [1] "Third"
#> NULL

Not a whole lot has changed compared to the previous example, except that now the final third() function prints both "Third", and NULL to the console.

3. Return the function's input argument: return(.)

Finally, let's try returning the unaltered input argument of each function in the chain.

Nothing will change for the first() function in the chain, since it takes no input, to begin with, but the other two functions gain a x in their final return() call:

library(magrittr)

first <- function(){
  print("First")
  return()
}

second <- function(x){
  print("Second")
  return(x)
}

third <- function(x){
  print("Third")
  return(x)
}

first() %>% 
  second() %>% 
  third()
#> [1] "Third"
#> [1] "Second"
#> [1] "First"
#> NULL

This is where we see some interesting behavior. We are now getting all three print() calls to output on the console, but the order seems strange. We get the print() outputs of the third(), second() and first() functions in that order, and then finally we get this NULL coming from the first return() call in first() (this is something we can just remove).

Explanation

To understand what is going on here, we need to understand the concept of lazy evaluation, and, more broadly, the concept of evaluation. I will be paraphrasing a lot from this great post by @ColinFay

Evaluation

... is the process of analyzing an expression in order to give the user something back. We send the console a symbol, R interprets the symbol, and returns the associated value:

# R evaluates the string and returns the value
"hello world"
#> [1] "hello world"

# `a` is a symbol
a <- "hello world"

# when `a` is evaluated, it returns the value (a string)
a
#> [1] "hello world"

Lazy-Evaluation

R employs the "lazy" evaluation strategy. What this means is that symbols are only evaluated if the expression is actually used. This means that we can write a function like this:

unused_arguments <- function(a, b){
  print("hello world")
}
unused_arguments()
#> [1] "hello world"

unused_argument <- function(a, b){
  print(a)
}

unused_argument("hello")
#> [1] "hello"

Without lazy evaluation, these functions would fail, since R would try to evaluate a and b, and realize that these symbols are undefined. Luckily, with lazy evaluation, R is "smart enough" to realize that these unused arguments are never called, and thus don't need to be evaluated.

What does lazy evaluation mean then for our above examples?

Effectively, %>% passes the unevaluated left-hand side as an argument to the right-hand side, so the dplyr chain we are working with, can be thought of as: third(second(first())) or perhaps even more clearly
third(x = second(first())).

Keeping this in mind, and also remembering that R will only evaluate symbols if it has to, let's look again at our examples:

1. Functions with no return() calls

library(magrittr)

first <- function(){
  print("First")
}

second <- function(x){
  print("Second")
}

third <- function(x){
  print("Third")
}

first() %>% 
  second() %>% 
  third()
#> [1] "Third"

Using our above interpretation may help clear up what is going on:

third(x = second(first())) 

Since third() never explicitly evaluates it's only argument, x, the calls to second() and first() are never even evaluated. Thus we are effectively just calling print("Third").

2. Empty return() calls

library(magrittr)

first <- function(){
  print("First")
  return()
}

second <- function(x){
  print("Second")
  return()
}

third <- function(x){
  print("Third")
  return()
}

first() %>% 
  second() %>% 
  third()
#> [1] "Third"
#> NULL

It's basically the same story here. The added calls to return() basically do nothing, with the exception of that extra return() call in the third() function, which returns NULL.

3. Return the function's input argument: return(x)

library(magrittr)

first <- function(){
  print("First")
  return()
}

second <- function(x){
  print("Second")
  return(x)
}

third <- function(x){
  print("Third")
  return(x)
}

first() %>% 
  second() %>% 
  third()
#> [1] "Third"
#> [1] "Second"
#> [1] "First"
#> NULL

Here, the argument x is explicitly being called in each return() call, thus it will need to be evaluated.
Thinking of the chain as third(x = second(first())), and thus the inner function call as second(x = first()) we can interpret what is happening:

• third() gets evaluated up until it requires x, thus the print("Third") line gets evaluated first, followed by evaluation of the second() function
• second() is then evaluated, again, up until it requires x: printing "Second" and evaluating first()
• first() is evaluated, printing "First", and then executing return() which outputs NULL

Solution

Finally, we can see how we might easily achieve our solution:

library(magrittr)

first <- function(){
  print("First")
}

second <- function(x){
  x # forces evaluation of the argument
  print("Second")
}

third <- function(x){
  x # forces evaluation of the argument
  print("Third")
}

first() %>% 
  second() %>% 
  third()
#> [1] "First"
#> [1] "Second"
#> [1] "Third"

The return() calls aren't even necessary! What's important is that we understand lazy evaluation, and how to "trigger" evaluation of a function argument. Since we precede each call to print() with an explicit call to the function argument x, it forces R to evaluate the x. In this case, understanding the pipe as third(second(first())), we see:

• third() runs until it must evaluate second(), which occurs at the first line
• second() runs until it must evaluate first()
• first() prints "First"
• second() prints "Second"
• third() prints "Third"

and we are happy :-D

Forcing argument evaluation with `force()`

In the final solution here, we forced evaluation of the argument `x` by simply calling it at the first line. A more "canonical" way to force symbol evaluation is by using the `force(x)` function, which effectively does the same thing, but makes it obvious that evaluation is desired (and not a mistake in the code).

Start by reviewing the proposed post in this issue and checking its R and magrittr examples for correctness and clarity. Done means turning the explanation of lazy evaluation and piping into a polished resource article, including the demonstrated solution.