my face
paulgray.net

Pipeable APIs

An introduction to pipeable APIs in fp-ts

June 03, 2021
javascripttypescriptfunctional programmingfp-ts

Pipeable APIs provide flexibility for adding new behavior and creating new functions for working with data types. With the pipe function from fp-ts, you can construct usages with good inference.

Data flow

The pipe function takes a value as it's first parameter, and then takes a list of functions which are sequentially called on that value. Each function receives the data that was returned from the previous step, and the last value is returned from the whole expression.

pipe(
  5,
  (n: number) => {
    // n is 5
    return n + 5
  },
  (n: number) => {
    // n is 10
    return n * 2
  }
) // final result is 20

The type of the value in the chain doesn't necessarily stay the same type, for example:

// Uses min/max to ensure n is between min and max
const clamp = (n: number, min: number, max: number) => ... 

pipe(
  "foo",
  (s: string) => {
    // s is "foo"
    return s + "bar"
  },
  (s: string) => {
    // s is "foobar"
    return s.length
  },
  (n: number) => {
    // n is 6
    return n * 2
  },
  (n: number) => {
    // n is 12
    return clamp(n, 0, 10)
  }
) // result is 10

This could be cleaned up a bit without the explanatory comments and annotations:

pipe(
  "foo",
  s => s + "bar",
  s => s.length,
  n => n * 2,
  n => clamp(n, 0, 10)
)

Specializing Functions

Let's play around with the functions here and provide names for some of the common operations:

const concat = (s1: string, s2: string) => s1 + s2
const len = (a: Array<unknown>) => a.length
const add = (a: number, b: number) => a + b
const multiply = (a: number, b: number) => a + b

And use those in the pipe chain:

pipe(
  "foo",
  s => concat(s, "bar"),
  s => len(s),
  n => multiply(n, 2),
  n => clamp(n, 0, 10)
)

This doesn't really improve on the previous example at all, but let's try to rearrange the parameters a bit, and curry the functions:

const concat = (s2: string) => (s1: string) => s1 + s2
const add = (a: number) => (b: number) => a + b
const multiply = (a: number) => (b: number) => a + b
const clamp = (min: number, max: number) => (n: number) => ...

Note the reversal of the s1/s2 parameters, and the arrangement of the parameters of clamp. This may seem odd at first, but it's a deliberate decision to make application in a pipe easier.

Using the new curried versions of each function we arrive at:

pipe(
  "foo",
  s => concat("bar")(s),
  s => len(s),
  n => multiply(2)(n),
  n => clamp(0, 10)(n)
)

An interesting pattern has emerged... Every anonymous function we've used in the pipe takes the value passed to it as the last argument in its own argument list. In JavaScript, you can replace the expression s => <expression>(s) with <expression>. So, for example, you can replace:

s => concat("bar")(s) with concat("bar"),

s => len(s) with len,

n => multiply(2)(n) with multiply(2), and

n => clamp(0, 10)(n) with clamp(0, 10).

Let's make those replacements in our snippet:

pipe(
  "foo",
  addStr("bar"),
  len,
  multiply(2),
  clamp(0, 10)
)

What we arrive at is a syntax that's very close to method chaining, but has a few additional benefits in extensibility.

In fp-ts

The fp-ts library heavily utilizes this pattern, as most functions are fashioned specifically to be used with pipe. Let's take a look at this snippet which uses some standard JS array methods:

const myNumbers = [1,2,3]

myNumbers
  .map(n => n + 1)
  .filter(n => n > 2)

Now, compare this to using fp-ts's pipe and Array module:

import { A } from 'fp-ts/Array'
import { pipe } from 'fp-ts/function'

pipe(
  myNumbers,
  A.map(n => n + 1),
  A.filter(n => n > 2)
)

The myNumbers array is passed through to the A.map function, and then the returned value is passed to the A.filter function. The value returned from A.filter is the final result returned from the pipe call. This snippet resembles it's chainable counterpart, and it's due to how the functions were designed.

Implementing the pipeable pattern

Implementing a pipeable API is just a matter of arranging the parameters in the right order. Take a look at these function signatures from fp-ts's Array module:

filter<A>(f: (a:A) => boolean) => (as: Array<A>) => Array<A>
map<A, B>(f: (a: A) => B) => (as: Array<A>) => Array<B>

Notice anything in common? Both functions take the value they're operating on last, in a parameter list by itself.

This was done specifically to make it easier to use in the context of a pipe chain. Here, filter and map can be used in a pipe the same way they'd have been used in a method chain.

Extensibility

Since the only requirement for a function to be used in a pipe chain is that the last parameter list be solely the value it's operating on, it's trivial to add custom functions that can be interspersed with other functions. For example, we can define a function that works on arrays by making the last parameter list take an array:

const partition = (f: (a: A) => boolean) => (as: Array<A>): Array<Either<A, B>> => ...

Since the last parameter is an array, we can inject this function anywhere inside of a pipe chain with an array:

pipe(
  myNumbers,
  A.map(n => n + 1),
  partition(n => n > 2)
)

It's almost as if we extended the Array type to include the partition function! This would be quite difficult with a chainable-style API.

Inference

When the subject of your function is generic, and the types of the other parameters depend on the type of the subject, using the function in a point free way helps with inference.

Consider the following usage which, while correct, doesn't type check:

const myNumbers = [1, 2, 3]

A.map(n => n + 1)(myNumbers)

The type of the parameter n depends on the type of the array we pass it in the second argument list (ns). In order to fix this usage, you'd have to explicitly note the type parameter in the anonymous function:

A.map((n: number) => n + 1)(ns)

This is because Typescript's inference works left to right, so in order to resolve the type of the expression A.map(n => n + 1)(ns) it must first resolve the type of the expression A.map(n => n + 1). Unfortunately, it doesn't have enough information in that context to be able to know that you will eventually pass an array of numbers (even though it's immediately after!).

Now consider using this with pipe:

pipe(
  myNumbers,
  A.map(n => n + 1)
)

Typescript will resolve the type of ns (which is Array<number>), before resolving the type of A.map(n => n + 1). The type of pipe signifies that the second parameter be a function that takes the first parameter: pipe(myNumbers, <function that takes Array<number>>) with this knowledge, the compiler knows that the anonymous function we supply to map will take a number, and so Typescript will infer it for us.

If instead we tried:

pipe(
  myNumbers,
  a => A.map(n => n + 1)(a)
)

We'd be back to square one, and we'd need to explicitly annotate the type of n.

Discoverability

A common lament of pipeable APIs is that they aren't as "discoverable" as chainable APIs. This refers to the use of an autocomplete feature in IDEs, where you can type an expression, like: "myNumbers." and your IDE will provide you a list of methods that are available on arrays.

Since a pipeable API uses free-standing functions as opposed to methods, this isn't as straightforward. A technique that's often used is to import the free-standing functions into a short identifier namespace, for example:

import * as A from 'fp-ts/Array'

Now, when you type the expression "A." your IDE's autocomplete will provide you with a list of those free-standing functions that operate on arrays.

Of course, this isn't as nice as method chaining, but the tradeoff of extensibility is perhaps worth it.

Pipe vs flow

flow is another function provided by fp-ts which is similar to pipe, but is used in fully point-free implementations. Instead of taking a value and applying functions to that value successively, flow takes a list of functions and composes them together, returning a new function which will invoke the functions passed to flow. flow is essentially like compose, except backwards, applying the leftmost function first. For example, let's suppose we have three functions, f, g, and h, and we wish to apply them in order, like:

h(g(f(1)))

First, f will be applied to the argument 1, then g will be applied to the return of f, and h applied to the return of g. flow can take these functions and build another function, combining them all:

const f_then_g_then_h = flow(f, g, h);

f_then_g_then_h(1)

As a general rule, whenever you see an expression like: s => pipe(s, ...), it can logically be replaced with flow(...). However, this replacement can't always practically be done due to the way Typescript's inference works. Like with pipe, using functions with a type argument is when it starts to fall over:

flow(
  A.filter(n => n > 1),
  A.map(n => n + 1)
)

This expression produces a logically correct function that filters an array of numbers, and then adds one to each, but it doesn't typecheck, since there's no such array in sight. The type of the expression A.filter(n => n > 1) is checked first, and the type parameter corresponding to the type of values in the array defaults to unknown.

Where flow does work is in cases where Typescript already has the type information it needs. If we assign this expression to a variable with the type annotated, Typescript can suddenly resolve the type:

const modNumbers: (as: Array<number>) => Array<number> =
  flow(
    A.filter(n => n > 1),
    A.map(n => n + 1)
  )

The future

The pipeline operator proposal is currently at stage 1 (which unfortunately means that it won't be added to typescript). If accepted, pipeable APIs benefit from getting some dedicated syntax, whre we can replace our original snippet:

pipe(
  "foo",
  addStr("bar"),
  len,
  multiply(2),
  clamp(0, 10)
)

with:

"foo"
  |> addStr("bar")
  |> len
  |> multiply(2)
  |> clamp(0, 10)