# How do Recipes actually work?

Unfortunately we had to cancel VizCon 2. So I decided to turn the slides I prepared on the recipe system of Plots.jl into a blog post.

Plots.jl is a plotting meta-package that provides a common interface to different Packages like GR.jl, Plotly, PyPlot.jl or PGFPlotsX.jl. But it's not just this. The recipe system of Plots and its processing pipeline allows users and package developers to define plotting routines for costum types without directly depending on Plots through the minimalst RecipesBase.jl and have them compose with types from other packages without knowning about each other. To get a better understanding of the underlying mechanism let's start with a quick overview of the Plots ecosystem.

## The Plots Ecosystem

### Dependencies

#### PlotUtils

PlotUtils.jl defines generic utils that could be used for other packages as well. It provides tools for: - Colors - Color gradients - Tick computation

#### RecipesBase

RecipesBase.jl is a lightweight Package without dependencies that allows to define custom plotting recipes with the @recipe macro. We will dive into more detail below.

#### PlotThemes

PlotThemes.jl provides different visual themes, defined as default attributes. For the plots in this post I chose theme(:bright).

### Plots

Plots.jl is at the core of the ecosystem. It defines the plotting functions plot and plot!, the argument processing pipeline and basic types like Plot, Subplot, Axis and Series. Furthermore, the code interfacing with the Plots backends lives here. GR and Plotly are installed by default and other backends are loaded conditionally using Requires.jl.

### Extensions

#### StatsPlots

StatsPlots.jl implements recipes for statistical plotting, like groupedbar, boxplot, violin, marginalhist, corrplot, etc. In addition it defines the handy @df macro providing Tables.jl support for plotting. However, it's currently being discussed if @df should be moved to Plots.

#### GraphRecipes

GraphRecipes.jl (formerly known as PlotRecipes) provides recipes for plotting graphs like the graphplot user recipe.

### Infrastructure

#### PlotReferenceImages

PlotReferenceImages.jl stores images that are used in the Plots test suite for visual comparison tests.

#### VisualRegressionTests

VisualRegressionTests.jl implements utils for comparing plots generated by Plots with reference images.

#### PlotDocs

The source code for the Plots documentation lives in PlotDocs.jl.

## What are Recipes?

Recipes are a way of defining visualizations without depending on Plots. The functionality relies on RecipesBase. This is a super lightweight package with zero dependencies and about 400 lines of code. It exports the @recipe macro which provides a nice syntax for defining plot recipes. Under the hood @recipe defines a new method for RecipesBase.apply_recipe which is called recursively in Plots at different stages of the argument processing pipeline. This way other packages can communicate with Plots, i.e. define custom plotting recipes, only depending on RecipesBase. Furthermore, the convenience macros @series, @userplot and @shorthands are exported by RecipesBase. I will explain their usage later.

### Recipes Syntax

The syntax in the @recipe macro is best explained using an example. Suppose, we have a custom type storing the results of a simulation x and y and a measure ε for the maximum error in y.

struct Result
x::Vector{Float64}
y::Vector{Float64}
ε::Vector{Float64}
end

If we want to plot the x and y values of such a result with an error band given by ε, we could run something like

res = Result(1:10, cumsum(rand(10)), cumsum(rand(10)) / 5)

using Plots

# plot the error band as invisible line with fillrange
plot(
res.x,
res.y .+ res.ε,
xlabel = "x",
ylabel = "y",
fill = (res.y .- res.ε, :lightgray, 0.5),
linecolor = nothing,
primary = false, # no legend entry
)

# add the data to the plots
plot!(res.x, res.y, marker = :diamond)

Instead of typing this plot command over and over for different results we can define a user recipe to tell Plots what to do with input of the type Result. Here is an example for such a user recipe with the additional feature to highlight datapoints with a maximal error above a certain threshold ε_max.

@recipe function f(r::Result; ε_max = 0.5)
# set a default value for an attribute with -->
xlabel --> "x"
ylabel --> "y"
markershape --> :diamond
# add a series for an error band
@series begin
# force an argument with :=
seriestype := :path
# ignore series in legend and color cycling
primary := false
linecolor := nothing
fillcolor := :lightgray
fillalpha := 0.5
fillrange := r.y .- r.ε
# ensure no markers are shown for the error band
markershape := :none
# return series data
r.x, r.y .+ r.ε
end
# get the seriescolor passed by the user
c = get(plotattributes, :seriescolor, :auto)
# highlight big errors, otherwise use the user-defined color
markercolor := ifelse.(r.ε .> ε_max, :red, c)
# return data
r.x, r.y
end

Let's walk through this recipe step by step. First, the function signature in the recipe definition determines the recipe type - in this case a user recipe. We will learn more about different recipe types soon. The function name f in is irrelevant and can be replaced by any other function name. @recipe does not use it. In the recipe body we can set default values for Plots attributes.

attr --> val
This will set attr to val unless it is specified otherwise by the user in the plot command.
plot(args...; kw..., attr = otherval)
Similarly we can force an attribute value with :=.
attr := val
This overwrites whatever the user passed to plot for attr and sets it to val.

We use the @series macro to add a new series for the error band to the plot. Within an @series block we can use the same syntax as above to force or set default values for attributes.

In @recipe we have access to plotattributes. This is a Dict storing the attributes that have been already processed at the current stage in the Plots pipeline. For user recipes, which are called early in the pipeline, this mostly contains the keyword arguments provided by the user in the plot command. In our example we want to highlight data points with an error above a certain threshold by changing the marker color. For all other data points we set the marker color to whatever is the default or has been provided as keyword argument.

Finally, in both, @recipes and @series blocks we return the data we wish to pass on to Plots (or the next recipe).

With the recipe above we can now plot Results with just

plot(res)

or

scatter(res, ε_max = 0.7, color = :green, marker = :star)

### Recipe Types

There are four main types of recipes which are determined by the signature.

#### User Recipes

User recipes are called early in the processing pipeline and allow designing custom visualizations.
@recipe function f(custom_arg_1::T, custom_arg_2::S, ...; ...)

#### Type Recipes

Type recipes define one-to-one mappings from custom types to something Plots supports
@recipe function f(::Type{T}, val::T) where T

#### Plot Recipes

Plot recipes are called after all input data is processed by type recipes but before the plot and subplots are set-up. They allow to build series with custom layouts and set plot-wide attributes.
@recipe function f(::Type{Val{:myplotrecipename}}, plt::AbstractPlot; ...)

#### Series Recipes

Series recipes are applied recursively until the current backend supports a series type. They are used for example to convert the input data of a bar plot to the coordinates of the shapes that define the bars.
@recipe function f(::Type{Val{:myseriesrecipename}}, x, y, z; ...)

### Examples

Let's look at some examples to get a better idea of the difference between these recipe types in practice.

#### User Recipes

We have already seen an example for a user recipe in the syntax section above. User recipes can also be used to define a custom visualization without necessarily wishing to plot a custom type. For this purpose we can create a type to dispatch on. The @userplot macro is a convenient way to do this.

@userplot MyPlot
expands to
mutable struct MyPlot
args
end
export myplot, myplot!
myplot(args...; kw...) = plot(MyPlot(args); kw...)
myplot!(args...; kw...) = plot!(MyPlot(args); kw...)
We can use this to define a user recipe for a pie plot.
# defines mutable struct UserPie and sets shorthands userpie and userpie!
@userplot UserPie
@recipe function f(up::UserPie)
y = up.args[end] # extract y from the args
# if we are passed two args, we use the first as labels
labels = length(up.args) == 2 ? up.args[1] : eachindex(y)
framestyle --> :none
aspect_ratio --> true
s = sum(y)
θ = 0
# add a shape for each piece of pie
for i in 1:length(y)
# determine the angle until we stop
θ_new = θ + 2π * y[i] / s
# calculate the coordinates
coords = [(0.0, 0.0); Plots.partialcircle(θ, θ_new, 50)]
@series begin
seriestype := :shape
label --> string(labels[i])
coords
end
θ = θ_new
end
# we already added all shapes in @series so we don't want to return a series
# here. (Technically we are returning an empty series which is not added to
# the legend.)
primary := false
()
end

Now we can just use the recipe like this:

userpie('A':'D', rand(4))

#### Type Recipes

Suppose we have a custom wrapper for vectors.

struct MyWrapper
v::Vector
end

We can tell Plots to just use the wrapped vector for plotting in a type recipe.

@recipe f(::Type{MyWrapper}, mw::MyWrapper) = mw.v

Now Plots knows what to do when it sees a MyWrapper.

mw = MyWrapper(cumsum(rand(10)))
plot(mw)

Due to the recursive application of type recipes they even compose automatically.

struct MyOtherWrapper
w
end

@recipe f(::Type{MyOtherWrapper}, mow::MyOtherWrapper) = mow.w

mow = MyOtherWrapper(mw)
plot(mow)

If we want an element-wise conversion of custom types we can use a less-known version of the type recipe. It requires defining a conversion function to a type that Plots supports (AbstractFloat, Integer) and a formatter for the tick labels. Consider the following simple time type.

struct MyTime
h::Int
m::Int
end

# show e.g. MyTime(1, 30) as "01:30"
time_string(mt) = join((lpad(string(c), 2, "0") for c in (mt.h, mt.m)), ":")
# map a MyTime object to the number of minutes that have passed since midnight.
# this is the actual data Plots will use.
minutes_since_midnight(mt) = 60 * mt.h + mt.m
# convert the minutes passed since midnight to a nice string showing MyTime
formatter(n) = time_string(MyTime(divrem(n, 60)...))

# define the recipe (it must return two functions)
@recipe f(::Type{MyTime}, mt::MyTime) = (minutes_since_midnight, formatter)

Now we can plot vectors of MyTime automatically with the correct tick labelling. DateTimes and Chars are implemented with such a type recipe in Plots for example.

times = MyTime.(0:23, rand(0:59, 24))
vals = log.(1:24)

plot(times, vals)

Again everything composes nicely.

plot(MyWrapper(vals), MyOtherWrapper(times))

#### Plot Recipes

Plot recipes define a new series type. They are applied after type recipes. Hence, standard Plots types can be assumed for input data :x, :y and :z in plotattributes. Plot recipes can access plot and subplot attributes before they are processed, for example to build layouts. Both, plot recipes and series recipes must change the series type. Otherwise we get a warning that we would run into a StackOverflow error.

We can define a seriestype :yscaleplot, that automatically shows data with a linear y scale in one subplot and with a logarithmic yscale in another one.

@recipe function f(::Type{Val{:yscaleplot}}, plt::AbstractPlot)
x, y = plotattributes[:x], plotattributes[:y]
layout := (1, 2)
for (i, scale) in enumerate((:linear, :log))
@series begin
title --> string(scale, " scale")
seriestype := :path
subplot := i
yscale := scale
end
end
end

We can call it with plot(...; ..., seriestype = :yscaleplot) or we can define a shorthand with the @shorthands macro.

@shorthands myseries

expands to

export myseries, myseries!
myseries(args...; kw...) = plot(args...; kw..., seriestype = :myseries)
myseries!(args...; kw...) = plot!(args...; kw..., seriestype = :myseries)

So let's try the yscaleplot plot recipe.

@shorthands yscaleplot

yscaleplot((1:10).^2)

Magically the composition with type recipes works again.

yscaleplot(MyWrapper(times), MyOtherWrapper((1:24).^2))

#### Series Recipes

If we want to call our userpie recipe with a custom type we run into errors.

userpie(MyWrapper(rand(4)))
ERROR: MethodError: no method matching keys(::MyWrapper)
Stacktrace:
[1] eachindex(::MyWrapper) at ./abstractarray.jl:209
Furthermore, if we want to show multiple pie charts in different subplots, we don't get what we expect either

userpie(rand(4, 2), layout = 2)

We could overcome these issues by implementing the required AbstractArray methods for MyWrapper (instead of the type recipe) and by more carefully dealing with different series in the userpie recipe. However, the simpler approach is writing the pie recipe as a series recipe and relying on Plots' processing pipeline.

@recipe function f(::Type{Val{:seriespie}}, x, y, z)
framestyle --> :none
aspect_ratio --> true
s = sum(y)
θ = 0
for i in eachindex(y)
θ_new = θ + 2π * y[i] / s
coords = [(0.0, 0.0); Plots.partialcircle(θ, θ_new, 50)]
@series begin
seriestype := :shape
label --> string(x[i])
x := first.(coords)
y := last.(coords)
end
θ = θ_new
end
end
@shorthands seriespie

Here we use the already processed values x and y to calculate the shape coordinates for each pie piece, update x and y with these coordinates and set the series type to :shape.

seriespie(rand(4))
seriespie(MyWrapper(rand(4)))
seriespie(rand(4, 2), layout = 2)

#### Remarks

Plot recipes and series recipes are actually very similar. In fact, a pie recipe could be also implemented as a plot recipe by acessing the data through plotattributes.

@recipe function f(::Type{Val{:plotpie}}, plt::AbstractPlot)
y = plotattributes[:y]
labels = plotattributes[:x]
framestyle --> :none
aspect_ratio --> true
s = sum(y)
θ = 0
for i in 1:length(y)
θ_new = θ + 2π * y[i] / s
coords = [(0.0, 0.0); Plots.partialcircle(θ, θ_new, 50)]
@series begin
seriestype := :shape
label --> string(labels[i])
x := first.(coords)
y := last.(coords)
end
θ = θ_new
end
end
@shorthands plotpie

plotpie(rand(4, 2), layout = (1, 2))

The series recipe syntax is just a little nicer in this case. Here's subtle difference between these recipe types: Plot recipes are applied in any case while series are only applied if the backend does not support the series type natively.

Let's try it the other way around and implement our yscaleplot recipe as a series recipe.

@recipe function f(::Type{Val{:yscaleseries}}, x, y, z)
layout := (1, 2)
for (i, scale) in enumerate((:linear, :log))
@series begin
title --> string(scale, " scale")
seriestype := :path
subplot := i
yscale := scale
end
end
end
@shorthands yscaleseries

Huh, that looks nicer than the plot recipe version as well. Let's try to plot.

yscaleseries((1:10).^2)
MethodError: Cannot convert an object of type Int64 to an object of type Plots.Subplot{Plots.GRBackend}
Closest candidates are:
convert(::Type{T}, !Matched::T) where T at essentials.jl:168
Plots.Subplot{Plots.GRBackend}(::Any, !Matched::Any, !Matched::Any, !Matched::Any, !Matched::Any, !Matched::Any, !Matched::Any, !Matched::Any) where T<:RecipesBase.AbstractBackend at /home/daniel/.julia/packages/Plots/rNwM4/src/types.jl:88

That is because the plot and subplots have already been built before the series recipe is applied. So, for everything that modifies plot-wide attributes we should stick to plot recipes, otherwise I recommend series recipes.

## Internals

That was fun - a lot of colorful plots! But now let's dig into the source code to understand what's actually going on under the hood.

### RecipesBase

The @recipe macro defines a new method for RecipesBase.apply_recipe.

@recipe function f(args...; kwargs...)
defines Vector{RecipeData} where

RecipesBase.apply_recipe(plotattributes, args...; kwargs...)
returning a Vector{RecipeData} where RecipeData holds the plotattributes Dict and the arguments returned in @recipe or in @series.
struct RecipeData
plotattributes::AbstractDict{Symbol,Any}
args::Tuple
end
This function sets and overwrites entries in plotattributes and possibly adds new series.

• attr --> val translates to haskey(plotattributes, :attr) || plotattributes[:attr] = val

• attr := val sets plotattributes[:attr] = val.

• @series allows to add new series within @recipe. It copies plotattributes from @recipe, applies the replacements in defined in its code block and returns corresponding new RecipeData object.

So RecipesBase.apply_recipe(plotattributes, args...; kwargs...) returns a Vector{RecipeData}. Plots can then recursively apply it again on the plotattributes and args of the elements of this vector, dispatching on a different signature.

### Plots

The standard plotting commands

plot(args...; plotattributes...)
plot!(args...; plotattributes...)
and shorthands like scatter or bar call the core plotting function
Plots._plot!(plt::Plot, plotattributes::AbstractDict{Symbol, Any}, args::Tuple)
in src/plot.jl.

In the following we will go through the major steps of the preprocessing pipeline implemented in Plots._plot!.

#### Preprocess plotattributes

Before Plots._plot! is called and after each recipe is applied, preprocessArgs! in src/args.jl preprocesses the plotattributes Dict. It replaces aliases, expands magic arguments, and converts some attribute types.

• lc = nothing is replaced by linecolor = RGBA(0, 0, 0, 0).

• marker = (:red, :circle, 8) expands to markercolor = :red, markershape = :circle and markersize = 8.

#### Process User Recipes

In the first step, _process_userrecipe in src/pipeline.jl is called.

kw_list = _process_userrecipes(plt, plotattributes, args)
_process_userrecipe roughly looks like this:

still_to_process = RecipeData[]
args = _preprocess_args(plotattributes, args, still_to_process)
kw_list = Dict{Symbol, Any}[]

while !isempty(still_to_process)
next_series = popfirst!(still_to_process)
if isempty(next_series.args)
# finish up and add to kw_list
else
rd_list = RecipesBase.apply_recipe(next_series.plotattributes, next_series.args...)
prepend!(still_to_process,rd_list)
end
end

It recursively applies RecipesBase.apply_recipe on the fields of the first element of the RecipeData vector still_to_process and prepends the resulting RecipeData vector to it. If the args of an element are empty, it finishes up and adds plotattributes to kw_list. When all RecipeData elements are fully processed the kw_list is returned.

#### Process Type Recipes

After user recipes are processed, at some point in the recursion above args is of the form (y, ), (x, y) or (x, y, z). In src/series.jl recipes for these signatures are defined. The two argument version, for example, looks like this.

@recipe function f(x, y)
did_replace = false
newx = _apply_type_recipe(plotattributes, x)
x === newx || (did_replace = true)
newy = _apply_type_recipe(plotattributes, y)
y === newy || (did_replace = true)
if did_replace
newx, newy
else
SliceIt, x, y, nothing
end
end

It recursively calls _apply_type_recipe on each argument until none of the arguments is replaced. It applies the type recipe with the corresponding signature and for vectors it tries to apply the recipe element-wise.

_apply_type_recipe(plotattributes, v) = RecipesBase.apply_recipe(plotattributes, typeof(v), v)[1].args[1]

# Handle type recipes when the recipe is defined on the elements.
# This sort of recipe should return a pair of functions... one to convert to number,
# and one to format tick values.
function _apply_type_recipe(plotattributes, v::AbstractArray)
isempty(skipmissing(v)) && return Float64[]
x = first(skipmissing(v))
args = RecipesBase.apply_recipe(plotattributes, typeof(x), x)[1].args
if length(args) == 2 && typeof(args[1]) <: Function && typeof(args[2]) <: Function
numfunc, formatter = args
Formatted(map(numfunc, v), formatter)
else
v
end
end

# don't do anything for ints or floats
_apply_type_recipe(plotattributes, v::AbstractArray{T}) where {T<:Union{Integer,AbstractFloat}} = v

When no argument is changed by _apply_type_recipe, the fallback SliceIt recipe is applied, which adds the data to plotattributes and returns RecipeData with empty args.

#### Process Plot Recipes

At this stage all arguments have been processed to something Plots supports. In _plot! we have a Vector{Dict} kw_list with an entry for each series and already populated :x, :y and :z keys. Now _process_plotrecipe in src/pipeline.jl is called until all plot recipes are processed.

still_to_process = kw_list
kw_list = KW[]
while !isempty(still_to_process)
next_kw = popfirst!(still_to_process)
_process_plotrecipe(plt, next_kw, kw_list, still_to_process)
end

If no series type is set in the Dict, _process_plotrecipe pushes it to kw_list and returns. Otherwise it tries to call RecipesBase.apply_recipe with the plot recipe signature. If there is a method for this signature and the seriestype has changed by applying the recipe, the new plotattributes are append to still_to_process. If there is no method for the current plot recipe signature, we append the current Dict to kw_list and rely on series recipe processing.

function _process_plotrecipe(plt::Plot, kw::AKW, kw_list::Vector{KW}, still_to_process::Vector{KW})
if !isa(get(kw, :seriestype, nothing), Symbol)
# seriestype was never set, or it's not a Symbol, so it can't be a plot recipe
push!(kw_list, kw)
return
end
try
st = kw[:seriestype]
st = kw[:seriestype] = get(_typeAliases, st, st)
datalist = RecipesBase.apply_recipe(kw, Val{st}, plt)
for data in datalist
preprocessArgs!(data.plotattributes)
if data.plotattributes[:seriestype] == st
error("Plot recipe $st returned the same seriestype:$(data.plotattributes)")
end
push!(still_to_process, data.plotattributes)
end
catch err
if isa(err, MethodError)
push!(kw_list, kw)
else
rethrow()
end
end
return
end

After all plot recipes have been applied, the plot and subplots are set-up.

_plot_setup(plt, plotattributes, kw_list)
_subplot_setup(plt, plotattributes, kw_list)

#### Process Series Recipes

We are almost finished. Now the series defaults are populated and _process_seriesrecipe in src/pipeline.jl is called for each series .

for kw in kw_list
# merge defaults
series_attr = Attr(kw, _series_defaults)
_process_seriesrecipe(plt, series_attr)
end

If the series type is natively supported by the backend, we finalize processing and pass the series along to the backend. Otherwise, the series recipe for the current series type is applied and _process_seriesrecipe is called again for the plotattributes in each returned RecipeData object. Here we have to check again that the series type changed to not run into a StackOverflow error.

function _process_seriesrecipe(plt, plotattributes)
st = plotattributes[:seriestype]
if is_seriestype_supported(st)
# if it's natively supported, finalize processing and pass along to the backend, otherwise recurse
else
# get a sub list of series for this seriestype
datalist = RecipesBase.apply_recipe(
plotattributes, Val{st}, plotattributes[:x], plotattributes[:y], plotattributes[:z]
)

# assuming there was no error, recursively apply the series recipes
for data in datalist
preprocessArgs!(data.plotattributes)
if data.plotattributes[:seriestype] == st
error("The seriestype didn't change in series recipe \$st.  This will cause a StackOverflow.")
end
_process_seriesrecipe(plt, data.plotattributes)
end
end
end

Due to this recursive processing, complex series types can be built up by simple blocks. For example if we add an @show st in _process_seriesrecipe and plot a histogram, we go through the following series types:

plot(histogram(randn(1000)))
st = :histogram
st = :barhist
st = :barbins
st = :bar
st = :shape

## Concluding Remarks

Recipes provide a nice composable system in the spirit of Julia's multiple dispatch design for defining visualizations for custom types without taking a dependency on a Plotting package. However, the system is tightly intertwined with the Plots processing pipeline and relies on the recursive application of recipes on plot attributes passed through as a Dict{Symbol, Any}. Furthermore, there are different types of recipes called at different stages during the Plots processing pipeline. In addition, there are some basic recipes defined in Plots, like the default slicer SliceIt, that are essential for the recursive application of recipes to work as expected.

For recipes to interact properly with other plotting packages (like Makie.jl) these packages probably have to port the processing pipeline and fallback recipes in some form. Another option could be to move parts of the fundamental processing pipeline, like _process_userrecipe, _process_plotrecipe, _process_seriesrecipe and _apply_type_recipe, and fallback recipes to RecipesBase and let other plotting packages use these. I am not sure if this is feasible and if other packages have to support key Plots design principles (like columns are series) for this to work. In any case, I am afraid porting the Plots recipe system to other plotting packages will not be a trivial task. However, I am really looking forward to hear ideas of others during our remote VizCon meetings.

## Acknowledgements

The recipe system and the processing pipeline was designed by Tom Breloff (@tbreloff). You can find a video of him explaining the Plots Ecosystem and the Recipe System at http://www.breloff.com/plots-video/.