A Flow-Insensitive, Modular Effect System for Purity


FTfJP 2013, Montpellier

Lukas Rytz, Nada Amin and Martin Odersky
EPFL, Switzerland

\( \def\resloc{\,\circ\,} \def\eff{~!~} \def\loc{\mathit{loc}} \def\tlet{\text{let }} \def\tin{\text{ in }} \def\mlocal{\text{[local] }} \def\any{\text{any}} \def\seq#1{\overline{#1}} \def\ba{\begin{array}} \def\ea{\end{array}} \def\ts{\vdash} \def\pure{\emptyset} \def\fresh{\emptyset} \newcommand{\colr}[2]{\colorbox{#1}{$#2$}} \def\state#1{\left<#1\right>} \def\halt{\text{halt}} \def\evalto{\longrightarrow} \definecolor{yellow-back}{RGB}{255,255,102} \)
## Use Cases for a Purity System * Verified documentation for programmers * Flexibility in scheduling execution * [DPJ](http://dpj.cs.uiuc.edu/DPJ/Home.html) tracks noninterference of expressions and executes them in parallel * Enables optimizations * In Scala, when inlining a method defined in an object, the object initializer only needs to be invoked if it is impure
## Verifying Purity * Methods that modify only local state (local variables, iterators, etc.) are pure * Challenge: aliasing makes tracking state difficult * Existing techniques: * State monad * Program verification (separation logic) * Ownership types * Effect systems based on pointer analysis * Region-based effects
## Our Requirements * Modular effect system that integrates with our generic effect checking framework * Lightweight annotations, easy to use and understand * Identify pure expressions within larger, potentially impure programs * Applicable to existing code, does not enforce a programming discipline or require refactorings
## Built on Ideas from JPure * **Modular**, intra-procedural effect analysis * Unlike JPure, our system is **flow-insensitive**, therefore easier to integrate into higher-order languages * **Lightweight** effect annotations * Our system can express nested functions with effects on outer parameters * **Ownership** annotations for non-shared state, **freshness** for methods returning newly allocated objects * To express getters, we introduce *conditional* freshness

Definition of Purity

A pure function does not modify any state
that existed before its execution.


  • Goal: show soundness of the purity effect system
  • Proofs are still work in progress
## Effects and Freshness * `@mod(a)` permits modifying the state of parameter `a` * `@loc()` for methods returning freshly allocated objects ```scala trait Iterator[+T] { def hasNext: Boolean @mod() def next(): T @mod(this) } trait List[+T] { def iterator: Iterator[T] @loc() } def contains(l: List[Int], e: Int): Boolean @mod() = { val it = l.iterator while (it.hasNext) if (it.next() == e) return true return false } ```
## Ownership and Locality * `@local` fields are owned by the object * An object is only fresh if all objects in its *locality* are fresh * `@mod(a)` permits modifying any state in the locality of `a` ```scala class ArrayBuffer[A] { @local private[this] var array = new Array[A](initSize) private[this] var size = 0 def +=(elem: A): Unit @mod(this) = { ensureSize(size + 1) array(size) = elem size += 1 } } ```
## Higher-Order Code * Parameter `a` in `@mod(a)` can belong to an outer method * A method with `@loc(t)` is fresh if `t` is fresh, e.g. getters for local fields ```scala trait A { def getOwnedB(): B @loc(this) } def f(a: A): Unit @mod(a) = { def nested(): Unit @mod(a) = a.getOwnedB.modify() nested() } ```
## Limitations in Practice * Caching: cache updates are transparent to the user, but not to the type system * Collections of fresh objects: type arguments with locality not (yet) supported ```scala def t(): Int @mod() = { val o: Option[Counter /* @loc() */] = Some(new Counter) o match { case Some(c) => c.inc(); c.get() case None => 0 } } ```

Formal Language

A lambda calculus with mutable records

\(\ba{lrl} t & ::= & \tlet x = p \tin t \mid x.l := y \mid x \\ p & ::= & (x: T) \rightarrow t \mid x~y \mid \{\seq{\mlocal l = x}\} \mid x.l \mid t \\ T & ::= & (x: T) \xrightarrow{e}_{\loc} T \mid \{\seq{\mlocal l: T}\} \\ e & ::= & \seq{x} \mid \any \\ \loc & ::= & \seq{x} \mid \any \ea\) \(\ba{l} \\[6pt] \Gamma ::= x: T \resloc \loc \qquad \boxed{\Gamma \ts p : T \resloc \loc \eff e}\\[4pt] \ea\)

Example with a Local Counter

\(\boxed{ \tlet c = \{x = 1\} \tin \\ \tlet \mathit{inc} = (\_: \{\}) \rightarrow (\tlet v = c.x \tin c.x := (v+1)) \tin~ \\ \tlet \_ = inc~\{\} \tin \\ \tlet r = c.x \tin r }\)


  • Function \(\mathit{inc}\) has an effect on \(c\) from its environment
  • Object \(c\) is freshly allocated, effects on it can be masked

Typing the \(\mathit{inc}\) Function

\(\boxed{ \tlet c = \{x = 1\} \tin \\ \tlet \mathit{inc} = (\_: \{\}) \rightarrow \colr{yellow-back}{\tlet v = c.x \tin c.x := (v+1)} }\)

Typing derivation for the body of \(\mathit{inc}\):



\[\ba{l} \frac{ \Gamma_c \ts c.x : \text{Int} \resloc \any \eff \colr{RedOrange}{\pure} \quad \frac{\displaystyle \Gamma_c,\Gamma_v \ts v+1 : \text{Int} \resloc \any \eff \pure }{\displaystyle \Gamma_c,\Gamma_v \ts c.x := (v+1) : \{x:\text{Int}\} \resloc \colr{SkyBlue}{c} \eff \colr{RedOrange}{c} } }{ \Gamma_c \ts \colr{yellow-back}{\tlet v = c.x \tin c.x := (v+1)}: \{x:\text{Int}\} \resloc \colr{SkyBlue}{c} \eff (\colr{RedOrange}{\pure} \sqcup \colr{RedOrange}{c}) } \\[12pt] \qquad \ba{rl} \text{where } & \Gamma_c = c: \{x:\text{Int}\} \resloc c, \\ & \Gamma_v = v:\text{Int} \resloc \any \ea \ea\]

Effects on Counter are Masked

\(\boxed{ \tlet c = \{x = 1\} \tin \\ \tlet \mathit{inc} = (\_: \{\}) \rightarrow (\tlet v = c.x \tin c.x := (v+1)) \tin \\ \tlet \_ = inc~\{\} \tin \quad \tlet r = c.x \tin \quad r }\)

Typing the let binding for \(c\):



\[\ba{l} \frac{ \emptyset \ts \{x = 1\}: \{x:\text{Int}\} \resloc \colr{SkyBlue}{\fresh} \eff \colr{RedOrange}{\pure} \quad \Gamma_c \ts \mathit{body} : \text{Int} \resloc \any \eff \colr{RedOrange}{c} }{ \emptyset \ts \tlet \colr{YellowGreen}{c} = \{x = 1\} \tin \mathit{body} : \text{Int} \resloc \any \eff \underbrace{[\colr{SkyBlue}{\fresh} / \colr{YellowGreen}{c}](\colr{RedOrange}{\pure} \sqcup \colr{RedOrange}{c})}_\emptyset } \\[12pt] \qquad \text{where } \Gamma_c = \{x:\text{Int}\} \resloc c \ea\]

Dynamic Semantics

  • Machine states \(\state{H,V,K,p}\) consist of
    • a heap \(~H ::= \emptyset \mid H, r \mapsto \{\mlocal l = v\}\)
    • a stack \(~V ::= \emptyset \mid V, x \mapsto v\)
    • a continuation \(~K ::= \halt \mid [x;V;K;p]\)
    • an expression \(p\)
  • Values: references or closures \(~v ::= r \mid [(x: T) \rightarrow t;V]\)
  • Stack & continuations: no bare references in terms, no duplication of function arguments through substitution

Type Safety

  • Progress: \(~\state{H,V,K,p} \evalto \state{H',V',K',p'}~\) goes from well-formed to well-formed
  • Preservation: if \(p\) is pure, then so is \(p'\)
  • Certain variables need to be fresh in the typing environment:
    • \(\tlet c = \{x=1\} \tin c.x := 2~~\) evaluates to \(~~c.s := 2\)
    • Preservation requires purity, so \(~c: \{x:\text{Int}\} \resloc \colr{yellow-back}{\fresh} \in \Gamma\)
  • The locality of fresh variables in \(\,\Gamma\) is ensured to be separate from all other state

Soundness

  • Purity: if \(p\) types as pure, then its evaluation does not modify existing state — however it can modify fresh state
  • \[\ba{l} \boxed{\tlet c = \{x=1\} ~~\tin~~ c.x := 2} \\[6pt] \state{\emptyset; \emptyset; \halt; \tlet c = \ldots} \evalto \\ \state{\emptyset; \emptyset; [c;\emptyset;\halt;c.x := 2];\{x=1\}} \evalto \\ \state{r_c \mapsto \colr{YellowGreen}{\{x=1\}}; c \mapsto r_c; \halt; \colr{RedOrange}{c.x := 2}} \evalto \\ \state{r_c \mapsto \colr{YellowGreen}{\{x=2\}}; c \mapsto r_c; \halt; c} \ea\]

  • By preservation \(~~c.x := 2~~\) types as pure, but \(c\) is modified

Implementation

Live demo at the end of the workshop

Screenshot Eclipse

# Thank you! # ❧ ### Q & A

Effect Typing for Scala


Tool Demo of the Effects Plugin for Scala


FTfJP 2013, Montpellier

Lukas Rytz, EPFL, Switzerland

## Big Picture * A [compiler plugin](https://github.com/lrytz/efftp) for Scala 2.10.1 (`scalac`, REPL, Eclipse) * Entirely based on annotations, no changes to Scala's syntax * Effect-annotated code compiles without the plugin (it's a [pluggable type system](http://bracha.org/pluggableTypesPosition.pdf)) * Currently supports I/O effects, checked exceptions and purity (state effects) * Documentation on the [Github Wiki](https://github.com/lrytz/efftp/wiki)
## Syntax * Effects are annotated on return types of methods ```scala def inc(i: Int): Int @pure = i + 1 ``` * Effects are inferred if the return type is inferred ```scala def inc(i: Int) = i + 1 // effect is inferred ``` * Methods with a return type, but without effect annotations have an unknown effect ```scala def inc(I: Int): Int = i + 1 // impure! ```
## Multiple Effect Domains * Multiple annotations for multiple domains ```scala def div(a: Int, b: Int): Int @noIo @mod() @throws[DivByZero] = ... ``` * By default, non-annotated domains are impure * The `@pure` annotation applies to all effect domains and marks them pure by default ```scala def div(a: Int, b: Int): Int @pure @throws[DivByZero] = ... ```
## Effect Polymorphism * The effect of a higher-order method usually depends on the effect of its argument: ```scala def h(f: Int => Int): Int @pure(f) = { f(1) } ``` * The _relative effect annotation_ `@pure(f)` marks `h` as pure, up to the effect of `f` (more precisely: `f.apply`) * Relative effects work with arbitrary types (not just functions) ```scala def sayHi(g: Greeter, name: String): Unit @pure(g.greet(%)) = { g.greet(name) } ```
# Demo!