Live Programming by Example
Using Direct Manipulation for Live Program Synthesis
Christopher Schuster, Cormac Flanagan
University of California, Santa Cruz

Gulfs of Execution and Evaluation

  • Donald A Norman. Cognitive engineering. 1986.

  • Live Programming
  • Programming by Example

Live Programming by Example

Difficult for generic imperative applications!

Proposed Solution

  • A domain-specific programming environment for programming GUI applications
  • Enforce functional rendering

Model-View-Update Applications

var i = 23;

function view() {
  return (<div>
    <p>Count: {i}</p>
    <button onclick={inc}>

function inc() {
  • Model holds entire application state
  • View generates visible output based on Model
  • Update processes events and changes the Model

Model-View-Update Applications

Model-View-Update Applications

Functional Rendering: Related Work

Many other approaches and implementations

  • Elm, Flux, Redux, Cycle.js

Functional Rendering for live programming

  • Burckhardt, Fahndrich, de Halleux, McDirmid, Moskal, Tillmann and Kato. It’s Alive! Continuous Feedback in UI Programming. PLDI ’13.

Live Programming

Live code updates to stateful applications with continous feedback

  • Changing the program code at runtime
  • Keep application state
  • Continous feedback
    • Re-evaluate expressions, re-render output, etc.

Live Programming

Live Programming

Programming by Example

Input-output examples for synthesizing
human-readable program code

  • Source code remains canonical representation of program

Live Program Synthesis of View function

view : Model -> Output
  • Current model at runtime used as input
  • User creates output example by direct manipulation of GUI
  • Repair current view according to input-output example

Live Programming by Example

Live Programming by Example

Program Synthesis: Related Work

Changes to constants

Inverting basic operators, gradient descent

  • Galenson, Reames, Bodik, Hartmann, Sen. Programmatic and Direct Manipulation, Together at Last. PLDI ’16.

Larger program changes

Generate-and-test, constraint solving

  • Gulwani. Automating String Processing in Spreadsheets Using Input-Output Examples. POPL ’11.
  • Chugh, Hempel, Spradlin and Albers. CodeHint: Dynamic and Interactive Synthesis of Code Snippets. ICSE ’14.

Formal Semantics

  • Formal specification of design space
  • Purity of View function enforced either statically or dynamically

Implementation (

Dynamically enforce function purity

  • Global variables rewritten to become fields of state object
  • Mutation of state object during view causes runtime error due to deep cloning and Object.freeze

String origin tracking

  • String literals tagged with origin information
  • Primitive operators rewritten for instrumentation
  • Tagged strings wrapped in Proxy to mimic native strings
  • Wang, Zhang, Xie, Xiong and Mei.. Automating Presentation Changes in Dynamic Web Applications via Collaborative Hybrid Analysis. FSE ’12.

Future Work

Direct Manipulation

  • Dragging, Dropping, Re-ordering, Resizing, ...
  • See professional UI Builders and IDEs

Live Program Synthesis

  • Extend synthesis beyond simple constants
  • Synthesize Update actions based on before-after UI examples

Thank you!

Appendix: Back-in-time Debugging

Appendix: Back-in-time Debugging