ScalarFlow: A Python Automatic Differentiation Library¶

ScalarFlow is a automatic differentiation library that borrows heavily from the TensorFlow 1.0 API. The emphasis is on simplicity, not performance.

Building and Running Computation Graphs¶

The ScalarFlow library makes it possible to build and execute computation graphs involving scalar quantities. For example:

import scalarflow as sf

with sf.Graph() as g:
    a = sf.Constant(7.0)
    b = sf.Constant(3.0)
    sum = sf.Add(a, b)

    result = g.run(sum)
    print(result) # Prints 10.0

Notice that in the example above, the nodes are added to the graph g, even though it is not provided as an explicit argument when constructing nodes in the computation graph. The scalarflow library maintains a default computation graph that can be set using the with keyword as above.

It is also possible to use the default computation computation graph outside of any context. For example:

a = sf.Constant(7.0)
b = sf.Constant(3.0)
sum = sf.Add(a, b)

result = sf.get_current_graph.run(sum)
print(result) # Prints 10.0

Node names¶

All nodes in the computation graph must have unique names. If the name argument is not provided a default name will be selected based on the node type:

x = sf.Constant(3.0, name='input')
squared = sf.Pow(x, 2.0)

print(x.name)       # Prints "input"
print(squared.name) # Prints "Pow_0"

Variables and Placeholders¶

In addiction to Constants, scalarflow includes two scalar-type Nodes: Variables and Placeholders.

Variables¶

Variables behave like constants, but their values may be set directly using the assign method:

with sf.Graph() as g:
    x = sf.Variable(4.0)
    sqrt = sf.Pow(x, .5)
    print(g.run(sqrt)) # Prints 2.0

    x.assign(25.0)
    print(g.run(sqrt)) # Prints 5.0

Variables are useful as trainable parameters in machine learning applications.

Placeholders¶

Placeholders must be assigned a value when run is called on the graph:

with sf.Graph() as g:
    x = sf.Constant(4.0)
    y = sf.Placeholder(name='y')
    sum = sf.Add(x, y)

    print(g.run(sum, feed_dict={'y': 5.0})) # Prints 9.0
    print(g.run(sum, feed_dict={'y': 10.0})) # Prints 14.0

Here, feed_dict is a dictionary that maps from placeholder node names to the value that should be used in the requested computation. Placeholder nodes are useful for representing inputs and outputs in machine learning training algorithms.

Node values¶

The run method of the graph will only execute the subset of nodes that are ancestors of the requested output. As a side effect, all of the values of those nodes are cached and are available through the value attribute:

with sf.Graph() as g:
    a = sf.Constant(7.0, name='a')
    b = sf.Constant(3.0, name='b')
    sum = sf.Add(a, b)
    sum_sqrd = sf.Pow(sum, 2.0)
    sum_to_fourth = sf.Pow(sum_sqrd, 2.0)

    g.run(sum_sqrd)

    print(sum.value) # prints 10.0, sum was computed!
    print(sum_to_fourth.value) # Something terrible happens, never computed.

Node derivatives¶

If the compute_derivatives argument is True, then run perform both a forward and backward pass. After the backward pass completes, partial derivatives will be available through the derivative attribute of each node that is involved in the computation:

with sf.Graph() as g:
    x = sf.Constant(4.0)
    y = sf.Pow(x, 2.0)  # y = x^2

    print(g.run(y)) # prints 16.0
    print(x.derivative) # prints dy/dx = 2x = 8.0

Scalarflow API¶

class scalarflow.Add(operand1, operand2, name='')¶: Addition. Node representing operand1 + operand2.

class scalarflow.BinaryOp(operand1, operand2, name)¶: Abstract base class for all nodes representing binary operators

class scalarflow.Constant(value, name='')¶: Constants behave like Variables that cannot be assigned values after they are created.

class scalarflow.Divide(operand1, operand2, name='')¶: Division. Node representing operand1 / operand2.

class scalarflow.Exp(operand, name='')¶: Exponential node: e^operand

class scalarflow.Graph¶

Computation Graph

A computation graph is a directed acyclic graph that represents a numerical computation performed on scalar-valued inputs. This class supports forward computations as well as reverse-mode automatic differentiation.

The Graph class also acts as a Python context manager, supporting the with keyword. For example:

with sf.Graph() as g:
    a = sf.Constant(1.0)
    b = sf.Constant(2.0)
    c = sf.Add(a, b)
    result = g.run(c)

nodes_by_name¶: a dictionary mapping from unique names to the corresponding node

gen_dot(filename, show_value=True, show_derivative=True)¶

Write a dot file representing the structure and contents of the graph.

The .dot file is a standard text-based format for representing graph structures. The graphviz library can convert dot files to images in many formats. There are also web-based tools for visualizing dot files. E.g. http://www.webgraphviz.com/

Parameters

filename – Name of the file to create
show_value – Show evaluated node values in the graph
show_derivative – Show evaluated node derivatives in the graph

run(node, feed_dict=None, compute_derivatives=False)¶

Run the computation graph and return the value of the indicated node.

After this method is called the value attribute of all the node and all of its ancestors will be set. The value attributes of non-ancestors are not defined.

If compute_derivatives is true, this method will also perform a backward pass to determine the numerical derivatives for the indicated node with respect to every ancestor. The derivatives will be accessible from the derivative attribute of the nodes. The derivatives of non-ancestors are not defined.

Parameters

node – Determine the value of this node
feed_dict – A dictionary mapping from Placeholder node names to values. E.g. {‘x’: 1.0, ‘y’: 2.0}.
compute_derivatives – True if we should perform a backward pass to compute partial derivatives.

Returns: The numeric value of of the indicated node.

class scalarflow.Log(operand, name='')¶: Log base e.

class scalarflow.Multiply(operand1, operand2, name='')¶: Multiplication. Node representing operand1 * operand2.

class scalarflow.Node(name)¶

Abstract base class for all nodes in a computation graph.

value¶: The most recently calculated value for this node. This is undefined if the node has never been involved in a computation.

derivative¶: The most recently calculated partial derivative for this node. This is undefined if the node has not been involved in a backward pass.

name¶

The name of this node. All nodes must have a unique name.

Type: string

property derivative: derivative should be read-only.

property value: Value should be read-only (except for variable nodes).

class scalarflow.Placeholder(name='')¶: Placeholders behave like Variables that can only be assigned values by including an appropriate value in the feed_dict passed to run.

class scalarflow.Pow(operand, power, name='')¶: Power E.g. operand^2 or operand^3

class scalarflow.Subtract(operand1, operand2, name='')¶: Subtraction. Node representing operand1 - operand2.

class scalarflow.UnaryOp(operand, name)¶: Abstract base class for all nodes representing unary operators

class scalarflow.Variable(value, name='')¶

Variable. A node that can be assigned a value.

assign(value)¶: Assign a new value to this variable

scalarflow.get_current_graph()¶

Return the currently active computation graph.

Inside of a graph context this will return the graph associated with that that context. Outside of any context this will return the default graph.