The nGraph Compiler is an optimizing compiler. As such, it provides a way to capture a given function graph and perform a series of optimization passes over that graph. The result is a semanticallyequivalent graph that, when executed using any backend, has optimizations inherent at the hardware level: superior runtime characteristics to increase training performance or reduce inference latency.

class
Matcher
¶ 
Matcher looks for node patterns in a computation graph. The patterns are described by an automaton that is described by an extended computation graph. The matcher executes by attempting to match the start node of the pattern to a computation graph value (output of a Node). In addition to determing if a match occurs, a pattern node may add graph nodes to a list of matched nodes, associate nodes with graph values, and start submatches. Submatches add match state changes to the enclosing match if the submatch succeeds; otherwise the state is reverted.
The default match behavior of a pattern node with a graph nodes is that the computation graph value is added to the end of the matched value list and the match succeeds if the node/pattern types match and the input values match. In the case of a commutative node, the inputs can match in any order. If the matcher is in strict mode, the graph value element type and shape must also match.
Pattern nodes that have different match behavior are in ngraph::pattern::op and have descriptions of their match behavior.
Public Functions

Matcher
(const Output<Node> &pattern_node, const std::string &name, bool strict_mode)¶ 
Constructs a Matcher object.
 Parameters

pattern_node
: is a pattern sub graph that will be matched against input graphsname
: is a string which is used for logging and disabling a matcherstrict_mode
: forces a matcher to consider shapes and ET of nodes

bool
match
(const Output<Node> &graph_value)¶ 
Matches a pattern to
graph_node
. Parameters

graph_value
: is an input graph to be matched against

bool
match
(const Output<Node> &graph_value, const PatternMap &previous_matches)¶ 
Matches a pattern to
graph_node
. Parameters

graph_value
: is an input graph to be matched againstprevious_matches
: contains previous mappings from labels to nodes to use

size_t
add_node
(Output<Node> node)¶ 
Lowlevel helper to match recurring patterns.
 Parameters

graph
: is a graph to be matched againstpattern
: is a recurring patternrpattern
: specifies a node to recur from nextpatterns
: a map from labels to matches

MatcherState
start_match
()¶ 
Try a match.

Fusion¶
There are several ways to describe what happens when we capture and translate the framework’s output of ops into an nGraph graph. Fusion is the term we shall use in our documentation; the action also can be described as: combining, folding, squashing, collapsing, or merging of graph functions.
Optimization passes may include algebraic simplifications, domainspecific simplifications, and fusion. Most passes share the same mode of operation (or the same operational structure) and consist of various stages (each one a step) where a developer can experiment with the intercepted or dynamic graph. These steps may be cycled or recycled as needed:
Locate a list of potentiallytransformable subgraphs in the given graph.
Transform the selected candidates into semanticallyequivalent subgraphs that execute faster, or with less memory (or both).
Verify that the optimization pass performs correctly, with any or all expected transformations, with the
NGRAPH_SERIALIZE_TRACING
option, which serializes a graph in the json format after a pass.Measure and evaluate your performance improvements with
NGRAPH_CPU_TRACING
, which produces timelines compatible withchrome://tracing
.
Optimizations can be experimented upon without using any backend by registering
a pass with pass manager (Manager
), calling run_passes
on a function, and
then inspecting the transformed graph.
Optimization passes can be programmed ahead of time if you know or can predict what your graph will look like when it’s ready to be executed (in other words: which ops can be automatically translated into nGraph Core ops).
The Interpreter
is simply a backend providing reference implementations of
ngraph ops in C++, with the focus on simplicity over performance.
Example¶
Let us first consider a simple example. A user would like to execute a graph that describes the following arithmetic expression:
\(a + b * 1\) or \(Add(a, Mul(b, 1))\)
In the above expressions, 1 is an identity element; any element multiplied by the identity element is equal to itself. In other words, the original expression \(a + b * 1\) is exactly equivalent to the expression \(a + b\), so we can eliminate this extra multiplication step.
The writer of an optimization pass which uses algebraic simplification would
probably want to first locate
all multiplication expressions where
multiplicands are multiplied by 1 (for stage 1) and to then replace
,
those expressions with just their multiplicands (for stage 2).
To make the work of an optimization pass writer easier, the nGraph Library
includes facilities that enable the finding of relevant candidates using
pattern matching (via pattern/matcher.hpp
), and the transforming of the
original graph into an optimized version (via pass/graph_rewrite.hpp
).