Overview

The whole stack

The whole nGraph Compiler stack

The nGraph Compiler stack consists of bridges, core, and backends. We’ll examine each of these briefly to get started.

A framework bridge interfaces with the “frontend” Core API. A framework bridge is a component that sits between a framework like TensorFlow or MXNet, and the nGraph Core frontend API. A framework bridge does two things: first, it translates a framework’s operations into graphs in nGraph’s in-memory Intermediary Representation. Second, it executes the nGraph IR graphs via the backend execution interface.

The details of bridge implementation vary from framework to framework, but there are some common patterns: a fairly typical example for a graph-based framework is illustrated here, and consists of basically two phases: a clustering phase and a translation phase.

The whole stack

Translation flow to an nGraph function

The clustering phase operates on the original framework’s graph. During this stage, we look for maximal subgraphs containing nodes that can be translated to data flow functions in nGraph. The ability to capture subgraphs of the original graph means that we maintain interoperability with the native framework runtime. Any node that is not placed in a cluster can still by handled by the native framework. On the other hand, identifying maximal subgraphs means that we can avoid unnecessary handoffs between the native framework runtime and nGraph; minimizing this is good for performance.

In the second phase, called translation, we cut out each cluster subgraph, translate it into an nGraph Function, and replace the cluster subgraph with a stand-in node called an “encapsulation node” that holds a pointer to the nGraph Function. Later, at runtime, those functions will be invoked when the framework asks us to execute the encapsulation node.

It’s worth noting that backends have total freedom to rewrite the nGraph Functions: they can do it for the sake of structural or algorithmic optimization of the graph, for easy integration with kernel libraries, or for any or no reason at all.

Namespaces in nGraph

What follows here is a table of all documented namespaces with brief descriptions:

Namespace

Description

Location in Repo

Docs

ngraph

The Intel nGraph C++ API

ngraph

Implicit namespace omitted from most API documentation

builder

Convenience functions that create additional graph nodes to implement commonly-used recipes; for example, auto-broadcast

builder

Coming Soon

descriptor

Descriptors are compile-time representations of objects that will appear at run-time

descriptor

Coming Soon

op

Ops used in graph construction

op

List of Core ops

runtime

The objects and methods used for executing the graph

runtime

Backend APIs