Dynamic Shapes

For an example on how to use dynamic shapes, see the Scenario Two: Known Partial Shape documentation.

Runtime Error Checking

Static type-checking in the presence of dynamic shapes will make optimistic assumptions about things like shape mismatches. For example, if an elementwise op is provided inputs of shapes (2,?) and (?,5), the type checker will proceed under the assumption that the user is not going to pass tensors with inconsistent shape at runtime, and therefore infer an output shape of (2,5). That means that shape mismatches can now occur at runtime.

PartialShape, Dimension, and Rank Classes

Partial shape information is expressed via the PartialShape, Dimension, and Rank classes.

Note

Rank is an alias for Dimension, used when the value represents the number of axes in a shape, rather than the size of one dimension in a shape.

class PartialShape

Class representing a shape that may be partially or totally dynamic.

XXX: THIS CLASS IS EXPERIMENTAL AND THE ENTIRE DESIGN IS SUBJECT TO CHANGE.

A PartialShape may have:

  • Dynamic rank. (Informal notation: ?)

  • Static rank, but dynamic dimensions on some or all axes. (Informal notation examples: {1,2,?,4}, {?,?,?})

  • Static rank, and dynamic dimensions on all axes. (Informal notation examples: {1,2,3,4}, {6}, {})

Public Functions

PartialShape(std::initializer_list<Dimension> init)

Constructs a shape with static rank from an initializer list of Dimension.

Examples:

Parameters
  • init: The Dimension values for the constructed shape.

PartialShape s{2,3,4};                     // rank=3, all dimensions static
PartialShape s{};                          // rank=0
PartialShape s{2,Dimension::dynamic(),3};  // rank=3, dimension 1 dynamic
PartialShape(const std::vector<Dimension> &dimensions)

Constructs a PartialShape with static rank from a vector of Dimension.

Parameters
  • dimensions: The Dimension values for the constructed shape.

PartialShape()

Constructs a static PartialShape with zero rank (the shape of a scalar).

PartialShape(const Shape &shape)

Constructs a static PartialShape from a Shape.

Parameters
bool is_static() const

Check if this shape is static.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Return

true if this shape is static, else false.

bool is_dynamic() const

Check if this shape is dynamic.

A shape is considered static if it has static rank, and all dimensions of the shape are static.

Return

false if this shape is static, else true.

Rank rank() const

Get the rank of the shape.

Return

The rank of the shape. This will be Rank::dynamic() if the rank of the shape is dynamic.

bool compatible(const PartialShape &s) const

Check whether this shape is compatible with the argument, i.e., whether it is possible to merge them.

Two shapes are compatible if

  • one or both of them has dynamic rank, or

  • both shapes have dynamic and equal rank, and their dimensions are elementwise compatible (see Dimension::compatible()).

Return

true if this shape is compatible with s, else false.

Parameters
  • s: The shape to be checked for compatibility with this shape.

bool same_scheme(const PartialShape &s) const

Check whether this shape represents the same scheme as the argument.

Two shapes

s1 and s2 represent the same scheme if
  • they both have dynamic rank, or

  • they both have static and equal rank r, and for every i from 0 to r-1, s1[i] represents the same scheme as s2[i] (see Dimension::same_scheme()).

Return

true if this shape represents the same scheme as s, else false.

Parameters
  • s: The shape whose scheme is being compared with this shape.

bool relaxes(const PartialShape &s) const

Check whether this shape is a relaxation of the argument.

Intuitively, a

PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “more permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s2 is also something you can form by plugging things into the dynamic dimensions of s1, but not necessarily the other way around.
Return

true if this shape relaxes s, else false.

Parameters
  • s: The shape which is being compared against this shape.

s1.relaxes(s2) is equivalent to s2.refines(s1).

Formally, PartialShape s1 is said to relax PartialShape s2 if:

  • s1 has dynamic rank, or

  • s1 and s2 both have static rank r, and for every i from 0 to r-1, either s1[i] is dynamic, or s1[i] == s2[i].

bool refines(const PartialShape &s) const

Check whether this shape is a refinement of the argument.

Intuitively, a

PartialShape s1 is said to relax s2 (or is a relaxation of s2) if it is “less permissive” than s2. In other words, s1 is a relaxation of s2 if anything you can form by plugging things into the dynamic dimensions of s1 is also something you can form by plugging things into the dynamic dimensions of s2, but not necessarily the other way around.
Return

true if this shape refines s, else false.

Parameters
  • s: The shape which is being compared against this shape.

s1.refines(s2) is equivalent to s2.relaxes(s1).

Formally, PartialShape s1 is said to refine PartialShape s2 if:

  • s2 has dynamic rank, or

  • s1 and s2 both have static rank r, and for every i from 0 to r-1, either s2[i] is dynamic, or s1[i] == s2[i].

bool merge_rank(Rank r)

Checks that this shape’s rank is compatible with r, and, if this shape’s rank is dynamic and r is static, updates this shape to have a rank of r with dimensions all dynamic.

Return

true if this shape’s rank is compatible with r, else false.

Shape to_shape() const

Convert a static PartialShape to a Shape.

Return

A new Shape s where s[i] = size_t((*this)[i]).

Exceptions
bool all_non_negative() const

Returns true if all static dimensions of the tensor are non-negative, else false.

const Dimension &operator[](size_t i) const

Index operator for PartialShape.

Return

A reference to the ith Dimension of this shape.

Parameters
  • i: The index of the dimension being selected.

Dimension &operator[](size_t i)

Index operator for PartialShape.

Return

A reference to the ith Dimension of this shape.

Parameters
  • i: The index of the dimension being selected.

Public Static Functions

PartialShape dynamic(Rank r = Rank::dynamic())

Construct a PartialShape with the given rank and all dimensions (if any) dynamic.

Return

A PartialShape with the given rank, and all dimensions (if any) dynamic.

bool merge_into(PartialShape &dst, const PartialShape &src)

Try to merge one shape into another.

Merges

src into dst, returning true on success and false on failure. If false is returned, the effect on dst is unspecified.
Return

true if merging succeeds, else false.

Parameters
  • [inout] dst: The shape that src will be merged into.

  • src: The shape that will be merged into dst.

To merge two partial shapes s1 and s2 is to find the most permissive partial shape s that is no more permissive than s1 or s2, if s exists. For example:

merge(?,?) -> ?
merge(?,{?,?}) -> {?,?}
merge({?,?},{?,?}) -> {?,?}
merge({1,2,3,4},?) -> {1,2,3,4}
merge({1,2},{1,?}) -> {1,2}
merge({1,2,?,?},{1,?,3,?}) -> {1,2,3,?}
merge({1,2,3},{1,2,3}) -> {1,2,3}

merge({1,?},{2,?}) fails [dimension 0 constraints are inconsistent]
merge({?,?},{?,?,?}) fails [ranks are inconsistent]

This function (merge_into) performs the “merge” operation described above on dst and src, but overwrites dst with the result and returns true if merging is successful; if merging is unsuccessful, the function returns false and may make unspecified changes to dst.

bool broadcast_merge_into(PartialShape &dst, const PartialShape &src, const op::AutoBroadcastSpec &autob)

Try to merge one shape into another along with implicit broadcasting.

Friends

std::ostream &operator<<(std::ostream &str, const PartialShape &shape)

Inserts a human-readable representation of a PartialShape into an output stream.

The output to the stream is in “informal” notation. In other words:

Return

A reference to str after insertion.

Parameters
  • str: The output stream targeted for insertion.

  • shape: The shape to be inserted into str.

  • If shape has dynamic rank, inserts the string ?.

  • If shape has static rank, inserts the string {, then inserts each dimension of shape into the output stream separated by commas, then inserts }.

Example:

PartialShape s1{PartialShape::dynamic())};
PartialShape s2{};
PartialShape s3{1,Dimension::dynamic(),2,3};
PartialShape s4{2,3,4};
std::cout << s1 << std::endl
          << s2 << std::endl
          << s3 << std::endl
          << s4 << std::endl;

Output:

?
{}
{1,?,2,3}
{2,3,4}
PartialShape operator+(const PartialShape &s1, const PartialShape &s2)

Elementwise addition of two PartialShape objects.

  • If s1 or s2 has dynamic rank, returns PartialShape::dynamic().

  • If s1 ands2` both have static rank, and their ranks are unequal, throws std::invalid_argument.

  • If s1 and s2 both have static rank, and their ranks are equal, returns a new shape whose ith dimension is s1[i] + s2[i].

Return

The result of elementwise adding s1 to s2 (see description).

Parameters
  • s1: Left operand for addition.

  • s2: Right operand for addition.

Exceptions
  • std::invalid_argument: If s1 and s2 have inconsistent ranks.

class Dimension

Class representing a dimension, which may be dynamic (undetermined until runtime), in a shape or shape-like object.

Static dimensions may be implicitly converted from int64_t. A dynamic dimension is constructed with Dimension() or Dimension::dynamic().

XXX: THIS CLASS IS NOT IN USE YET AND THE ENTIRE DESIGN IS SUBJECT TO CHANGE.

Public Functions

Dimension(int64_t dimension)

Construct a static dimension.

Parameters
Exceptions
Dimension()

Construct a dynamic dimension.

bool is_static() const

Check whether this dimension is static.

Return

true if the dimension is static, else false.

bool is_dynamic() const

Check whether this dimension is dynamic.

Return

false if the dimension is static, else true.

operator int64_t() const

Convert this dimension to int64_t. This dimension must be static.

Exceptions
  • std::invalid_argument: If this dimension is dynamic.

operator size_t() const

Convert this dimension to size_t. This dimension must be static and non-negative.

Exceptions
  • std::invalid_argument: If this dimension is dynamic or negative.

bool same_scheme(const Dimension &dim) const

Check whether this dimension represents the same scheme as the argument (both dynamic, or equal).

Return

true if this dimension and dim are both dynamic, or if they are both static and equal; otherwise, false.

Parameters
  • dim: The other dimension to compare this dimension to.

bool compatible(const Dimension &d) const

Check whether this dimension is capable of being merged with the argument dimension.

Two dimensions are considered compatible if it is possible to merge them. (See

Dimension::merge.)
Return

true if this dimension is compatible with d, else false.

Parameters
  • d: The dimension to compare this dimension with.

bool relaxes(const Dimension &d) const

Check whether this dimension is a relaxation of the argument.

A dimension

d1 relaxes (or is a relaxation of) d2 if d1 and d2 are static and equal, or d1 is dynamic.
Return

true if this dimension relaxes d, else false.

Parameters
  • d: The dimension to compare this dimension with.

d1.relaxes(d2) is equivalent to d2.refines(d1).

bool refines(const Dimension &d) const

Check whether this dimension is a refinement of the argument.

A dimension

d2 refines (or is a refinement of) d1 if d1 and d2 are static and equal, or d2 is dynamic.
Return

true if this dimension relaxes d, else false.

Parameters
  • d: The dimension to compare this dimension with.

d1.refines(d2) is equivalent to d2.relaxes(d1).

Dimension operator+(const Dimension &dim) const

Addition operator for Dimension.

Return

Dimension::dynamic() if either of *this or dim is dynamic; else, a static dimension with value int64_t(*this)+in64_t(dim).

Parameters
  • dim: Right operand for addition.

Dimension operator-(const Dimension &dim) const

Subtraction operator for Dimension.

Return

Dimension::dynamic() if either of *this or dim is dynamic; else, a static dimension with value int64_t(*this)-int64_t(dim).

Parameters
  • dim: Right operand for subtraction.

Dimension operator*(const Dimension &dim) const

Multiplication operator for Dimension.

Return

0 if either of *this or dim is static and 0; else, Dimension::dynamic() if either of *this or dim is dynamic; else, a static dimension with value int64_t(*this)*int64_t(dim).

Parameters
  • dim: Right operand for multiplicaiton.

Dimension &operator+=(const Dimension &dim)

Add-into operator for Dimension.

Return

A reference to *this, after updating *this to the value *this + dim.

Parameters
  • dim: Right operand for addition.

Dimension &operator*=(const Dimension &dim)

Multiply-into operator for Dimension.

Return

A reference to *this, after updating *this to the value *this * dim.

Parameters
  • dim: Right operand for multiplication.

Public Static Functions

bool merge(Dimension &dst, const Dimension d1, const Dimension d2)

Try to merge two Dimension objects together.

  • If d1 is dynamic, writes d2 to dst and returns true.

  • If d2 is dynamic, writes d1 to dst and returns true.

  • If d1 and d2 are static and equal, writes d1 to dst and returns true.

  • If d1 and d2 are both static and unequal, leaves dst unchanged and returns false.

Return

true if merging succeeds, else false.

Parameters
  • [out] dst: Reference to write the merged Dimension into.

  • d1: First dimension to merge.

  • d2: Second dimension to merge.

bool broadcast_merge(Dimension &dst, const Dimension d1, const Dimension d2)

Try to merge two Dimension objects together with implicit broadcasting of unit-sized dimension to non unit-sized dimension.

static Dimension dynamic()

Create a dynamic dimension.

Return

A dynamic dimension.

Public Static Attributes

const int64_t s_dynamic_val = {(std::numeric_limits<int64_t>::max())}

Constant for the value used internally to represent a dynamic dimension.