Machine learning powers diverse services in industry including search,
translation, recommendation systems, security, and more. The scale and
importance of these models require that they be efficient, expressive, and
portable across an array of heterogeneous hardware devices. These constraints are often at odds; in order to simultaneously accommodate them we propose a new high-level intermediate representation called Relay. Relay is a purely
functional, statically typed IR designed to balance efficient compilation,
expressiveness, and portability. We present a prototype of Relay and
highlight its important design decisions. Our prototype is part of the open
source NNVM compiler framework, which powers Amazon’s deep learning framework MxNet.