Probabilistic modeling and inference problems are often computationally intractable, so practitioners frequently design custom approximation algorithms for the problem at hand. However, current probabilistic programming languages typically provide only a small set of black-box inference algorithms that cannot be customized. This limitation renders most probabilistic programming languages unsuitable for most applications of probabilistic modeling and inference. This paper introduces novel inference meta-programming constructs for expressing custom inference algorithms in probabilistic programming languages. These constructs enable probabilistic programmers to (i) dynamically decompose inference problems into subproblems; (ii) apply inference tactics to subproblems; (iii) alternate between incorporating new data and performing inference given existing data; and (iv) explore multiple execution traces of the probabilistic program at once. Together, these inference meta-programming constructs allow probabilistic programmers to express a broad class of algorithms, including novel hybrids of gradient-based search, combinatorial search, Markov chain Monte Carlo, sequential Monte Carlo, and variational inference. Experimental results on a collection of probabilistic programs, written in the Venture probabilistic programming language, highlight the significant performance and accuracy benefits that programmable inference can deliver.
Fri 22 Jun
|11:00 - 11:25|
Marco Cusumano-TownerMIT-CSAIL, Benjamin BichselETH Zurich, Switzerland, Timon Gehr, Martin VechevETH Zürich, Vikash MansingkhaMITMedia Attached
|11:25 - 11:50|
Timon Gehr, Sasa MisailovicUniversity of Illinois at Urbana-Champaign, USA, Petar TsankovETH Zurich, Laurent VanbeverETH Zürich, Pascal WiesmannETH Zurich, Switzerland, Martin VechevETH ZürichMedia Attached
|11:50 - 12:15|
Vikash MansingkhaMIT, Ulrich SchaechtleMassachusetts Institute of Technology, USA, Shivam Handa, Alexey Radul, Yutian ChenGoogle Deepmind, n.n., Martin RinardMassachusetts Institute of TechnologyMedia Attached