This week, with the lead of Yuanjun Gao, we discussed two papers, “Human-level concept learning through probabilistic program induction.” by Lake et al [1] and “One-Shot Generalization in Deep Generative Models.” by Danilo J. Rezende et al [2]. The papers aim to mimic humans’ ability to learn from small numbers of examples. The first paper introduces Bayesian Program Learning framework (BPL)—a probabilistic model which allows such learning ability—and the second paper implements the idea in deep generative model.

This week we scrutinized, in a discussion led by Shizhe Chen, two recent papers: “The Concrete Distribution: a Continuous Relaxation of Discrete Random Variables” by Chris Maddison and colleagues [1], and “Categorical Reparameterization by Gumbel-Softmax” by Eric Jang and collaborators [2]. Additionally, we considered a third paper: “GANS for Sequences of Discrete Elements with the Gumbel-Softmax Distribution” by Kusner and Hernández-Lobato [3]. These notes refer mainly to [1] and [2], which are currently under review for ICLR 2017. We also briefly address [3] at the end, which was presented in the recent “Adversarial Training” workshop at NIPS 2016.

This week we read and discussed two papers: a paper by Johnson et al. [1] titled “Composing graphical models with neural networks for structured representations and fast inference” and a paper by Gao et al. [2] titled “Linear dynamical neural population models through nonlinear embeddings.” Although the two papers have different focuses — the former proposes a general modeling and inference framework, whereas the latter focuses in particular on modeling neural activity — they are similar in that both use structured latent variable models as part of a variational autoencoder framework in order to perform inference. The benefit to this type of approach is that it allows for an increase in the flexibility of the models we can consider while retaining interpretability, even in high dimensions.