concrete

TensorFlow implementation of variational inference for Hidden Markov Model(HMM) using the continuous relaxation of categorical latent variables described in both The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables by Chris J. Maddison, Andriy Mnih, Yee Whye Teh and Categorical Reparameterization with Gumbel-Softmax by Eric Jang, Shixiang Gu, Ben Poole.

Math

1. Gumbel distribution
2. Concrete random variable / Gumbel-Softmax distribution
   1. definition (reparameterization)
   2. properties (categorical under zero temperature; close to uniform under high temperature; convexity under certain conditions)

Problem Setup & Training Details

1. Section 3.3 & Appendix C of The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
   1. the modified training objective / ELBO
   2. Concrete in log-space
   3. temperature (annealing)
2. Section 2.2 & 3.2 of Categorical Reparameterization with Gumbel-Softmax
   1. Straight-Through Gumbel-Softmax (Figure 2)
   2. Score Function / REINFORCE & control variates
3. Extension:
   1. gradient estimates
      1. REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
      2. Backpropagation through the Void: Optimizing control variates for black-box gradient estimation
      3. ARM: Augment-REINFORCE-Merge Gradient for Discrete Latent Variable Models
   2. HHMM
      1. The Hierarchical Hidden Markov Model: Analysis and Applications
      2. Linear-time inference in Hierarchical HMMs
      3. Hierarchical Hidden Markov Models with General State Hierarchy
      4. Infinite Hierarchical Hidden Markov Models

Experiments

• Gaussian observations, K = 2, N = 500, fixed temperature & exponential decay