Energy Based Models

• Score, log-likelihood gradient, energy gradient: The gradient of the log-llikelihood is the score, which is equal to the negative gradient of the energy (How to Train Your Energy-Based Models p.4)
• The Energy-based model - Wikipedia page for EBMs isn’t bad and links through to Boltzmann machine - Wikipedia
  • Hinton used block Gibbs Sampling to estimate the gradient of the log-likelihood for Boltzmann machines
  • mentions that people now use: Langevin dynamics; Hamiltonians - see Optimisation

Papers

• Implicit Generation and Generalization in Energy-Based Models
• How to Train Your Energy-Based Models - tutorial piece from Diederik P. Kingma
• Artificial Kuramoto Oscillatory Neurons