Title: Formal Limitations on the Measurement of Mutual Information
Authors: David McAllester, Karl Stratos
Published: 10th November 2018 (Saturday) @ 13:12:27
Link: http://arxiv.org/abs/1811.04251v4
Abstract
Measuring mutual information from finite data is difficult. Recent work has considered variational methods maximizing a lower bound. In this paper, we prove that serious statistical limitations are inherent to any method of measuring mutual information. More specifically, we show that any distribution-free high-confidence lower bound on mutual information estimated from N samples cannot be larger than O(ln N ).