Mickey Mouse I-Diagram Information Theory

I(Y;Z) Preserved Relevant Information Y H(Y|X) Label Uncertainty Z H(Z|X) Encoding Uncertainty H(Y|Z) Decoder Uncertainty I(X;Y) Relevant Information I(X;Y|Z) Residual Information I(X;Z|Y) Redundant Information H(Z|Y) Reverse Decoder Uncertainty I(X;Z) Preserved Information H(X|Y,Z) X Figure 2: Mickey Mouse I-diagram for the information quantities in the model p(x, y, z) = p̂(x, y) pθ(z | x). X is for the data, Y for the labels, Z for the latent encodings. See section 2 for more details and section B.1 for a description of all the quantities. I[Y; Z] = I[X; Y; Z] because I[Y; Z | X] = 0. Best viewed in color.

A colorful Mickey Mouse-shaped information-theory diagram labels the relationships among data X, labels Y, and latent encodings Z. The joke comes from presenting a dense academic I-diagram in the unmistakable shape of Mickey Mouse, making highly technical research look like a Disney-themed meme. It is used as a reaction to unnecessarily complicated explanations, overengineered diagrams, or papers that hide a funny visual reference in serious mathematics.

• @ithueti / 3531 2021-01-13