Speaker
Description
Information theory guides the design of information processing systems. It permits the construction of optimal communication channels between systems to sense, process, and represent information. A key concept thereby is relative entropy, quantifying the amount of information lost in a transaction. Relative entropy has, however, no sense of relevance: a bit of information on an irrelevant question weights as much as one on a relevant one. To introduce weighting into communication and reasoning, relative attention entropy is introduced here, which can be derived axiomatically. Attention entropy can ensure the preferential communication of relevant information. Furthermore, new relative entropy based algorithms for the inference of large numbers of model parameters are presented, Metric Gaussian Variational Inference (MGVI) and geometric Variational Inference (geoVI). Their application to problems of field inference with millions to billion of unknown parameters is illustrated by recent astrophysical applications, ranging from gamma ray sky to black hole imaging. These algorithms also permit the combination of independent pre-trained neural networks into reasoning systems for questions, none of their constituents could answer alone.
