Analysing Information Decomposition Between Modalities

EasyChair Preprint 9721

Abstract

Due to the rise of multimodal deep-learning, there are now a lot of datasets and ways to represent and combine information from different signals. However, despite such advances questions on characterizing feature interactions in multimodal datasets (i.e. which are datasets that contain information from different signals or sources) is not well studied.
We propose a method based on classical information theory to measure the degree of redundancy, uniqueness, and synergy among the input modalities. We work with two estimators for information that work well with high-dimensional datasets. We conduct experiments with real-world datasets, to assess the quality of the proposed procedure.

Keyphrases: deep learning, information, multimodal

@booklet{EasyChair:9721,
  author    = {Aiden Boyd and Rishi Agrawal and Jennifer Moss and Steph Mcgory},
  title     = {Analysing Information Decomposition Between Modalities},
  howpublished = {EasyChair Preprint 9721},
  year      = {EasyChair, 2023}}