Notes

Homomorphism Autoencoder (HomoAE) Paper: "Homomorphism Autoencoder - Learning Group Structured Representations from Observed Transitions".

A Homomorphism Autoencoder (HomoAE) is a type of autoencoder that is trained to preserve the homomorphism (structure-preserving) properties of the input data in its encoded representation. This is achieved by adding a homomorphism constraint to the standard autoencoder loss function. The constraint ensures that the encoded representation preserves certain properties of the input, such as symmetry or commutativity. The decoder then tries to reconstruct the original input based on the encoded representation, which should also possess the same homomorphism properties. The result is a neural network that can learn to preserve structural information in the data and can be used for tasks such as dimensionality reduction, data generation, and anomaly detection.

From the paper: "How can we acquire world models that vertically represent the outside world both in terms of what is there and in terms of how our actions affect it? Can we acquire such models by interacting with the world, and can we state mathematical desiderata for their relationship with a hypothetical reality existing outside our heads? As machine learning is moving towards representations containing not just observation but also interventional knowledge, we study these problems using tools from representation learning and group theory. Under the assumption that our actuators act upon the world, we propose methods to learn internal representations of not just sensory information but also of actions that modify our sensory representations in a way that is consistent with the actions and transitions in the world. We use an autoencoder equipped with a group representation linearly acting on its latent space, trained on 2-step reconstruction such as to enforce a suitable homomorphism property on the group representation. Compared to existing work, our approach makes fewer assumption on the group representation and on which transformations the agent can sample from the group. We motivate our method theoretically, and demonstrate empirically that it can learn the correct representation of the groups and the topology of the environment. We also compare its performance in trajectory prediction with previous methods."