Notes

Strategies for Continual Machine Learning

• Architectural
  • Use only part of a network.
  • Select subnetwork based on task.
  • Or, add new parts to a network when new tasks are learnt.
  • Progressive Networks.
• Regularization-Based
  • Add loss terms that try to keep the network constant:
    • In behavior or
    • In weights.
    • Elastic Weight Consolidation & Synaptic Intelligence.
• Data Replay-Based
  • Store, or learn to generate some of the previous data (Experience Replay).
  • Keep training on them while you train on new data (Generative Replay).
  • Deep Generative Replay.

An Example of Architectural methods (Progressive Networks) "Progressive Neural Networks" is a paper published by Google Brain team in 2017, that describes a method for incremental learning, which allows neural networks to learn new tasks without forgetting the previous ones. The paper propose a technique called Progressive Networks (PN), which is based on the idea of growing the neural network incrementally as new tasks are encountered. The PN approach consists of a multi-task neural network, where each task is associated with a specific sub-network called a "column". Each column is trained to perform a specific task, and new columns can be added as new tasks are encountered. The new columns are connected to the previously learned columns, allowing the network to transfer knowledge from previous tasks to new ones. The paper shows that the PN approach can learn a wide range of tasks, with different levels of difficulty, and it can also achieve better performance compared to other methods for incremental learning, such as fine-tuning or freezing the previous layers.

The Progressive Network approach is useful in scenarios where the number of tasks or the amount of data is not known in advance, and it can be useful in applications such as lifelong learning, online learning and online adaptation.