Notes

Content of the Lecture

• Deep Learning
• Ontogenesis
• Representation/Perception
• Learning

Deep Learning It represents a revolution in Computer Science! All attention on System Architecture instead of on specific algorithms. Can we say that the Game is Over? Is intelligence just about computational power and data? Shall we stop studying the brain? Shall we give up on understanding the (human) mind?

Limitations of DL

• DL has always looked through human eyes, i.e., it is reliant on language and images that are processed by human understanding.
• DL cannot behave in natural environments (yet, mainly due to a problem in modelling corner situations).
• DL has no Situation Awareness (You are nobody for ChatGPT, Moravec's paradox).
• DL has no Personality (The system cannot take Responsibility, Veracity, Ethics).
• DL has no Efficient Learning.

Ontogenesis, Kolmogorov Complexity & Self-Reinforcing Networks

Ontogenesis It represents the process by which a human/animal is generated.

• Brain is wired to function at birth (e.g., ingestive, reproductive, defensive behavior).
• Reptile brain is inherited through eons.
• Expansion into cerebral hemispheres. (Homeomorphic Expansion: the Midbrain is hardwired genetically and the Cortex is a prestructured image of midbrain, subject to learning and self-organization).

Information Content of the Brain

• 10ˆ10 neurons.
• 10ˆ14 synaptic connections.
• 33 bits address per connection.
• 10ˆ15 bytes to describe the brain's wiring.

Genetic Information One gigabyte (3.3 billion nucleotides).

Training Information A few gigabytes of VR.

Information Gap

• One gigabyte of genetic information.
• Some gigabytes to describe the learning environment.
• A petabyte to describe the brain's wiring. Then where do 99% of the information come from?

Kolmogorov Complexity

• The shortest algorithm to create a structure.
• Julia Set: image created on the base of an algorithm.
• The amount of information that is required to describe this structure in infinite.
• Highly Efficient Kolmogorov Algorithms
  • Can create a lot of structure.
  • But only certain structures can be created that way.
• The brain must have a very efficient algorithm in the development of its structure.

The Brain's Kolmogorov Algorithm

• Low Kolmogorov Complexity.
• Powerfully reduced search space.
• Network Self-Organization.
• Attractor Networks: an universe of structured patterns.
• Ontogenesis of retinotopic fiber projections as example of network self-organization. (Development of wiring between the eye's retina and the optic tectum). During cooperation, neighboring cells help each other building stronger connections.

Emergence of Self-Reinforcing Network

• One-dimensional retina and tectum.
• Periodic Boundary Conditions.
• Small symmetry-braking component in initial state.
• Networks emerge on the basis of self-consistency of connectivity.

Two Types of Nets

• Neural Fields
• Topological Mappings

Representation, Perception & Learning

Representation and Perception

• Scene Representation/Perception
• Computer Graphics: Composing Modalities (Coloring, Material, Shading, Geometry, Edges, Silhouette). They generate images by putting together factors, a sort of ontogenesis of a scene.
• Perception (Computer Vision) is an Inverse Problem: going from images to the composing modalities.
• Object Recognition by Network Emergence. (To recognize a person, you need a model of the person in your cortex somewhere, there is an overlay of network fragments that in a given situation can be activated such that they together form a self-stabilizing network).

Learning

• Perception precedes Learning!
• Perception must be there at Birth.
• Learning only within the focus of attention.
• The Focus of Attention is shaped by Behavior.
• Learning is shaped by Strategy.
• Learning is gated by Emotions. (Locus Coeruleus communicates through Epinephrin, which is thought to be crucial for synaptic plasticity, hence it may play a substantial role in the memory of emotion-related events).
• Learning refines Consistency of Sensory-Motor Coordination.
• Learning takes Snapshots of Perceived Structures.

Conclusion Efficient Learning needs:

• A Functional Initial State.
• Behavioral Drive.
• Embodiment.
• Scene Representation.
• Perception.
• Focal Attention.