NYU researchers develop AI that mimics a toddler’s language learning journey


Researchers at New York University have developed an AI system that might be able to learn language like a toddler. The AI used video recordings from a child’s perspective to understand basic aspects of language development.

Today’s AI systems have impressive capabilities, but they learn inefficiently: they need millions of examples and consume massive amounts of computing power. They probably don’t develop a real understanding of things like humans might.

An AI that could learn like a child would be able to understand meaning, respond to new situations, and learn from new experiences. If AI could learn like humans, we would see a new generation of artificial intelligence that is faster, more efficient, and more versatile.

AI learns to associate visual cues with words

In a study published in the journal Science, researchers at New York University examined how toddlers associate words with specific objects or visual concepts, and whether an AI system can learn similarly.



The research team used video recordings from the perspective of a child between the ages of 6 and 25 months to train a “relatively generic neural network”.

The AI, called Child’s View for Contrastive Learning (CVCL), processed 61 hours of visual and speech data recorded in the context of the child’s visual scene. The speech data includes, for example, statements made by the child’s parents about an object visible in the video.

Using this data, the system learned the properties and connections between different sensory modalities to understand the meaning of words in the child’s visual environment.

The AI was able to learn many word-object associations that occurred in the child’s everyday experience with the same accuracy as an AI trained on 400 million captioned images from the Internet. It was also able to generalize to new visual objects and align its visual and linguistic concepts.

The results show that basic aspects of word meaning can be learned from a child’s experience, the researchers said. Whether this leads to actual understanding, and how easily and to what extent generalization is possible, remains to be seen.


Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top