Threat Picture

Latest Cybersecurity News

Yann LeCun: Teaching Computers to See, One Check at a Time

Long before chatbots, his team taught computers to read handwritten checks. His later work asks how machines can learn the way the physical world works.
By Charles Joseph · Updated
Pen-and-ink watercolor portrait of Yann LeCun with graying hair, black-framed glasses and a dark blazer.
Yann LeCun. Portrait adapted from a photograph by Jérémy Barande (CC BY-SA 2.0).
Share
Share
Copy Link

Yann LeCun is a computer scientist who helped make image recognition a practical use of AI. His work on neural networks, systems that learn patterns from examples, helped computers read handwritten numbers and identify features in pictures.

One early application went into banks. In 1996, NCR began deploying technology developed by LeCun and his colleagues to read handwritten checks. The system found the amount, considered different ways to divide the handwriting and passed uncertain cases to a person.

LeCun's later work asks how machines can learn enough about the physical world to predict what happens next. His early research began with a smaller but stubborn problem: the same digit can look different every time someone writes it.

The Same Digit Can Look Very Different

LeCun trained as an engineer in Paris and completed his doctorate in 1987. After postdoctoral work in Toronto with Geoffrey Hinton, he joined Bell Laboratories in 1988.

Handwriting presented a revealing problem. Two people can write the same digit with different shapes, while a small shift in position can change which pixels in an image are dark.

A system based on a rigid picture of each digit would struggle with those variations. LeCun and his colleagues used neural networks, connected calculations whose numerical settings change during training. Given enough examples, a network could learn which visual patterns helped distinguish one digit from another.

Their 1989 paper on handwritten ZIP codes described a network trained with digit images supplied by the U.S. Postal Service. LeCun worked with Bernhard Boser, John Denker, Donnie Henderson, Richard Howard, Wayne Hubbard and Lawrence Jackel.

The network learned a path from an input image to a classification. Its design gave it a useful starting assumption: nearby pixels belong together, and a visual pattern can matter wherever it appears.

That idea is central to a convolutional neural network. A small detector examines part of the image, then applies the same learned calculation at other locations.

For example, a detector useful for finding a stroke doesn't need to be learned again from scratch each time the stroke shifts sideways. Reusing it helps the network recognize related patterns across the picture.

Later layers combine the earlier responses into features useful for the final decision. Other processing steps reduce sensitivity to small shifts, so slight changes in handwriting needn't produce entirely different answers.

Training adjusts the network's numerical settings using backpropagation, which calculates how changes would affect its errors. The method helps the system learn which visual patterns are useful rather than requiring programmers to specify every one.

Sponsored

From Recognizing Digits to Reading an Amount

LeCun's work built on earlier research, including Kunihiko Fukushima's Neocognitron and ideas about visual processing. The Bell Labs teams helped turn those architectural ideas into effective systems that could be trained on real recognition problems.

The LeNet family of networks became an important part of that effort. In a 1998 paper, LeCun, Léon Bottou, Yoshua Bengio and Patrick Haffner described LeNet-5 and a wider approach to document recognition.

Reading a bank check required several pieces to work together. The software had to locate the numeric amount, consider where one character ended and another began, recognize possible characters and evaluate whether the result made sense as an amount.

Handwriting doesn't always leave neat gaps. Considering several possible divisions allowed the system to use the recognition results when deciding which interpretation was most plausible.

It could also reject cases it wasn't confident about, leaving them for a person to handle. That option was part of making the technology useful: the system needed a sensible way to manage uncertainty as well as a way to produce an answer.

Building Places for the Research

LeCun became head of image-processing research at AT&T Labs in 1996 and joined New York University in 2003. At NYU, he also became the founding director of the Center for Data Science.

In 2013, he became the founding director of Facebook AI Research, known as FAIR. The group worked on longer-term questions about how machines could learn, reason and plan.

Under LeCun's leadership, FAIR also shared papers, code and tools with other researchers. Its account of its first five years describes this exchange as part of the group's approach to making progress.

In 2018, he became chief AI scientist at Facebook, later renamed Meta. He held the role through 2025. His academic work continued alongside the industrial position.

LeCun, Bengio and Hinton shared the 2018 Turing Award, announced in 2019, for their contributions to deep neural networks. The award recognized both their ideas and the engineering needed to make the methods work.

Deep learning uses multiple layers of calculations, with each layer learning patterns that the next can use. LeCun's handwriting work provided an early, practical demonstration of how that approach could help machines interpret images.

Learning From What Happens Next

Even while image recognition improved, LeCun was arguing for a broader research direction. In a 2018 NYU seminar, he described the difficulty of teaching machines about the physical world through human labels or huge numbers of trial-and-error attempts.

A label might tell a system that an image contains a cup. It doesn't by itself teach the relationships a machine would need to predict what happens when an object moves or an action changes its surroundings.

LeCun's proposed direction involves world models: learned representations of an environment that help a system predict consequences. A robot could use such a model to consider possible actions before choosing one.

In his 2022 position paper, he outlined an approach combining world models, goals and representations at different levels. The model would preserve information useful for reasoning and planning while leaving out unpredictable detail.

A useful prediction doesn't have to reproduce every pixel of a future scene. It needs to preserve the information that helps the machine decide what to do.

LeCun argues that learning directly from images, video and interaction is essential to this kind of intelligence. His interest in world models predates the public arrival of ChatGPT.

Sponsored

Predicting Features, Then Trying Actions

One step appeared in I-JEPA, a 2023 paper led by Mahmoud Assran with LeCun and other collaborators. The system used visible parts of an image to predict numerical representations of withheld regions.

The training task encouraged it to learn useful features from the image itself. This is called self-supervised learning: the material supplies the prediction task, reducing the need for people to attach a label to every example.

In 2025, V-JEPA 2 extended the work with image and video training, followed by robot-interaction data. The team demonstrated picking and placing tasks with robot arms in two laboratories.

The model had learned from other robot data, but it could perform these tasks without new training data collected in those particular deployments. It was a promising result for picking and placing objects in the tested settings.

LeCun became executive chairman of Advanced Machine Intelligence Labs in January 2026. He also remains the Jacob T. Schwartz Professor at NYU.

AMI Labs is pursuing world models that learn from sensor data and predict the consequences of actions. Its founding team also includes Alex LeBrun, Pascale Fung and Saining Xie.

The company plans to develop systems with memory, reasoning and controllable behavior, with potential uses in robotics and industry. These remain research goals. For LeCun, the direction is clear: machines need to learn enough about their surroundings to consider what an action will do before they take it.

QUOTE:

"Amateurs hack systems, professionals hack people."