New Technology / Automotive Technology
Monitor automotive technology, EV innovation, software-defined vehicles and mobility transformation through structured tech briefings.
![[AI UNRAVELED SPECIAL] The 99.9% Wall: Solving the Long Tail of Autonomy via System 2 Reasoning](https://media.rss.com/djamgatech/20260205_120227_08e4d59afd9bab4a0d98f517f8116f77.jpg)
[AI UNRAVELED SPECIAL] The 99.9% Wall: Solving the Long Tail of Autonomy via System 2 Reasoning
Summary
The self-driving industry has made significant progress, reaching 99.9% of the way towards full autonomy. However, the final 0.1% presents unprecedented engineering challenges due to chaotic real-world scenarios that current AI systems struggle to navigate.
A shift from reactive to deliberative autonomy is necessary, emphasizing a deeper understanding of physical reality rather than mere mimicry of human behavior. This transition involves developing AI systems that can simulate various scenarios and maintain a continuous memory of past and predicted events.
Geopolitical tensions are creating separate autonomy silos, particularly between the United States and China, complicating the sharing of critical data needed for training AI models. This fragmentation limits the ability of AI to learn from diverse driving conditions, potentially leading to safety risks.
As vehicles transition to System 2 reasoning, the liability for decisions shifts from human drivers to the algorithms themselves. This change raises questions about accountability and the robustness of AI decision-making in unpredictable scenarios.
Perspectives
Focused on the challenges and shifts in the self-driving industry.
Proponents of System 2 Reasoning
- Advocate for a shift from reactive to deliberative autonomy
- Emphasize the need for AI to understand physical reality
- Highlight the importance of continuous memory in decision-making
- Support the use of advanced benchmarks for evaluating AI capabilities
- Argue that improved reasoning can enhance safety and reduce accidents
Critics of Current Approaches
- Question the effectiveness of AI models trained on localized data
- Raise concerns about the accountability of algorithms in decision-making
- Highlight the risks of relying on technology without sufficient human oversight
- Critique the fragmentation of data due to geopolitical tensions
Neutral / Shared
- Acknowledge the progress made in autonomous vehicle technology
- Recognize the complexity of human behavior in traffic scenarios
- Note the evolving landscape of insurance in relation to autonomous driving
- Identify the potential for future technologies to enhance vehicle intelligence
Metrics
engineering challenge
99.9%
level of autonomy achieved
Reaching this level indicates significant progress but highlights the remaining challenges.
getting to 99.9% was an engineering marvel
investment
over a billion euros EUR
investment in JAPA architecture
This significant funding indicates a strong commitment to advancing predictive AI technologies.
Jan LeCun's AMI labs just raised over a billion euros to champion an architecture called JAPA
reduction
95%
disengagements in full-self driving versions
A significant reduction in disengagements indicates improved safety and reliability of autonomous vehicles.
version 13 achieved 95% reduction in disengagement.
power
1.5 gigawatts
compute demands for training AI models
High power requirements highlight the increasing complexity and resource needs of AI systems.
Elon Musk's XAI brought their Colossus 2 cluster online and he pulls 1.5 gigawatts of power.
data_restriction
In 2025, the US Bureau of Industry and Security finalized rules, effectively banning Chinese and Russian software in con
US regulations on foreign software in autonomous vehicles
This regulation limits the integration of diverse technological advancements in the US.
In 2025, the US Bureau of Industry and Security finalized rules, effectively banning Chinese and Russian software in connected vehicles and automated driving systems.
data_localization
Any data collected by a foreign EV operating on Chinese soil must remain physically stored in process within Chinese bor
China's data localization laws
This law restricts the sharing of critical data necessary for AI training.
Any data collected by a foreign EV operating on Chinese soil must remain physically stored in process within Chinese borders.
safety_risk
If the ultimate goal of these billion-dollar AI labs is to build a general world model that understands universal fundam
Implications of localized training data
Localized training can lead to significant blind spots in AI understanding.
doesn't geofencing the training data essentially give the AI a localized lobotomy?
safety
significantly safer than human drivers
comparison of physics-aware models to human driving behavior
Demonstrates the potential of AI to reduce accidents caused by human error.
the data proves these physics-aware models are significantly safer than human drivers
Key entities
Timeline highlights
00:00–05:00
The self-driving industry is grappling with the final 0.1% of autonomy, which presents significant engineering challenges due to unpredictable real-world scenarios. A shift from mimicking human behavior to a more deliberative approach is necessary to address these complexities.
- The self-driving industry faces the 99.9% wall, where achieving the final 0.1% of autonomy is the hardest engineering challenge due to the chaotic nature of real-world scenarios. This challenge requires a shift from mimicking human behavior to a more deliberative approach to autonomy
- Current AI systems struggle with unpredictable human actions, as illustrated by a scenario where a teenager jumps in front of a vehicle. While human drivers can react instinctively, AI programmed for collision avoidance may fail to respond appropriately
- The focus is now on teaching AI universal physics instead of just imitating human drivers. This change is essential for addressing the complexities of driving scenarios and overcoming the long tail of chaos
- International geopolitics is complicating the development of a global data map necessary for autonomous vehicles. This fragmentation creates additional hurdles for the industry in establishing a cohesive framework for self-driving technology
- A significant evolution in autonomous vehicles is the need for cars to legally justify their split-second decisions in plain English. This requirement marks a major shift in the design and operation of autonomous systems
05:00–10:00
The self-driving industry faces significant challenges due to unpredictable human behavior and new micromobility devices that disrupt traditional traffic patterns. To address these issues, the industry is shifting towards System 2 reasoning, emphasizing a deeper understanding of physical reality over mere mimicry of human actions.
- The long tail of edge cases in autonomous driving is complicated by active human interference, known as multi-agent friction. This unpredictability challenges AI systems in navigating real-world scenarios, especially in urban areas where pedestrians exploit vehicle software limitations
- In cities like Beijing, new micromobility devices disrupt traditional traffic patterns, moving in unpredictable ways that existing kinematic models do not account for. This creates scenarios that challenge AI systems, similar to a novice driver lacking the intuition to respond to sudden changes
- To tackle these challenges, the industry is transitioning to System 2 reasoning, which emphasizes understanding physical reality over merely mimicking human behavior. This shift is essential for developing AI that can effectively navigate complex driving environments
- The JAPA architecture, developed by Jan LeCuns AMI labs, advances predictive AI by focusing on world state representations. This method prioritizes critical factors like occupancy and momentum, enhancing the AIs ability to navigate safely
10:00–15:00
The AI in autonomous vehicles employs latent space to simulate numerous scenarios, enhancing its ability to predict unpredictable traffic elements. Transitioning to System 2 reasoning, which includes a continuous 10-second memory buffer, significantly reduces disengagements in full-self driving versions.
- The AI in autonomous vehicles utilizes latent space as an internal imagination, allowing it to simulate millions of potential scenarios rapidly. This capability is crucial for predicting the actions of pedestrians and other unpredictable elements in traffic
- System 2 reasoning enhances AI capabilities through temporal transformers, which maintain a continuous 10-second memory buffer. This enables vehicles to predict the future positions of objects, addressing challenges like the child behind the car issue
- Transitioning to System 2 reasoning requires significant computational power, leading to a bifurcation in the industry at the silicon level. New full-self driving versions have achieved a 95% reduction in disengagements by adopting a physics-aware world model
- Occupational networks 3.0 and high-resolution voxalization are key technologies for this transition. Voxals allow the AI to understand physical space better, improving navigation around obstacles
15:00–20:00
The self-driving industry is facing significant challenges due to geopolitical tensions that are fragmenting global data sets, creating separate autonomy silos between the United States and China. This division complicates the development of AI models that require diverse data to understand various driving conditions effectively.
- Transitioning to System 2 reasoning in autonomous vehicles requires significant computational power, akin to running a modern video game on outdated hardware. This highlights the need for advanced silicon capable of handling complex simulations of physics
- World models for autonomous vehicles rely on vast amounts of data, particularly edge cases, to develop physical intuition. Geopolitical tensions are fragmenting global data sets, creating separate autonomy silos between the United States and China
- The Western approach to autonomous driving prioritizes safety with expensive sensors and detailed maps. In contrast, the Eastern approach focuses on scale and vision-centric systems that adapt dynamically to their environment
- Data sovereignty issues are limiting AI training capabilities, as regulations prevent sharing rich data sets across borders. This results in AI models being trained on localized data, restricting their understanding of diverse driving conditions
- The legal landscape for autonomous vehicles is evolving to focus on the reasoning behind a vehicles decisions. This necessitates developing a reasoning trace that explains the AIs choices in critical situations
- Vision-language action models, like Googles Gemini Robotics, translate complex mathematical decisions made by AI into understandable human language. This allows the AI to provide logical explanations of its actions after incidents
20:00–25:00
The OS World V benchmark is recognized as the gold standard for evaluating autonomous vehicles' reasoning capabilities and auditability, crucial for the insurance industry's adaptation. As these vehicles transition to System 2 reasoning, liability for decisions shifts from human drivers to the algorithms, marking a significant change in accountability.
- The OS World V benchmark is the gold standard for testing autonomous vehicles, emphasizing their ability to reason through multi-step decisions and explain their actions. This level of auditability is essential for the insurance industry to adapt to the evolving landscape of autonomous driving
- Insurance companies like Lemonade are offering discounts of up to 50% for drivers using supervised system-2 full-self driving, as data shows these models are significantly safer than human drivers. Unlike humans, these systems do not engage in risky behaviors such as texting or driving under the influence
- As autonomous vehicles adopt more deliberative thinking, liability for decisions shifts from the human driver to the algorithm itself, marking the beginning of agentic liability. If an AI agent makes a wrong choice, the financial responsibility falls on the manufacturers of the vehicle
- The future black box flight recorder will track speed and steering while articulating its decision-making process in human language. This evolution reflects the industrys commitment to accountability and transparency in autonomous vehicle operations
- The industry recognizes that simply adding more sensors or increasing test miles is insufficient to solve the final 0.1% of driving challenges. Vehicles are being redefined as thinking agents with physical intuition, capable of navigating complex social and geopolitical landscapes
- The upcoming 6G V2X concept envisions a collective intelligence layer for autonomous vehicles, enabling them to share experiences and updates in real time. If one vehicle encounters an unprecedented edge case, it can upload its learned response to a network, benefiting all vehicles in the vicinity