Artificial intelligence has seen the fastest adoption curve of any new technology in history. Most of that impact, however, has stayed behind the desks of knowledge workers, helping them draft emails, summarize reports, and write code. 

What about the people working outside the office? The ones building our roads, bridges, manufacturing facilities, data centers and homes?

I’ve spent my career working at the intersection of advanced technology and the physical world. While studying at Stanford, I joined a team optimizing the university’s central energy facility. This project would eventually cut campus emissions by 68%, reduce water usage by 18%, and save the school $520 million over time. I later spent nine years at Google, working as a product manager to develop and ship frontier AI-sensing systems designed to tackle real-world problems.

What I’ve learned across these experiences is that the world’s most important problems are physical, not digital, yet these real-world problems are often the hardest for technology to address.

Physical industries like construction, logistics, and manufacturing make up a major portion of the global economy, yet they’ve seen only a fraction of the value AI has delivered elsewhere. It’s not due to lack of interest. 89% of companies in these sectors plan to use AI, but most of them hit a wall when it comes to deployment. 

Why? Because the physical world is fundamentally harder for machines to interpret. Unlike chatbots, which rely on simple text for input, in construction and similar fields, text makes up a small fraction of the available data. The rest comes from messy, multimodal sensor inputs: video footage, machine telemetry, GPS data, weather stations, vibration monitors, and more. Each of these signal types speaks a different language. Traditionally, you’ve needed a bespoke software tool for interpreting each kind of signal, and you could only analyze them one-by-one, meaning much of that data would go to waste.

This is the challenge, and the opportunity, that led me to co-found Archetype AI, where we are pioneering the new category of Physical AI. While large language models are designed to understand text from the web, physical AI expands on this with AI systems that can make sense of real-world sensor data to help solve real life problems.

Predictions Based on Sensor Data

Our work with customers like KAJIMA, one of Japan’s oldest and most respected construction companies, is showing how AI can unlock the value of this data to make a measurable impact on construction projects. These are massive undertakings involving thousands of workers, tens of millions of dollars of equipment, and hundreds of millions of dollars in materials. Even small improvements can mean big benefits.  

On a multi-year project to widen a 100-year-old canal in flood-prone Niigata, KAJIMA project managers needed daily insight into construction progress and the causes of delays. When weather stops work, it can trigger cascading delays and cost overruns that add up to millions over the course of a project. Better predicting weather impacts, however, allows teams to quickly reallocate resources and revise schedules to stay on track.

Using Newton, our foundation model for the physical world, we created a system with KAJIMA that could analyze and interpret sensor data directly from KAJIMA’s jobsite, including more than two years of weather logs and footage from 27 cameras (nearly 12,000 videos). Managers could then see visual summaries of daily operations, flag deviations from work plans, and compare productivity across weather conditions, all in a unified interface.\

Generational Shift

This kind of knowledge about how to navigate fluid jobsite conditions and make rapid, high-stakes decisions has historically lived in the heads of veteran engineers and project managers. As seasoned project managers retire, though, years of planning intuition and problem-solving insight depart with them. The construction industry is now facing not just a labor shortage, but a widening knowledge gap. When used by veteran teams, our Newton AI platform can learn from their expertise: why did a delay happen, what conditions caused a deviation, what distinguishes a normal idle period from a red flag? Over time, Newton can become a kind of institutional memory teams can build on. 

AI will not just preserve knowledge from construction teams, but ultimately augment them with new insights that weren’t previously possible. AI has an unprecedented ability to analyze and discover hidden patterns in vast volumes of sensor data. Imagine knowing not just that a weather front is coming, but which construction activities it will affect, which teams will be delayed, and which scheduling workarounds will be best for staying on track. Teams embedding physical AI in their operations can go beyond mere observation to predict and address challenges before they start. 

Construction has always been a complex high-stakes endeavor. But now, thanks to physical AI, we have the tools to help teams build smarter and faster than ever before.