Teaching Robots to Use Their Hands: Inside Human Archive’s 500 Hours of India at Work

Karishma Shah
By
Karishma Shah
Co-founder of Exceptional Technology. Karishma leads editorial and operations for exceptional.technology — the showcase of the best of Indian technology and engineering — and co-runs Excep...
Summarize with AI

In November 2025, four friends in their early twenties walked away from UC Berkeley and Stanford with a single conviction: the robots coming next will not be held back by motors or models, but by a lack of something to learn from. Large language models had the whole internet to read — some 15 trillion tokens of text. Robots that must work with their hands had almost nothing. So Raj Patel, Rushil Agarwal, Samay Maini and Shloke Patel started Human Archive, a company building the dataset the physical-AI age is missing — and they started it where the work is: India.

First-person view from Human Archive's capture rig: hands cleaning a window

A first-person view from Human Archive’s capture rig — hands at work in a real Indian workplace. (Official footage: Human Archive)

500 hours, 60 workplaces, 310,000 labelled steps

The company’s calling card is HA-Ego-500, an open-source dataset it has just released: 500 hours of first-person video recorded in more than 60 real work environments — homes, factories, restaurants, commercial laundries, hair salons, warehouses, garment workshops, construction sites and more — carrying over 310,000 structured action annotations. Hands are engaged in 92% of the steps, the company says, and the same tasks appear across many different workplaces — cleaning spans 28 environments, packaging 18 — so a robot trained on the data learns the task, not just one room. The release covers 390 unique task–environment combinations, and 23 of those environments have more than four hours of footage each.

Every clip is broken into steps a few seconds long — the median step lasts about four seconds — each carrying a plain-language description, a task phase, the state of each hand, and the objects visible in the frame. “Wipe the door handle with the cloth in the right hand,” one step reads. It is the kind of dense, fine-grained supervision that robot learning has been asking for: the company says its own policy-training experiments show that increasing annotation density improves both vision-language-action policies and video world-action models.

The rig: seven sensors, one synchronized moment

Video alone, the company argues, is not enough. A robot needs to know how hard to grip, how a wrist turns, how weight shifts — so Human Archive built its own capture hardware. The HA Headset V2 records 4K video at 30 frames per second alongside 1080p at 60, with dual stereo depth cameras, a mono RGB camera, infrared dot projection and a nine-axis motion sensor, the company says. Around it sit wrist cameras, body motion sensors, and a tactile glove with 162 fibre sensing points that measures the force of a grip. More than 1,000 headsets are deployed, the company says, and over 50 different devices collect different data points — all synchronized, so a frame of video, a depth map and a squeeze of the glove line up exactly.

From raw footage to robot-ready labels

Every released hour passes through a seven-stage pipeline. Custom multi-camera hardware records people at work; a proprietary quality model cuts the raw sessions into clips and filters out anything where hands leave the frame or the task is not useful; trained annotators verify the clip boundaries; every video is anonymized, with bystander faces blurred; then two model passes build an object inventory and densely segment the clip into steps, before a trained annotator verifies every step and an independent reviewer approves it. More than 100 trained annotators work at the company’s quality-control centre, the company says.

The company puts its labels to the test against frontier AI models. Given a 30-second clip, its own annotations score a mean F1 of 0.771 on finding actions at roughly the right time — ahead of Gemini 3.6 Flash at 0.722, GPT-5 at 0.677 and Kimi K3 at 0.652, the company says — and on exact action boundaries, the human-verified labels score 0.474 against 0.230 for the best model, roughly double, while on hand state they lead 0.647 to 0.402. Two annotators labelling the same footage independently agree with a score of 0.763, the company says — ahead of the figures reported for the Charades and MultiTHUMOS datasets.

Why India is where the data lives

India sits at the centre of the operation. The company has signed more than 120 partnerships across hotels, restaurants, quick-commerce platforms, construction sites and factories, the company says, and Asia — India above all — is its largest data-collection hub. It has collected tens of thousands of hours of footage so far, with millions as the goal. The ambition is backed by an $8.2 million seed round led by Wing Venture Capital and NVP Capital, with Y Combinator and angel investors from OpenAI, Nvidia, Google and Meta, the company says — a bet that the scarcest resource in robotics is not compute, but real human skill, recorded well.

Consent is built into the model: when a worker arrives at a home, the customer is offered a choice through the app — a discounted price in exchange for consenting to data collection, or the full price for an unrecorded visit, the company says. All collected data is anonymized, faces are blurred, and the company’s commercial contracts comply with India’s Digital Personal Data Protection Act, the company says.

Human Archive is not building a robot. It is building the training corpus that makes robots possible — and with HA-Ego-500 open to every lab and researcher, 500 hours of India at work are now part of the commons the whole field learns from. The next hardware is already coming: a stereo and six-camera rig with a 180-degree field of view, capturing the whole scene and both hands at once.

Keep reading: The Chip That Thinks at the Sensor: Inside Sensesemi, the Bengaluru Startup Building India’s Edge-AI Silicon · The AI Workforce Behind India’s Supply Chains: Inside Enmovil’s CADDIE · More Industrial Automation & Technology stories

Like stories like this? Add Exceptional Technology as a preferred source on Google to see more of our reporting in your search results and Discover feed.

Share This Article
Co-founder of Exceptional Technology. Karishma leads editorial and operations for exceptional.technology — the showcase of the best of Indian technology and engineering — and co-runs Excep Technology, a Pune-based creative and technology studio.