Some of the most exciting AI breakthroughs never announce themselves with a giant data centre or a supercomputer. They show up quietly, inside the phone already in your pocket — answering faster, working without signal, and keeping your personal data exactly where it belongs: with you. A Bengaluru startup has spent years building the invisible infrastructure that makes that possible, and now its engineering sits at the heart of one of Microsoft’s most important AI platforms.
NimbleEdge, the Bengaluru-based on-device AI infrastructure company, helped Microsoft bring Foundry Local to Android — the framework that lets developers run powerful small language models directly on smartphones, with no cloud round trips required. The collaboration was revealed on stage at Microsoft Ignite 2025 in San Francisco, and Microsoft’s September 2026 update to the platform still names NimbleEdge as a key partner in delivering it.
A phone running a personal on-device AI assistant — the experience NimbleEdge’s infrastructure enables (AI-generated visual based on the company’s official product imagery).
A mini AI server inside your phone
Foundry Local is, in the company’s words, a “mini AI server inside your phone.” It is a background service on the device that lets apps consume local AI through familiar OpenAI-compatible APIs, without each app having to load heavy models on its own. Developers can build agentic apps that reason, respond and collaborate with users in real time — while the intelligence stays on the device.
NimbleEdge architected the hard parts of that vision. The company built the background service that handles robust, long-running small-language-model downloads, manages shared resources across apps, and serves inference over a secure-bound AIDL service with mutual certificate verification — engineered to work consistently across the huge diversity of Android devices, chipsets and operating systems. Its DeliteAI stack works alongside Foundry Local to simplify deployment, capturing in-session context from the user interface, orchestrating agents with prompt templating, tool calling, persistent memory and voice interactions, and keeping performance consistent across hardware.
More Read
In plain language: instead of every app phoning home to a distant server for every smart feature, the smarts live on the phone itself, shared safely between apps, ready in milliseconds.
Why on-device changes everything
Running AI on the device brings three gifts that the cloud simply cannot match. Privacy comes first: sensitive data — health details, financial information, personal conversations — never has to leave the phone. Speed comes second: with no network round trip, responses arrive instantly, even on a patchy connection or none at all. And cost comes third: fewer cloud calls means cheaper apps that work for everyone, not just users on fast networks.
The scale of the bet is already visible. One of India’s largest digital payments platforms became the first enterprise to adopt Foundry Local for Android, and Microsoft says that platform’s app serves more than 618 million users. NimbleEdge and Microsoft’s collaboration is not new, either: the two companies have worked together since the ONNX runtime days, scaling deployments to more than 30 million devices. Microsoft’s Rajat Monga, CVP for AI Frameworks, put it simply: with Foundry Local for Android, built with contributions from NimbleEdge, developers can build real-time, AI-native app experiences with powerful, low-latency intelligence on users’ phones.
From Bengaluru to the world’s phones
NimbleEdge was founded by Varun Khare, its CEO, and Neeraj Poddar, its CTO. Khare’s research journey ran through UC Berkeley, the Max Planck Institute and IIT Kanpur, and his pioneering work on federated learning at OpenMined — the open-source community for privacy-preserving AI — grew into NimbleEdge’s founding idea: that the compute already sitting in billions of consumer devices could carry personalisation without centralising personal data.
Long before the current AI wave, the company was helping Indian consumer internet leaders such as Dream11 and Swiggy run machine learning models directly on users’ devices, delivering real-time, in-session recommendations in food delivery, travel and gaming. That same philosophy now extends beyond phones: the company’s DeliteAI platform is open source, and its Lazrus desktop software brings personal automation — research, digital tasks and cross-platform actions — to laptops and PCs without valuable data leaving the device.
That is the arc of this story: a Bengaluru team that saw, years ahead of the curve, that the future of AI would not only be bigger models in bigger data centres, but smarter software running quietly, privately and instantly on the devices we already carry. With Foundry Local, that vision is now shipping inside a Microsoft platform reaching developers — and users — around the world. NimbleEdge is a beautiful example of Indian deep tech building the infrastructure the whole world runs on, and its moment is only just beginning.
Keep reading: Before the Water Rises: Inside Aurassure, the Bhubaneswar Startup Building India’s Climate Early-Warning Network · One Tonne, One Square Metre: Inside ANSCER’s ATR 1000, the Robot That Lifts, Spins and Hauls · More Industrial Automation & Technology stories
Like stories like this? Add Exceptional Technology as a preferred source on Google to see more of our reporting in your search results and Discover feed.
