Production ready toolkit to run AI locally

Run frontier LLMs and VLMs with day-0 model support across GPU, NPU, and CPU, with comprehensive runtime coverage for PC (Python/C++), mobile (Android & iOS), and Linux/IoT (Arm64 & x86 Docker). Supporting OpenAI GPT-OSS, IBM Granite-4, Qwen-3-VL, Gemma-3n, Ministral-3, and more.

Low-latency AI engine for mobile devices & wearables

An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.

[ICLR 2019] ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Declarative way to run AI models in React Native on device, powered by ExecuTorch.

On-device LLM execution in React Native with Vercel AI SDK compatibility

Official PyTorch implementation of "EdgeSAM: Prompt-In-the-Loop Distillation for On-Device Deployment of SAM"

离线版设备端人脸识别 动作活体、炫彩活体、近红外双目活体检测 以及1:N M:N 人脸搜索算法SDK 封装；全程可开飞行模式不用联网 🧒 on_device Face Recognition 、 Liveness detection and 1:N & M:N Face Search SDK

NativeMind: Your fully private, open-source, on-device AI assistant

TinyChatEngine: On-Device LLM Inference Library

Example apps for Foundation Models Framework in iOS 26 and macOS 26

[CVPR 2025] Official PyTorch implementation of "EdgeTAM: On-Device Track Anything Model"

Optimized Whisper models for streaming and on-device use

Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.

Sub-Millisecond RAG on Apple Silicon. No Server. No API. One File. Pure Swift

On-device Neural Engine

Android Input Method Editor (IME) based on Whisper

Android Voice Activity Detection (VAD) library. Supports WebRTC VAD GMM, Silero VAD DNN, Yamnet VAD DNN models.

[ECCV 2018] AMC: AutoML for Model Compression and Acceleration on Mobile Devices