How the OffGrid AI Toolkit Works

Component Architecture:

• Electron Launcher: Main application container and process manager
• Ollama Backend: Local AI inference engine running Gemma models
• Caddy Server: Zero-config web server for mobile access
• Web Dashboard: Browser-based interface with markdown support

How Components Interact:

1. Electron spawns and manages Ollama and Caddy processes
2. Ollama loads AI models from USB into RAM/VRAM
3. Dashboard sends requests to Ollama via localhost:11434
4. Caddy serves dashboard to network devices on port 8000
5. All components run without installation or admin rights

Included Models:

• Gemma3 4B: Fast responses, runs on 8GB RAM
• Gemma3 12B: Best balance of speed and intelligence
• Gemma3 27B: Maximum capability for complex tasks
• Medical Models: Specialized for health guidance
• Vision Models: Analyze images and identify objects

View detailed performance data on our Model Benchmarks page.

Capabilities:

• Natural language understanding and generation
• Code writing and debugging
• Document analysis and summarization
• Image recognition and description
• Medical triage and first-aid guidance
• Technical troubleshooting and repair instructions

Explore our Ready-Made Prompts to get started quickly.

Privacy by Design:

• No Network Requests: System blocks all external connections
• No Telemetry: Zero tracking or usage data collection
• No User Accounts: No registration or authentication required
• No Cloud Storage: All data stays on your USB drive

Security Measures:

• Process isolation with PID verification
• Sanitized markdown rendering to prevent injection
• Path restrictions to USB directory only
• Read-only model files prevent tampering
• Automatic cleanup on shutdown

Learn more about how we protect your privacy on our Privacy Advocates page.

Open Technologies Used:

• Ollama: Local AI inference engine (Apache 2.0)
• Electron: Cross-platform desktop framework (MIT)
• Caddy: Modern web server (Apache 2.0)
• Gemma Models: Google's efficient AI models
• marked.js: Markdown parsing library

Customization Options:

Power users can modify the system:

• Add custom AI models to the models folder
• Modify the dashboard HTML/CSS/JavaScript
• Create custom prompt templates
• Extend with additional tools and scripts

Hidden Developer Mode:

Press Ctrl+Shift+M in the launcher to access developer tools and console output.

Running AI Without Installation:

We use Electron with embedded Ollama and dynamic environment variables that configure paths at runtime, not install time.

Mobile Access Without Apps:

Caddy serves a progressive web app that works in any browser. QR codes eliminate manual IP entry. See our Mobile Access Guide for setup instructions.

Finding the System on Any Network:

Multi-strategy IP detection: automatic network scanning, QR code generation, and fallback to manual entry.

Windows Firewall Issues:

Automatic firewall rule configuration for ports 8000 and 11434 ensures mobile devices can connect.

Process Coordination:

Electron manages child processes with PID tracking and graceful shutdown sequences.