Large Language Models
Dropbear uses large language models for high-level task planning, natural language interaction, and embodied reasoning. An LLM serves as the robot's cognitive layer — translating human instructions into sequences of robot actions.
LLM Integration Architecture
The LLM layer sits above the control stack and interfaces with the robot through a set of callable functions (tools). When you say "go to the kitchen and fetch me a glass of water," the LLM breaks it down:
Task Decomposition Example:
"Fetch me a glass of water"
→ navigate_to("kitchen")
→ locate_object("glass")
→ grasp("glass")
→ navigate_to("sink")
→ fill_glass()
→ navigate_to("user_location")
→ hand_over("glass")
Tool Calling
The robot exposes its capabilities as LLM-callable tools:
| Tool | Description |
|---|---|
| navigate_to(location) | Walk to a named location or coordinates |
| look_at(target) | Point the head camera at a target |
| describe_scene() | Use vision to describe what the robot sees |
| grasp(object_name) | Detect, approach, and grasp a named object |
| deploy_skill(skill_name) | Activate a learned behavior from the marketplace |
| speak(text) | Text-to-speech output through the robot's speaker |
Supported Models
Dropbear's LLM layer is model-agnostic. You can use any provider:
- Cloud — Claude (Anthropic), GPT-4 (OpenAI), Gemini (Google)
- On-edge — Llama 3.x, Phi-3, or any GGUF model running locally on Jetson Orin via llama.cpp
- Hybrid — fast local model for real-time decisions, cloud model for complex planning
Conversational Interface
The robot can engage in natural conversation while performing tasks. It understands context, asks clarifying questions, and reports status. The conversational interface works via the head-mounted microphone and speaker, the Hyperspawn mobile app, or any connected chat platform.
🧠 Active Research
LLM-driven embodied intelligence is an active research area. Our current focus is on reliable task planning with safety constraints — ensuring the robot doesn't attempt actions beyond its physical capabilities.