TetraHome/Tetra-AI-HA-Integration
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

first commit

Browse Source
This commit is contained in:
techartdev
2026-02-20 13:07:28 +02:00
commit e9cd5e1fff
24 changed files with 3083 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
IMPLEMENTATION_PLAN.md
+355
View File
@@ -0,0 +1,355 @@
# OpenClaw Integration — Implementation Plan
## 1. Overview
This document describes the architecture and phased implementation plan for the
**OpenClaw Integration** — a native Home Assistant custom integration that acts as
a satellite companion to the existing **OpenClaw Assistant** addon.
### Goals
| # | Goal | Priority |
|---|------|----------|
| 1 | Chat card (text, streaming, files, voice) embedded in HA dashboard | P0 |
| 2 | Sensor / binary-sensor entities for status, model, sessions | P0 |
| 3 | Bidirectional addon ↔ integration communication (zero manual setup) | P0 |
| 4 | Native HA conversation agent (Assist / Voice PE) | P1 |
| 5 | Service calls & events for automations | P1 |
| 6 | Media player entity for TTS output | P2 |
---
## 2. Existing Addon — Key Facts
| Property | Value |
|----------|-------|
| Slug | `openclaw_assistant_dev` |
| Gateway port | `18789` (configurable via `gateway_port`) |
| Auth | Token-based (`gateway.auth.token` in `openclaw.json`) |
| Config file | `/config/.openclaw/openclaw.json` (addon container) |
| OpenAI-compatible endpoint | `POST /v1/chat/completions` (opt-in via `enable_openai_api`) |
| Gateway bind | `127.0.0.1` (loopback) or `0.0.0.0` (lan) |
| Node process | `openclaw gateway run` |
| Ingress port | `48099` (nginx) |
---
## 3. Architecture
```
┌──────────────────────────────────────────────────────────┐
│ Home Assistant Core │
│ │
│ ┌─────────────┐ ┌──────────────┐ ┌────────────────┐ │
│ │ Sensors / │ │ Conversation │ │ Services / │ │
│ │ Binary │ │ Agent │ │ Events │ │
│ │ Sensors │ │ (Assist/VPE) │ │ │ │
│ └──────┬───────┘ └──────┬───────┘ └───────┬────────┘ │
│ │ │ │ │
│ └────────┬────────┴───────────────────┘ │
│ │ │
│ ┌───────▼────────┐ │
│ │ OpenClawAPI │ (api.py — HTTP client) │
│ │ Client │ │
│ └───────┬────────┘ │
│ │ HTTP / SSE │
├──────────────────┼────────────────────────────────────────┤
│ │ │
│ ┌───────────────▼──────────────────────────────┐ │
│ │ OpenClaw Gateway (addon) │ │
│ │ │ │
│ │ /v1/chat/completions (SSE streaming) │ │
│ │ /api/status (JSON) │ │
│ │ /api/sessions (JSON) │ │
│ │ /api/models (JSON) │ │
│ └──────────────────────────────────────────────┘ │
│ │
│ OpenClaw Assistant Addon Container │
└───────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────┐
│ Lovelace Dashboard │
│ │
│ ┌────────────────────────────────────────┐ │
│ │ openclaw-chat-card │ │
│ │ ┌──────────────────────────────────┐ │ │
│ │ │ Message history (markdown) │ │ │
│ │ │ Typing indicator │ │ │
│ │ │ File / image attachments │ │ │
│ │ │ Voice input button │ │ │
│ │ │ Voice mode toggle │ │ │
│ │ └──────────────────────────────────┘ │ │
│ └────────────────────────────────────────┘ │
│ ▲ │
│ │ HA WebSocket API │
│ │ (openclaw.send_message service) │
│ │ + event subscriptions │
└───────┼─────────────────────────────────────┘
│
▼
Home Assistant Core (services / events)
```
### 3.1 Auto-Discovery (Zero-Config)
The integration discovers the addon with **no manual setup** using two mechanisms:
1. **Supervisor API** — at config-flow time the integration calls
`GET /addons/openclaw_assistant_dev/info` via the HA Supervisor client.
This gives us: addon state, network ports, options.
2. **Shared filesystem** — the addon mounts `/addon_configs/<slug>` which maps
to `/config` inside the container. The integration can read
`openclaw.json` to get the gateway auth token, mode, and port.
**Net effect:** the user clicks "Add Integration → OpenClaw" and everything
connects automatically — no token or URL entry required.
### 3.2 Communication Protocol
| Direction | Transport | Endpoint / Mechanism |
|-----------|-----------|----------------------|
| Integration → Chat | HTTP POST + SSE stream | `/v1/chat/completions` (OpenAI compat) |
| Integration → Status | HTTP GET (polled every 30 s) | `/api/status` |
| Integration → Sessions | HTTP GET (polled every 60 s) | `/api/sessions` |
| Integration → Models | HTTP GET (on startup + hourly) | `/v1/models` |
| Addon → HA (events) | HA REST API (long-lived token) | `POST /api/events/openclaw_*` |
| Frontend → Integration | HA WebSocket API | service calls + event subscriptions |
### 3.3 Latency Budget (Voice Mode)
| Step | Target | Notes |
|------|--------|-------|
| STT (browser/HA) | ≤ 500 ms | WebSpeech API or Whisper via HA |
| Network to OpenClaw | ≤ 100 ms | localhost or LAN |
| OpenClaw inference | ≤ 1500 ms | First token via SSE stream |
| TTS | ≤ 500 ms | HA `tts.speak` or browser SpeechSynthesis |
| **Total** | **≤ 2600 ms** | Under 3 s target |
---
## 4. File Structure
```
openclaw_integration/
├── IMPLEMENTATION_PLAN.md ← this file
├── hacs.json ← HACS metadata
├── README.md
│
├── custom_components/
│ └── openclaw/
│ ├── __init__.py ← integration setup, coordinator
│ ├── manifest.json ← HA integration manifest
│ ├── config_flow.py ← auto-discovery config flow
│ ├── const.py ← constants, defaults
│ ├── api.py ← OpenClaw gateway HTTP client
│ ├── coordinator.py ← DataUpdateCoordinator for polling
│ ├── sensor.py ← sensor entities
│ ├── binary_sensor.py ← binary sensor entities
│ ├── conversation.py ← native conversation agent
│ ├── services.py ← service call handlers
│ ├── services.yaml ← service definitions
│ ├── strings.json ← English UI strings
│ └── translations/
│ └── en.json ← English translations
│
└── www/
└── openclaw-chat-card.js ← Lovelace custom card (Lit)
```
---
## 5. Phased Implementation
### Phase 1 — Foundation (MVP)
**Goal:** Integration installs, auto-discovers addon, exposes sensors.
| Task | File(s) | Description |
|------|---------|-------------|
| 1.1 | `manifest.json` | Integration metadata, dependencies |
| 1.2 | `const.py` | Domain, default ports, config keys |
| 1.3 | `api.py` | `OpenClawApiClient` — HTTP client to gateway |
| 1.4 | `config_flow.py` | Auto-discovery via Supervisor + shared config |
| 1.5 | `coordinator.py` | `DataUpdateCoordinator` polling status/sessions/model |
| 1.6 | `__init__.py` | Integration setup, platforms, coordinator init |
| 1.7 | `sensor.py` | `openclaw_status`, `openclaw_last_activity`, `openclaw_session_count`, `openclaw_model` |
| 1.8 | `binary_sensor.py` | `openclaw_connected` |
| 1.9 | `strings.json`, `translations/en.json` | UI text |
| 1.10 | `hacs.json` | HACS configuration |
**Acceptance criteria:**
- `Add Integration → OpenClaw` works with zero config
- All 5 entities appear and update
- Integration reconnects gracefully when addon restarts
### Phase 2 — Services, Events & Conversation Agent
**Goal:** Automations can send/receive messages; Assist pipeline works.
| Task | File(s) | Description |
|------|---------|-------------|
| 2.1 | `services.yaml`, `services.py` | `openclaw.send_message`, `openclaw.clear_history` |
| 2.2 | `__init__.py` (update) | Fire `openclaw_message_received` HA event on response |
| 2.3 | `conversation.py` | Register as `conversation` agent |
| 2.4 | `api.py` (update) | SSE streaming support (`async for` over response chunks) |
| 2.5 | Testing | Verify Assist Text/Voice pipeline end-to-end |
**Acceptance criteria:**
- `openclaw.send_message` callable from automations
- `openclaw_message_received` event fires and can trigger automations
- OpenClaw appears in Assist agent picker
- Voice PE works: wake word → STT → OpenClaw → TTS → speaker
### Phase 3 — Chat Card (Frontend)
**Goal:** Full chat UI as a Lovelace custom card.
| Task | File(s) | Description |
|------|---------|-------------|
| 3.1 | `openclaw-chat-card.js` | Base card shell (Lit element, card config) |
| 3.2 | (continued) | Message history with timestamps, markdown rendering |
| 3.3 | (continued) | Streaming response display (typing indicator) |
| 3.4 | (continued) | File/image attachment support (upload via service) |
| 3.5 | (continued) | Voice input (WebSpeech/MediaRecorder → send audio) |
| 3.6 | (continued) | Voice mode toggle (continuous listen → auto-respond with TTS) |
| 3.7 | `__init__.py` (update) | Register card as Lovelace resource |
| 3.8 | Card editor | Visual card configuration editor |
**Acceptance criteria:**
- Card renders chat history with markdown
- Real-time streaming of AI responses
- File upload works in both directions
- Voice input captures and sends audio
- Voice mode maintains continuous conversation
### Phase 4 — Media Player & Polish
**Goal:** Native TTS output, wake word integration, production hardening.
| Task | File(s) | Description |
|------|---------|-------------|
| 4.1 | `media_player.py` | Media player entity for TTS output routing |
| 4.2 | | Wake word detection integration |
| 4.3 | | WebSocket connection management (token refresh, reconnect) |
| 4.4 | | Error handling, rate limiting, connection pooling |
| 4.5 | | Performance profiling (voice latency budget) |
| 4.6 | | Documentation and HACS submission |
---
## 6. Entity Reference
### Sensors
| Entity ID | Class | State | Attributes |
|-----------|-------|-------|------------|
| `sensor.openclaw_status` | — | `online` / `offline` / `processing` | `gateway_version`, `uptime` |
| `sensor.openclaw_last_activity` | `timestamp` | ISO 8601 datetime | `last_message_preview` |
| `sensor.openclaw_session_count` | — | integer (active count) | `sessions` (list of IDs) |
| `sensor.openclaw_model` | — | model name string | `provider`, `context_window` |
### Binary Sensors
| Entity ID | Class | State |
|-----------|-------|-------|
| `binary_sensor.openclaw_connected` | `connectivity` | `on` / `off` |
### Services
| Service | Fields | Description |
|---------|--------|-------------|
| `openclaw.send_message` | `message` (str), `session_id` (str, optional), `attachments` (list, optional) | Send a message to OpenClaw |
| `openclaw.clear_history` | `session_id` (str, optional) | Clear conversation history |
### Events
| Event | Data | Description |
|-------|------|-------------|
| `openclaw_message_received` | `message`, `session_id`, `model`, `timestamp` | Fired when OpenClaw sends a response |
---
## 7. Key Technical Decisions
### 7.1 Why HTTP+SSE instead of WebSocket?
- The addon already exposes an **OpenAI-compatible** HTTP endpoint with SSE streaming
- No additional gateway code needed on the addon side
- SSE is simpler to manage (no bi-directional state, automatic reconnect)
- WebSocket can be added later for real-time push features
### 7.2 Why Supervisor API for discovery?
- Available in all HAOS / Supervised installs (target audience)
- Provides addon state, network config, and options without filesystem access
- Filesystem fallback (`/addon_configs/`) handles edge cases
### 7.3 Why register as a conversation agent?
- Native Assist integration = works with Voice PE, S3 satellite, etc.
- No need for third-party HACS integrations (Extended OpenAI Conversation)
- Event-driven: HA handles STT/TTS pipeline, we just process text
### 7.4 Frontend card communication
- Card uses HA WebSocket API (already authenticated)
- Calls `openclaw.send_message` service
- Subscribes to `openclaw_message_received` events
- No separate WebSocket to gateway needed (avoids CORS, auth issues)
---
## 8. Challenges & Mitigations
| Challenge | Mitigation |
|-----------|------------|
| Gateway not reachable (loopback bind) | Integration runs in same host; `127.0.0.1` works. Document `lan` mode for remote setups. |
| Token rotation / mismatch | Re-read `openclaw.json` on connection failure; config flow stores token in HA config entry |
| Voice latency > 3s | Use SSE streaming (first token fast), browser-side SpeechSynthesis for TTS, Whisper locally for STT |
| Addon not installed | Config flow gracefully fails with a message directing user to install the addon first |
| HACS distribution | Standard HACS custom integration + Lovelace card resources |
| Large file uploads | Chunk uploads through service call; gateway handles multipart |
---
## 9. Dependencies
### Python (integration)
| Package | Purpose | In HA? |
|---------|---------|--------|
| `aiohttp` | HTTP client for gateway API | ✅ built-in |
| `homeassistant` | HA core APIs | ✅ built-in |
### JavaScript (frontend card)
| Library | Purpose | Bundled? |
|---------|---------|----------|
| `lit` | Web component framework | ✅ available via HA |
| `marked` | Markdown rendering | Bundle in card JS |
No additional pip dependencies required — the integration uses only HA built-ins.
---
## 10. Testing Strategy
| Level | Tool | Coverage |
|-------|------|----------|
| Unit | `pytest` + `pytest-homeassistant-custom-component` | API client, coordinator, config flow |
| Integration | HA dev container with addon mock | End-to-end entity updates, service calls |
| Frontend | Manual + Playwright | Card rendering, streaming, voice |
---
## 11. Timeline Estimate
| Phase | Duration | Milestone |
|-------|----------|-----------|
| Phase 1 | 1–2 weeks | Sensors visible, auto-discovery works |
| Phase 2 | 1–2 weeks | Conversation agent in Assist, services/events |
| Phase 3 | 2–3 weeks | Chat card MVP (text + streaming + files) |
| Phase 4 | 2–3 weeks | Voice mode, media player, polish |
**Total estimated: 6–10 weeks to full feature set.**