From 726ceb3ef6217e9178c861f5f002cc3f47c5d1ca Mon Sep 17 00:00:00 2001 From: techartdev Date: Fri, 20 Feb 2026 21:52:53 +0200 Subject: [PATCH] readme and version floor update --- CHANGELOG.md | 3 + IMPLEMENTATION_PLAN.md | 420 ++++++++++++----------------------------- hacs.json | 2 +- 3 files changed, 122 insertions(+), 303 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b2ba95b..e4bb1f3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,9 @@ All notable changes to the OpenClaw Home Assistant Integration will be documente ### Documentation - Expanded README voice documentation with practical guidance for `voice_provider` (`browser` vs `assist_stt`) and provider-specific troubleshooting. +### Changed +- Lowered declared HACS minimum Home Assistant Core version to `2025.1.0` after compatibility hardening updates. + ## [0.1.33] - 2026-02-20 ### Fixed diff --git a/IMPLEMENTATION_PLAN.md b/IMPLEMENTATION_PLAN.md index 51200db..db218fe 100644 --- a/IMPLEMENTATION_PLAN.md +++ b/IMPLEMENTATION_PLAN.md @@ -1,355 +1,171 @@ # OpenClaw Integration — Implementation Plan -## 1. Overview +## 1. Scope and current state -This document describes the architecture and phased implementation plan for the -**OpenClaw Integration** — a native Home Assistant custom integration that acts as -a satellite companion to the existing **OpenClaw Assistant** addon. +This plan reflects the actual implementation status as of release `0.1.34`. -### Goals +### Project goals -| # | Goal | Priority | -|---|------|----------| -| 1 | Chat card (text, streaming, files, voice) embedded in HA dashboard | P0 | -| 2 | Sensor / binary-sensor entities for status, model, sessions | P0 | -| 3 | Bidirectional addon ↔ integration communication (zero manual setup) | P0 | -| 4 | Native HA conversation agent (Assist / Voice PE) | P1 | -| 5 | Service calls & events for automations | P1 | -| 6 | Media player entity for TTS output | P2 | +| # | Goal | Priority | Status | +|---|------|----------|--------| +| 1 | Stable HA integration with addon auto-discovery | P0 | ✅ Done | +| 2 | Sensors + binary sensor for gateway visibility | P0 | ✅ Done | +| 3 | Native Assist conversation agent | P0 | ✅ Done | +| 4 | Lovelace chat card (text + voice) | P0 | ✅ Done | +| 5 | Voice provider choice (`browser` / `assist_stt`) | P0 | ✅ Done | +| 6 | Production polish and broad compatibility hardening | P1 | 🚧 In progress | +| 7 | Optional native media player/TTS routing entity | P2 | ⏳ Not started | --- -## 2. Existing Addon — Key Facts +## 2. Architecture (implemented) -| Property | Value | -|----------|-------| -| Slug | `openclaw_assistant_dev` | -| Gateway port | `18789` (configurable via `gateway_port`) | -| Auth | Token-based (`gateway.auth.token` in `openclaw.json`) | -| Config file | `/config/.openclaw/openclaw.json` (addon container) | -| OpenAI-compatible endpoint | `POST /v1/chat/completions` (opt-in via `enable_openai_api`) | -| Gateway bind | `127.0.0.1` (loopback) or `0.0.0.0` (lan) | -| Node process | `openclaw gateway run` | -| Ingress port | `48099` (nginx) | +### 2.1 Core backend flow + +- Integration communicates with OpenClaw gateway over HTTP. +- Primary chat endpoint: `POST /v1/chat/completions` (OpenAI-compatible). +- Coordinator uses lightweight connectivity checks and model probing with graceful fallback. +- Services and conversation agent share response extraction logic for modern OpenAI-compatible payloads. + +### 2.2 Frontend flow + +- Card sends chat requests via HA services/websocket (`openclaw.send_message`). +- Card receives replies through HA event subscription (`openclaw_message_received`). +- Backend keeps in-memory history and exposes websocket history sync (`openclaw/get_history`). +- Settings endpoint (`openclaw/get_settings`) provides integration-level card options. + +### 2.3 Voice modes + +- `browser` provider: + - Uses `SpeechRecognition` / `webkitSpeechRecognition`. + - Supports manual mic and continuous voice mode (wake-word flow). +- `assist_stt` provider: + - Captures local mic audio and sends to HA STT API (`/api/stt/`). + - Manual one-shot transcription flow. + - Provider metadata negotiation for language/sample rate/channels to reduce `415` failures. --- -## 3. Architecture +## 3. Repository map (current) ``` -┌──────────────────────────────────────────────────────────┐ -│ Home Assistant Core │ -│ │ -│ ┌─────────────┐ ┌──────────────┐ ┌────────────────┐ │ -│ │ Sensors / │ │ Conversation │ │ Services / │ │ -│ │ Binary │ │ Agent │ │ Events │ │ -│ │ Sensors │ │ (Assist/VPE) │ │ │ │ -│ └──────┬───────┘ └──────┬───────┘ └───────┬────────┘ │ -│ │ │ │ │ -│ └────────┬────────┴───────────────────┘ │ -│ │ │ -│ ┌───────▼────────┐ │ -│ │ OpenClawAPI │ (api.py — HTTP client) │ -│ │ Client │ │ -│ └───────┬────────┘ │ -│ │ HTTP / SSE │ -├──────────────────┼────────────────────────────────────────┤ -│ │ │ -│ ┌───────────────▼──────────────────────────────┐ │ -│ │ OpenClaw Gateway (addon) │ │ -│ │ │ │ -│ │ /v1/chat/completions (SSE streaming) │ │ -│ │ /api/status (JSON) │ │ -│ │ /api/sessions (JSON) │ │ -│ │ /api/models (JSON) │ │ -│ └──────────────────────────────────────────────┘ │ -│ │ -│ OpenClaw Assistant Addon Container │ -└───────────────────────────────────────────────────────────┘ - -┌─────────────────────────────────────────────┐ -│ Lovelace Dashboard │ -│ │ -│ ┌────────────────────────────────────────┐ │ -│ │ openclaw-chat-card │ │ -│ │ ┌──────────────────────────────────┐ │ │ -│ │ │ Message history (markdown) │ │ │ -│ │ │ Typing indicator │ │ │ -│ │ │ File / image attachments │ │ │ -│ │ │ Voice input button │ │ │ -│ │ │ Voice mode toggle │ │ │ -│ │ └──────────────────────────────────┘ │ │ -│ └────────────────────────────────────────┘ │ -│ ▲ │ -│ │ HA WebSocket API │ -│ │ (openclaw.send_message service) │ -│ │ + event subscriptions │ -└───────┼─────────────────────────────────────┘ - │ - ▼ - Home Assistant Core (services / events) -``` - -### 3.1 Auto-Discovery (Zero-Config) - -The integration discovers the addon with **no manual setup** using two mechanisms: - -1. **Supervisor API** — at config-flow time the integration calls - `GET /addons/openclaw_assistant_dev/info` via the HA Supervisor client. - This gives us: addon state, network ports, options. - -2. **Shared filesystem** — the addon mounts `/addon_configs/` which maps - to `/config` inside the container. The integration can read - `openclaw.json` to get the gateway auth token, mode, and port. - -**Net effect:** the user clicks "Add Integration → OpenClaw" and everything -connects automatically — no token or URL entry required. - -### 3.2 Communication Protocol - -| Direction | Transport | Endpoint / Mechanism | -|-----------|-----------|----------------------| -| Integration → Chat | HTTP POST + SSE stream | `/v1/chat/completions` (OpenAI compat) | -| Integration → Status | HTTP GET (polled every 30 s) | `/api/status` | -| Integration → Sessions | HTTP GET (polled every 60 s) | `/api/sessions` | -| Integration → Models | HTTP GET (on startup + hourly) | `/v1/models` | -| Addon → HA (events) | HA REST API (long-lived token) | `POST /api/events/openclaw_*` | -| Frontend → Integration | HA WebSocket API | service calls + event subscriptions | - -### 3.3 Latency Budget (Voice Mode) - -| Step | Target | Notes | -|------|--------|-------| -| STT (browser/HA) | ≤ 500 ms | WebSpeech API or Whisper via HA | -| Network to OpenClaw | ≤ 100 ms | localhost or LAN | -| OpenClaw inference | ≤ 1500 ms | First token via SSE stream | -| TTS | ≤ 500 ms | HA `tts.speak` or browser SpeechSynthesis | -| **Total** | **≤ 2600 ms** | Under 3 s target | - ---- - -## 4. File Structure - -``` -openclaw_integration/ -├── IMPLEMENTATION_PLAN.md ← this file -├── hacs.json ← HACS metadata +OpenClawHomeAssistantIntegration/ +├── IMPLEMENTATION_PLAN.md +├── CHANGELOG.md ├── README.md -│ +├── hacs.json ├── custom_components/ │ └── openclaw/ -│ ├── __init__.py ← integration setup, coordinator -│ ├── manifest.json ← HA integration manifest -│ ├── config_flow.py ← auto-discovery config flow -│ ├── const.py ← constants, defaults -│ ├── api.py ← OpenClaw gateway HTTP client -│ ├── coordinator.py ← DataUpdateCoordinator for polling -│ ├── sensor.py ← sensor entities -│ ├── binary_sensor.py ← binary sensor entities -│ ├── conversation.py ← native conversation agent -│ ├── services.py ← service call handlers -│ ├── services.yaml ← service definitions -│ ├── strings.json ← English UI strings -│ └── translations/ -│ └── en.json ← English translations -│ -└── www/ - └── openclaw-chat-card.js ← Lovelace custom card (Lit) +│ ├── __init__.py +│ ├── api.py +│ ├── binary_sensor.py +│ ├── config_flow.py +│ ├── const.py +│ ├── conversation.py +│ ├── coordinator.py +│ ├── exposure.py +│ ├── manifest.json +│ ├── sensor.py +│ ├── services.yaml +│ ├── strings.json +│ ├── translations/en.json +│ └── www/openclaw-chat-card.js +└── www/openclaw-chat-card.js ``` --- -## 5. Phased Implementation +## 4. Implemented milestones -### Phase 1 — Foundation (MVP) +### Milestone A — Integration foundation -**Goal:** Integration installs, auto-discovers addon, exposes sensors. +- Config flow and options flow. +- Gateway API client and coordinator. +- Sensor + binary sensor platforms. +- HACS-ready packaging. -| Task | File(s) | Description | -|------|---------|-------------| -| 1.1 | `manifest.json` | Integration metadata, dependencies | -| 1.2 | `const.py` | Domain, default ports, config keys | -| 1.3 | `api.py` | `OpenClawApiClient` — HTTP client to gateway | -| 1.4 | `config_flow.py` | Auto-discovery via Supervisor + shared config | -| 1.5 | `coordinator.py` | `DataUpdateCoordinator` polling status/sessions/model | -| 1.6 | `__init__.py` | Integration setup, platforms, coordinator init | -| 1.7 | `sensor.py` | `openclaw_status`, `openclaw_last_activity`, `openclaw_session_count`, `openclaw_model` | -| 1.8 | `binary_sensor.py` | `openclaw_connected` | -| 1.9 | `strings.json`, `translations/en.json` | UI text | -| 1.10 | `hacs.json` | HACS configuration | +### Milestone B — Assist + automations -**Acceptance criteria:** -- `Add Integration → OpenClaw` works with zero config -- All 5 entities appear and update -- Integration reconnects gracefully when addon restarts +- Conversation agent integration. +- Services: `openclaw.send_message`, `openclaw.clear_history`. +- Event emission: `openclaw_message_received`. +- Compatibility fix for intent response handling across HA versions. -### Phase 2 — Services, Events & Conversation Agent +### Milestone C — Chat card reliability -**Goal:** Automations can send/receive messages; Assist pipeline works. +- Auto resource registration and cleanup of duplicate/legacy card resources. +- Versioned card URL strategy to avoid stale cache issues. +- Robust response parsing for multiple OpenAI-compatible response shapes. +- Backend chat history sync to restore UI state after navigation/reload. -| Task | File(s) | Description | -|------|---------|-------------| -| 2.1 | `services.yaml`, `services.py` | `openclaw.send_message`, `openclaw.clear_history` | -| 2.2 | `__init__.py` (update) | Fire `openclaw_message_received` HA event on response | -| 2.3 | `conversation.py` | Register as `conversation` agent | -| 2.4 | `api.py` (update) | SSE streaming support (`async for` over response chunks) | -| 2.5 | Testing | Verify Assist Text/Voice pipeline end-to-end | +### Milestone D — Voice hardening -**Acceptance criteria:** -- `openclaw.send_message` callable from automations -- `openclaw_message_received` event fires and can trigger automations -- OpenClaw appears in Assist agent picker -- Voice PE works: wake word → STT → OpenClaw → TTS → speaker - -### Phase 3 — Chat Card (Frontend) - -**Goal:** Full chat UI as a Lovelace custom card. - -| Task | File(s) | Description | -|------|---------|-------------| -| 3.1 | `openclaw-chat-card.js` | Base card shell (Lit element, card config) | -| 3.2 | (continued) | Message history with timestamps, markdown rendering | -| 3.3 | (continued) | Streaming response display (typing indicator) | -| 3.4 | (continued) | File/image attachment support (upload via service) | -| 3.5 | (continued) | Voice input (WebSpeech/MediaRecorder → send audio) | -| 3.6 | (continued) | Voice mode toggle (continuous listen → auto-respond with TTS) | -| 3.7 | `__init__.py` (update) | Register card as Lovelace resource | -| 3.8 | Card editor | Visual card configuration editor | - -**Acceptance criteria:** -- Card renders chat history with markdown -- Real-time streaming of AI responses -- File upload works in both directions -- Voice input captures and sends audio -- Voice mode maintains continuous conversation - -### Phase 4 — Media Player & Polish - -**Goal:** Native TTS output, wake word integration, production hardening. - -| Task | File(s) | Description | -|------|---------|-------------| -| 4.1 | `media_player.py` | Media player entity for TTS output routing | -| 4.2 | | Wake word detection integration | -| 4.3 | | WebSocket connection management (token refresh, reconnect) | -| 4.4 | | Error handling, rate limiting, connection pooling | -| 4.5 | | Performance profiling (voice latency budget) | -| 4.6 | | Documentation and HACS submission | +- Wake word and always voice mode options. +- Brave guard with explicit allow override option. +- Language normalization and preferred Assist pipeline language handling. +- Improved TTS voice/language selection. +- Multi-pending response handling to prevent stuck typing state. +- `assist_stt` provider with negotiated STT metadata. +- `AudioWorkletNode` capture first, with `ScriptProcessorNode` fallback. --- -## 6. Entity Reference +## 5. Current configuration surface -### Sensors +### Integration options -| Entity ID | Class | State | Attributes | -|-----------|-------|-------|------------| -| `sensor.openclaw_status` | — | `online` / `offline` / `processing` | `gateway_version`, `uptime` | -| `sensor.openclaw_last_activity` | `timestamp` | ISO 8601 datetime | `last_message_preview` | -| `sensor.openclaw_session_count` | — | integer (active count) | `sessions` (list of IDs) | -| `sensor.openclaw_model` | — | model name string | `provider`, `context_window` | +- Prompt/context behavior options. +- Tool-call execution toggle. +- Wake word + always voice mode. +- Brave web speech override. +- Voice provider selector (`browser` or `assist_stt`). -### Binary Sensors +### Card-level behavior -| Entity ID | Class | State | -|-----------|-------|-------| -| `binary_sensor.openclaw_connected` | `connectivity` | `on` / `off` | - -### Services - -| Service | Fields | Description | -|---------|--------|-------------| -| `openclaw.send_message` | `message` (str), `session_id` (str, optional), `attachments` (list, optional) | Send a message to OpenClaw | -| `openclaw.clear_history` | `session_id` (str, optional) | Clear conversation history | - -### Events - -| Event | Data | Description | -|-------|------|-------------| -| `openclaw_message_received` | `message`, `session_id`, `model`, `timestamp` | Fired when OpenClaw sends a response | +- Can consume integration settings from websocket settings endpoint. +- Optional card config overrides remain available. --- -## 7. Key Technical Decisions +## 6. Remaining roadmap (next iterations) -### 7.1 Why HTTP+SSE instead of WebSocket? +### R1 — Voice provider validation and UX polish (next) -- The addon already exposes an **OpenAI-compatible** HTTP endpoint with SSE streaming -- No additional gateway code needed on the addon side -- SSE is simpler to manage (no bi-directional state, automatic reconnect) -- WebSocket can be added later for real-time push features +1. Validate `assist_stt` against multiple HA STT providers/languages. +2. Improve surfaced error details when `/api/stt/` returns non-200. +3. Add optional manual recording duration setting for `assist_stt`. +4. Add provider-specific status text for easier user troubleshooting. -### 7.2 Why Supervisor API for discovery? +### R2 — Compatibility and resilience -- Available in all HAOS / Supervised installs (target audience) -- Provides addon state, network config, and options without filesystem access -- Filesystem fallback (`/addon_configs/`) handles edge cases +1. Continue hardening for older/newer HA Core API differences. +2. Add broader runtime checks for changed Assist pipeline payload shapes. +3. Expand fallback behavior when pipeline metadata is unavailable. -### 7.3 Why register as a conversation agent? +### R3 — Optional enhancements (after stabilization) -- Native Assist integration = works with Voice PE, S3 satellite, etc. -- No need for third-party HACS integrations (Extended OpenAI Conversation) -- Event-driven: HA handles STT/TTS pipeline, we just process text - -### 7.4 Frontend card communication - -- Card uses HA WebSocket API (already authenticated) -- Calls `openclaw.send_message` service -- Subscribes to `openclaw_message_received` events -- No separate WebSocket to gateway needed (avoids CORS, auth issues) +1. Evaluate media-player based TTS routing entity. +2. Explore optional continuous flow for HA STT/TTS pipeline mode. +3. Add automated tests around settings websocket payload and card voice state transitions. --- -## 8. Challenges & Mitigations +## 7. Verification checklist before each release -| Challenge | Mitigation | -|-----------|------------| -| Gateway not reachable (loopback bind) | Integration runs in same host; `127.0.0.1` works. Document `lan` mode for remote setups. | -| Token rotation / mismatch | Re-read `openclaw.json` on connection failure; config flow stores token in HA config entry | -| Voice latency > 3s | Use SSE streaming (first token fast), browser-side SpeechSynthesis for TTS, Whisper locally for STT | -| Addon not installed | Config flow gracefully fails with a message directing user to install the addon first | -| HACS distribution | Standard HACS custom integration + Lovelace card resources | -| Large file uploads | Chunk uploads through service call; gateway handles multipart | +1. Bump and sync versions in: + - `custom_components/openclaw/manifest.json` + - `custom_components/openclaw/__init__.py` (`_CARD_URL`) + - `www/openclaw-chat-card.js` loader shim +2. Restart HA and hard-refresh dashboard. +3. Confirm browser console reports the expected card version. +4. Validate both providers: + - `browser`: manual + continuous mode + - `assist_stt`: manual transcription flow +5. Update `CHANGELOG.md` and `README.md` for any new option/behavior. --- -## 9. Dependencies +## 8. Minimum supported environment -### Python (integration) - -| Package | Purpose | In HA? | -|---------|---------|--------| -| `aiohttp` | HTTP client for gateway API | ✅ built-in | -| `homeassistant` | HA core APIs | ✅ built-in | - -### JavaScript (frontend card) - -| Library | Purpose | Bundled? | -|---------|---------|----------| -| `lit` | Web component framework | ✅ available via HA | -| `marked` | Markdown rendering | Bundle in card JS | - -No additional pip dependencies required — the integration uses only HA built-ins. - ---- - -## 10. Testing Strategy - -| Level | Tool | Coverage | -|-------|------|----------| -| Unit | `pytest` + `pytest-homeassistant-custom-component` | API client, coordinator, config flow | -| Integration | HA dev container with addon mock | End-to-end entity updates, service calls | -| Frontend | Manual + Playwright | Card rendering, streaming, voice | - ---- - -## 11. Timeline Estimate - -| Phase | Duration | Milestone | -|-------|----------|-----------| -| Phase 1 | 1–2 weeks | Sensors visible, auto-discovery works | -| Phase 2 | 1–2 weeks | Conversation agent in Assist, services/events | -| Phase 3 | 2–3 weeks | Chat card MVP (text + streaming + files) | -| Phase 4 | 2–3 weeks | Voice mode, media player, polish | - -**Total estimated: 6–10 weeks to full feature set.** +- Home Assistant Core: `2025.1.0+` (declared HACS minimum; chosen conservatively for broad compatibility). +- Browser: modern Chromium/Firefox/Safari for card UI; voice capability depends on browser APIs and permissions. diff --git a/hacs.json b/hacs.json index 5fddc60..19cae51 100644 --- a/hacs.json +++ b/hacs.json @@ -3,5 +3,5 @@ "render_readme": true, "domains": ["sensor", "binary_sensor", "conversation"], "iot_class": "local_polling", - "homeassistant": "2026.2.0" + "homeassistant": "2025.1.0" }