# OpenClaw Integration — Implementation Plan

## 1. Overview

This document describes the architecture and phased implementation plan for the
**OpenClaw Integration** — a native Home Assistant custom integration that acts as
a satellite companion to the existing **OpenClaw Assistant** addon.

### Goals

| # | Goal | Priority |
|---|------|----------|
| 1 | Chat card (text, streaming, files, voice) embedded in HA dashboard | P0 |
| 2 | Sensor / binary-sensor entities for status, model, sessions | P0 |
| 3 | Bidirectional addon ↔ integration communication (zero manual setup) | P0 |
| 4 | Native HA conversation agent (Assist / Voice PE) | P1 |
| 5 | Service calls & events for automations | P1 |
| 6 | Media player entity for TTS output | P2 |

---

## 2. Existing Addon — Key Facts

| Property | Value |
|----------|-------|
| Slug | `openclaw_assistant_dev` |
| Gateway port | `18789` (configurable via `gateway_port`) |
| Auth | Token-based (`gateway.auth.token` in `openclaw.json`) |
| Config file | `/config/.openclaw/openclaw.json` (addon container) |
| OpenAI-compatible endpoint | `POST /v1/chat/completions` (opt-in via `enable_openai_api`) |
| Gateway bind | `127.0.0.1` (loopback) or `0.0.0.0` (lan) |
| Node process | `openclaw gateway run` |
| Ingress port | `48099` (nginx) |

---

## 3. Architecture

```
┌──────────────────────────────────────────────────────────┐
│                    Home Assistant Core                    │
│                                                          │
│  ┌─────────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │  Sensors /   │  │ Conversation │  │  Services /    │  │
│  │  Binary      │  │ Agent        │  │  Events        │  │
│  │  Sensors     │  │ (Assist/VPE) │  │                │  │
│  └──────┬───────┘  └──────┬───────┘  └───────┬────────┘  │
│         │                 │                   │           │
│         └────────┬────────┴───────────────────┘           │
│                  │                                        │
│          ┌───────▼────────┐                               │
│          │  OpenClawAPI   │  (api.py — HTTP client)       │
│          │  Client        │                               │
│          └───────┬────────┘                               │
│                  │ HTTP / SSE                              │
├──────────────────┼────────────────────────────────────────┤
│                  │                                        │
│  ┌───────────────▼──────────────────────────────┐         │
│  │         OpenClaw Gateway  (addon)            │         │
│  │                                              │         │
│  │  /v1/chat/completions   (SSE streaming)      │         │
│  │  /api/status            (JSON)               │         │
│  │  /api/sessions          (JSON)               │         │
│  │  /api/models            (JSON)               │         │
│  └──────────────────────────────────────────────┘         │
│                                                           │
│              OpenClaw Assistant Addon Container            │
└───────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────┐
│           Lovelace Dashboard                │
│                                             │
│  ┌────────────────────────────────────────┐ │
│  │       openclaw-chat-card               │ │
│  │  ┌──────────────────────────────────┐  │ │
│  │  │  Message history (markdown)      │  │ │
│  │  │  Typing indicator                │  │ │
│  │  │  File / image attachments        │  │ │
│  │  │  Voice input button              │  │ │
│  │  │  Voice mode toggle               │  │ │
│  │  └──────────────────────────────────┘  │ │
│  └────────────────────────────────────────┘ │
│       ▲                                     │
│       │  HA WebSocket API                   │
│       │  (openclaw.send_message service)    │
│       │  + event subscriptions              │
└───────┼─────────────────────────────────────┘
        │
        ▼
   Home Assistant Core (services / events)
```

### 3.1 Auto-Discovery (Zero-Config)

The integration discovers the addon with **no manual setup** using two mechanisms:

1. **Supervisor API** — at config-flow time the integration calls
   `GET /addons/openclaw_assistant_dev/info` via the HA Supervisor client.  
   This gives us: addon state, network ports, options.

2. **Shared filesystem** — the addon mounts `/addon_configs/<slug>` which maps
   to `/config` inside the container. The integration can read  
   `openclaw.json` to get the gateway auth token, mode, and port.

**Net effect:** the user clicks "Add Integration → OpenClaw" and everything
connects automatically — no token or URL entry required.

### 3.2 Communication Protocol

| Direction | Transport | Endpoint / Mechanism |
|-----------|-----------|----------------------|
| Integration → Chat | HTTP POST + SSE stream | `/v1/chat/completions` (OpenAI compat) |
| Integration → Status | HTTP GET (polled every 30 s) | `/api/status` |
| Integration → Sessions | HTTP GET (polled every 60 s) | `/api/sessions` |
| Integration → Models | HTTP GET (on startup + hourly) | `/v1/models` |
| Addon → HA (events) | HA REST API (long-lived token) | `POST /api/events/openclaw_*` |
| Frontend → Integration | HA WebSocket API | service calls + event subscriptions |

### 3.3 Latency Budget (Voice Mode)

| Step | Target | Notes |
|------|--------|-------|
| STT (browser/HA) | ≤ 500 ms | WebSpeech API or Whisper via HA |
| Network to OpenClaw | ≤ 100 ms | localhost or LAN |
| OpenClaw inference | ≤ 1500 ms | First token via SSE stream |
| TTS | ≤ 500 ms | HA `tts.speak` or browser SpeechSynthesis |
| **Total** | **≤ 2600 ms** | Under 3 s target |

---

## 4. File Structure

```
openclaw_integration/
├── IMPLEMENTATION_PLAN.md          ← this file
├── hacs.json                       ← HACS metadata
├── README.md
│
├── custom_components/
│   └── openclaw/
│       ├── __init__.py             ← integration setup, coordinator
│       ├── manifest.json           ← HA integration manifest
│       ├── config_flow.py          ← auto-discovery config flow
│       ├── const.py                ← constants, defaults
│       ├── api.py                  ← OpenClaw gateway HTTP client
│       ├── coordinator.py          ← DataUpdateCoordinator for polling
│       ├── sensor.py               ← sensor entities
│       ├── binary_sensor.py        ← binary sensor entities
│       ├── conversation.py         ← native conversation agent
│       ├── services.py             ← service call handlers
│       ├── services.yaml           ← service definitions
│       ├── strings.json            ← English UI strings
│       └── translations/
│           └── en.json             ← English translations
│
└── www/
    └── openclaw-chat-card.js       ← Lovelace custom card (Lit)
```

---

## 5. Phased Implementation

### Phase 1 — Foundation (MVP)

**Goal:** Integration installs, auto-discovers addon, exposes sensors.

| Task | File(s) | Description |
|------|---------|-------------|
| 1.1 | `manifest.json` | Integration metadata, dependencies |
| 1.2 | `const.py` | Domain, default ports, config keys |
| 1.3 | `api.py` | `OpenClawApiClient` — HTTP client to gateway |
| 1.4 | `config_flow.py` | Auto-discovery via Supervisor + shared config |
| 1.5 | `coordinator.py` | `DataUpdateCoordinator` polling status/sessions/model |
| 1.6 | `__init__.py` | Integration setup, platforms, coordinator init |
| 1.7 | `sensor.py` | `openclaw_status`, `openclaw_last_activity`, `openclaw_session_count`, `openclaw_model` |
| 1.8 | `binary_sensor.py` | `openclaw_connected` |
| 1.9 | `strings.json`, `translations/en.json` | UI text |
| 1.10 | `hacs.json` | HACS configuration |

**Acceptance criteria:**
- `Add Integration → OpenClaw` works with zero config
- All 5 entities appear and update
- Integration reconnects gracefully when addon restarts

### Phase 2 — Services, Events & Conversation Agent

**Goal:** Automations can send/receive messages; Assist pipeline works.

| Task | File(s) | Description |
|------|---------|-------------|
| 2.1 | `services.yaml`, `services.py` | `openclaw.send_message`, `openclaw.clear_history` |
| 2.2 | `__init__.py` (update) | Fire `openclaw_message_received` HA event on response |
| 2.3 | `conversation.py` | Register as `conversation` agent |
| 2.4 | `api.py` (update) | SSE streaming support (`async for` over response chunks) |
| 2.5 | Testing | Verify Assist Text/Voice pipeline end-to-end |

**Acceptance criteria:**
- `openclaw.send_message` callable from automations
- `openclaw_message_received` event fires and can trigger automations
- OpenClaw appears in Assist agent picker
- Voice PE works: wake word → STT → OpenClaw → TTS → speaker

### Phase 3 — Chat Card (Frontend)

**Goal:** Full chat UI as a Lovelace custom card.

| Task | File(s) | Description |
|------|---------|-------------|
| 3.1 | `openclaw-chat-card.js` | Base card shell (Lit element, card config) |
| 3.2 | (continued) | Message history with timestamps, markdown rendering |
| 3.3 | (continued) | Streaming response display (typing indicator) |
| 3.4 | (continued) | File/image attachment support (upload via service) |
| 3.5 | (continued) | Voice input (WebSpeech/MediaRecorder → send audio) |
| 3.6 | (continued) | Voice mode toggle (continuous listen → auto-respond with TTS) |
| 3.7 | `__init__.py` (update) | Register card as Lovelace resource |
| 3.8 | Card editor | Visual card configuration editor |

**Acceptance criteria:**
- Card renders chat history with markdown
- Real-time streaming of AI responses
- File upload works in both directions
- Voice input captures and sends audio
- Voice mode maintains continuous conversation

### Phase 4 — Media Player & Polish

**Goal:** Native TTS output, wake word integration, production hardening.

| Task | File(s) | Description |
|------|---------|-------------|
| 4.1 | `media_player.py` | Media player entity for TTS output routing |
| 4.2 | | Wake word detection integration |
| 4.3 | | WebSocket connection management (token refresh, reconnect) |
| 4.4 | | Error handling, rate limiting, connection pooling |
| 4.5 | | Performance profiling (voice latency budget) |
| 4.6 | | Documentation and HACS submission |

---

## 6. Entity Reference

### Sensors

| Entity ID | Class | State | Attributes |
|-----------|-------|-------|------------|
| `sensor.openclaw_status` | — | `online` / `offline` / `processing` | `gateway_version`, `uptime` |
| `sensor.openclaw_last_activity` | `timestamp` | ISO 8601 datetime | `last_message_preview` |
| `sensor.openclaw_session_count` | — | integer (active count) | `sessions` (list of IDs) |
| `sensor.openclaw_model` | — | model name string | `provider`, `context_window` |

### Binary Sensors

| Entity ID | Class | State |
|-----------|-------|-------|
| `binary_sensor.openclaw_connected` | `connectivity` | `on` / `off` |

### Services

| Service | Fields | Description |
|---------|--------|-------------|
| `openclaw.send_message` | `message` (str), `session_id` (str, optional), `attachments` (list, optional) | Send a message to OpenClaw |
| `openclaw.clear_history` | `session_id` (str, optional) | Clear conversation history |

### Events

| Event | Data | Description |
|-------|------|-------------|
| `openclaw_message_received` | `message`, `session_id`, `model`, `timestamp` | Fired when OpenClaw sends a response |

---

## 7. Key Technical Decisions

### 7.1 Why HTTP+SSE instead of WebSocket?

- The addon already exposes an **OpenAI-compatible** HTTP endpoint with SSE streaming
- No additional gateway code needed on the addon side
- SSE is simpler to manage (no bi-directional state, automatic reconnect)
- WebSocket can be added later for real-time push features

### 7.2 Why Supervisor API for discovery?

- Available in all HAOS / Supervised installs (target audience)
- Provides addon state, network config, and options without filesystem access
- Filesystem fallback (`/addon_configs/`) handles edge cases

### 7.3 Why register as a conversation agent?

- Native Assist integration = works with Voice PE, S3 satellite, etc.
- No need for third-party HACS integrations (Extended OpenAI Conversation)
- Event-driven: HA handles STT/TTS pipeline, we just process text

### 7.4 Frontend card communication

- Card uses HA WebSocket API (already authenticated)
- Calls `openclaw.send_message` service
- Subscribes to `openclaw_message_received` events
- No separate WebSocket to gateway needed (avoids CORS, auth issues)

---

## 8. Challenges & Mitigations

| Challenge | Mitigation |
|-----------|------------|
| Gateway not reachable (loopback bind) | Integration runs in same host; `127.0.0.1` works. Document `lan` mode for remote setups. |
| Token rotation / mismatch | Re-read `openclaw.json` on connection failure; config flow stores token in HA config entry |
| Voice latency > 3s | Use SSE streaming (first token fast), browser-side SpeechSynthesis for TTS, Whisper locally for STT |
| Addon not installed | Config flow gracefully fails with a message directing user to install the addon first |
| HACS distribution | Standard HACS custom integration + Lovelace card resources |
| Large file uploads | Chunk uploads through service call; gateway handles multipart |

---

## 9. Dependencies

### Python (integration)

| Package | Purpose | In HA? |
|---------|---------|--------|
| `aiohttp` | HTTP client for gateway API | ✅ built-in |
| `homeassistant` | HA core APIs | ✅ built-in |

### JavaScript (frontend card)

| Library | Purpose | Bundled? |
|---------|---------|----------|
| `lit` | Web component framework | ✅ available via HA |
| `marked` | Markdown rendering | Bundle in card JS |

No additional pip dependencies required — the integration uses only HA built-ins.

---

## 10. Testing Strategy

| Level | Tool | Coverage |
|-------|------|----------|
| Unit | `pytest` + `pytest-homeassistant-custom-component` | API client, coordinator, config flow |
| Integration | HA dev container with addon mock | End-to-end entity updates, service calls |
| Frontend | Manual + Playwright | Card rendering, streaming, voice |

---

## 11. Timeline Estimate

| Phase | Duration | Milestone |
|-------|----------|-----------|
| Phase 1 | 1–2 weeks | Sensors visible, auto-discovery works |
| Phase 2 | 1–2 weeks | Conversation agent in Assist, services/events |
| Phase 3 | 2–3 weeks | Chat card MVP (text + streaming + files) |
| Phase 4 | 2–3 weeks | Voice mode, media player, polish |

**Total estimated: 6–10 weeks to full feature set.**