github/openclaw
1
0
Fork 0
mirror of https://github.com/openclaw/openclaw.git synced 2026-04-19 02:47:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

refactor(webchat): SwiftUI-only WebChat UI

Browse Source 
# Conflicts:
#	apps/macos/Package.swift
This commit is contained in:
Peter Steinberger
2025-12-17 23:05:28 +01:00
parent ca85d217ec
commit 875cf9a054
7451 changed files with 218063 additions and 776607 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/mac/browser.md
View File

@@ -37,7 +37,6 @@ Optional (advanced, can be hidden behind Debug initially):
Clawdis already uses:
- Gateway WebSocket: `18789`
- WebChat HTTP: `18788`
- Bridge (voice/iris): `18790`
For the clawd browser-control server, use “family” ports:

2
docs/mac/canvas.md
View File

@@ -85,7 +85,7 @@ Expose Canvas via the existing `clawdis-mac` → control socket → app routing
- Capture a snapshot image of the current canvas view.
- Optionally set panel placement (screen `x/y` + `width/height`) when showing/navigating.
This should be modeled after `WebChatManager`/`WebChatWindowController` but targeting `clawdis-canvas://…` URLs.
This should be modeled after `WebChatManager`/`WebChatSwiftUIWindowController` but targeting `clawdis-canvas://…` URLs.
Related:
- For “invoke the agent again from UI” flows, prefer the macOS deep link scheme (`clawdis://agent?...`) so *any* UI surface (Canvas, WebChat, native views) can trigger a new agent run. See `docs/clawdis-mac.md`.

7
docs/mac/remote.md
View File

@@ -29,9 +29,8 @@ This flow lets the macOS app act as a full remote control for a Clawdis gateway
4) Health checks and Web Chat will now run through this SSH tunnel automatically.
## Web Chat over SSH
- The mac app serves the WebChat assets locally (from the app bundle) and connects to the gateway over the forwarded WebSocket control port (default 18789).
- The gateway’s own loopback WebChat HTTP server (default 18788, see `webchat.port`) is not required in remote mode.
- Keep the feature enabled in *Settings → Config → Web chat*. Disable it to hide the menu entry entirely.
- Web Chat connects to the gateway over the forwarded WebSocket control port (default 18789).
- There is no separate WebChat HTTP server anymore.
## Permissions
- The remote host needs the same TCC approvals as local (Automation, Accessibility, Screen Recording, Microphone, Speech Recognition, Notifications). Run onboarding on that machine to grant them once.
@@ -44,7 +43,7 @@ This flow lets the macOS app act as a full remote control for a Clawdis gateway
## Troubleshooting
- **exit 127 / not found**: `clawdis` isn’t on PATH for non-login shells. Add it to `/etc/paths`, your shell rc, or symlink into `/usr/local/bin`/`/opt/homebrew/bin`.
- **Health probe failed**: check SSH reachability, PATH, and that Baileys is logged in (`clawdis status --json`).
- **Web Chat stuck**: confirm the gateway is running on the remote host and `webchat.enabled` is true; ensure the forwarded port matches *Settings → Config*. Since RPC is in-process, PATH is no longer a factor.
- **Web Chat stuck**: confirm the gateway is running on the remote host and the forwarded port matches the gateway WS port; the UI requires a healthy WS connection.
- **Voice Wake**: trigger phrases are forwarded automatically in remote mode; no separate forwarder is needed.
## Notification sounds

27
docs/mac/webchat.md
View File

@@ -5,32 +5,23 @@ read_when:
---
# Web Chat (macOS app)
The macOS menu bar app embeds the WebChat UI in a WKWebView and reuses the **primary Clawd session** (`main` by default, configurable via `inbound.session.mainKey`).
The macOS menu bar app shows the WebChat UI as a native SwiftUI view and reuses the **primary Clawd session** (`main` by default, configurable via `inbound.session.mainKey`).
- **Local mode**: loads the gateway’s loopback WebChat HTTP server (default port 18788, see `webchat.port`).
- **Remote mode**: serves the WebChat assets locally from the mac app bundle (via `WebChatServer`) and only forwards the gateway WebSocket control port over SSH.
- **Local mode**: connects directly to the local Gateway WebSocket.
- **Remote mode**: forwards the Gateway WebSocket control port over SSH and uses that as the data plane.
## Launch & debugging
- Manual: Lobster menu → “Open Chat”.
- Auto-open for testing: run `dist/Clawdis.app/Contents/MacOS/Clawdis --webchat` (or pass `--webchat` to the binary launched by launchd). The window opens on startup.
- Inspect: right-click the web view → “Inspect Element” (developerExtras enabled). Console logs go to the Swift logger (subsystem `com.steipete.clawdis`, category `WebChat`). The HTML boot script also writes status text into the `#app` div until the panel mounts.
- WK logs: navigation lifecycle, readyState, js location, and JS errors/unhandled rejections are mirrored to OSLog for easier diagnosis.
- Logs: see `./scripts/clawlog.sh` (subsystem `com.steipete.clawdis`, category `WebChatSwiftUI`).
## How it’s wired
- Assets: `apps/macos/Sources/Clawdis/Resources/WebChat/` contains the `pi-web-ui` dist plus a local import map pointing at bundled vendor modules and a tiny `pi-ai` stub. Everything is served from the static host at `/` (legacy `/webchat/*` still works).
- Bridge: none. The web UI connects directly to the Gateway WebSocket (default 18789) and uses `chat.history`/`chat.send` plus `chat/presence/tick/health` events. No `/rpc` or file-watcher socket path remains.
- Session: always primary; multiple transports (WhatsApp/Telegram/Desktop) share the same session key so context is unified.
- Debug-only: a native SwiftUI “glass” chat UI (same WS transport, attachments + thinking selector) can replace the WKWebView. Enable it via Debug → “Use SwiftUI web chat (glass, gateway WS)” (default off).
- Implementation: `apps/macos/Sources/Clawdis/WebChatSwiftUI.swift` hosts `ClawdisChatUI` and speaks to the Gateway over `GatewayConnection`.
- Data plane: Gateway WebSocket methods `chat.history`, `chat.send`, `chat.abort`; events `chat`, `agent`, `presence`, `tick`, `health`.
- Session: currently always primary (`main`). Session switching UI is intentionally hidden for now.
## Security / surface area
- Loopback server only; remote mode forwards only the gateway WebSocket control port over SSH. CSP is set to `default-src 'self' 'unsafe-inline' data: blob:`.
- Web Inspector is opt-in via right-click; otherwise WKWebView stays in the app sandbox.
- Remote mode forwards only the Gateway WebSocket control port over SSH.
## Known limitations
- Text-only, single-turn (no streaming); tools/attachments not yet plumbed.
- Uses a stubbed pi-ai for model metadata; model selection is fixed to the primary Clawd backend.
## Updating the bundle
1) Ensure `../pi-mono` is present and built (`pnpm install` + `pnpm build` inside `packages/web-ui`).
2) Copy vendor deps into `Resources/WebChat/vendor` (currently synced from `../pi-mono/node_modules`).
3) Rebuild/restart the mac app with `./scripts/restart-mac.sh` so the new assets land in the bundle.
- The UI is optimized for the primary session and typical “chat” usage (not a full browser-based sandbox surface).