Commit Graph

11467 Commits

Author SHA1 Message Date
Tarun Sukhani
b70cecc307 memory-neo4j: long-term fixes — streaming, abort signals, configurable depth/decay
- Semantic dedup vector pre-screen: skip LLM calls when cosine similarity < 0.8
- Propagate abort signal into sleep cycle phases and extraction pipeline
- Configurable graph search depth (1-3 hops) via graphSearchDepth config
- Streaming extraction: SSE-based callOpenRouterStream with abort responsiveness
- Configurable per-category decay curves for memory consolidation
- Updated tests with SSE streaming mocks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
1f80d4f0d2 memory-neo4j: medium-term fixes — index, batching, parallelism, module extraction
- Add composite index on (agentId, category) for faster filtered queries
- Combine graph search into single UNION Cypher query (was 2 sequential)
- Parallelize conflict resolution with LLM_CONCURRENCY chunks
- Batch entity operations (merge, mentions, relationships, tags, category,
  extraction status) into a single managed transaction
- Make auto-capture fire-and-forget with shared captureMessage helper
- Extract attention-gate.ts and message-utils.ts modules from index.ts
  and extractor.ts for better separation of concerns
- Update tests to match new batched/combined APIs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
1ae3afbd6b memory-neo4j: code review quick wins — security, perf, docs fixes
- Fix initPromise retry: reset to null on failure so subsequent calls
  retry instead of returning cached rejected promise
- Remove dead code: findPromotionCandidates, findDemotionCandidates,
  calculateEffectiveImportance (~190 lines, never called)
- Add agentId filter to deleteMemory() to prevent cross-agent deletion
- Fix phase label swaps: 1b=Semantic Dedup, 1c=Conflict Detection
  (CLI banner, phaseNames map, SleepCycleResult/Options type comments)
- Add autoRecallMinScore and coreMemory config to plugin JSON schema
  so the UI can validate and display these options
- Add embedding LRU cache (200 entries, SHA-256 keyed) to eliminate
  redundant API calls across auto-recall, auto-capture, and tools
- Add Ollama concurrency limiter (chunks of 4) to prevent thundering
  herd on single-threaded embedding server

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
d311438cb4 memory-neo4j: fix Ollama embedding context overflow for token-dense inputs 2026-02-16 17:56:38 +08:00
Tarun Sukhani
27cb766209 memory-neo4j: strengthen auto-capture filtering and add Slack metadata stripping
- Raise MIN_CAPTURE_CHARS from 10 to 30 to reject trivially short messages
- Add noise patterns for conversational filler (haha, lol, hmm, etc.)
- Add noise pattern to reject /new and /reset session prompts
- Raise importance threshold for assistant auto-captures to >= 0.7
- Add Slack protocol prefix/suffix stripping in stripMessageWrappers()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
4a3d424890 updated dependency 2026-02-16 17:56:38 +08:00
Tarun Sukhani
fff48a146d memory-neo4j: add auto-recall filtering, assistant capture, importance scoring, conflict detection
Five high-impact improvements to the memory system:

1. Min RRF score threshold on auto-recall (default 0.25) — filters low-relevance
   results before injecting into context
2. Deduplicate auto-recall against core memories already present in context
3. Capture assistant messages (decisions, recommendations, synthesized facts)
   with stricter attention gating and "auto-capture-assistant" source type
4. LLM-judged importance scoring at capture time (0.1-1.0) with 5s timeout
   fallback to 0.5, replacing the flat 0.5 default
5. Conflict detection in sleep cycle (Phase 1b) — finds contradictory memories
   sharing entities, uses LLM to resolve, invalidates the loser

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
9f6372241c hooks: fire session_end on /new and /reset so plugins clear bootstrap state 2026-02-16 17:56:38 +08:00
Tarun Sukhani
9cfb56696f memory-neo4j: extract stripMessageWrappers helper, use in cleanup for accurate gating 2026-02-16 17:56:38 +08:00
Tarun Sukhani
6747967b83 memory-neo4j: strip channel metadata wrappers, reject system infra messages in attention gate 2026-02-16 17:56:38 +08:00
Tarun Sukhani
7674fa8c15 memory-neo4j: cleanup targets auto-capture only, trust explicit memory_store 2026-02-16 17:56:38 +08:00
Tarun Sukhani
91efe2e432 memory-neo4j: tighten attention gate, add gate to memory_store, add cleanup command 2026-02-16 17:56:38 +08:00
Tarun Sukhani
ae1d35aab3 fix: remove unnecessary type assertion in neo4j config 2026-02-16 17:56:38 +08:00
Tarun Sukhani
bcbeba400e memory-neo4j: strip injected context blocks, add core category, widen embeddings context 2026-02-16 17:56:38 +08:00
Tarun Sukhani
c002574371 logging: standardize subsystem compact format, add timestamp tests 2026-02-16 17:56:38 +08:00
Tarun Sukhani
f1753aa336 logging: use local time (with tz offset) everywhere instead of UTC 2026-02-16 17:56:38 +08:00
Tarun Sukhani
516459395c updated time 2026-02-16 17:56:38 +08:00
Tarun Sukhani
b0a9eb9407 memory-neo4j: drop entity vector embeddings (use fulltext search) 2026-02-16 17:56:38 +08:00
Tarun Sukhani
f1f32d5723 feat(memory-neo4j): log auto-capture at info level even when 0 stored 2026-02-16 17:56:38 +08:00
Tarun Sukhani
5761b23760 chore: fix update-and-restart to rebase on origin/main and force-push 2026-02-16 17:56:38 +08:00
Tarun Sukhani
370adb0f4b memory-neo4j: add 'openclaw memory neo4j index' reindex command
Adds a CLI command to re-embed all Memory and Entity nodes after
changing the embedding model or provider. Drops old vector indexes,
re-embeds in batches via the configured provider, and recreates
indexes with the correct dimensions.
2026-02-16 17:56:38 +08:00
Tarun Sukhani
bf5a7a05dd sandbox: scope skill loading to workspace for sandboxed agents
Prevents managed/bundled skill file paths from leaking into sandboxed
agent skill snapshots, which caused 'path escapes sandbox root' errors.
Adds scopeToWorkspace option to loadSkillEntries/buildWorkspaceSkillSnapshot.
Also fixes stale Docker mount detection on container probe failure.
2026-02-16 17:56:38 +08:00
Tarun Sukhani
50f095ecb0 chore: fix lint curly brace in embeddings.ts 2026-02-16 17:56:38 +08:00
Tarun Sukhani
8e5fe5fc14 memory-neo4j: add context-length-aware embedding truncation 2026-02-16 17:56:38 +08:00
Tarun Sukhani
e65b052d27 logging: fix sub-logger inheriting undefined minLevel from parent 2026-02-16 17:56:38 +08:00
Tarun Sukhani
f5859e09ab logging: fix inverted levelToMinLevel mapping vs tslog v4 level IDs 2026-02-16 17:56:38 +08:00
Tarun Sukhani
d096055a4b chore: fix lint errors in memory-neo4j 2026-02-16 17:56:38 +08:00
Tarun Sukhani
0908731c54 logging: isolate test logs to /tmp/openclaw-test under vitest 2026-02-16 17:56:38 +08:00
Tarun Sukhani
3082c53a76 memory-neo4j: harden error handling, concurrency safety, config validation + add tests 2026-02-16 17:56:38 +08:00
Tarun Sukhani
c1371b639e memory-neo4j: configurable extraction model + sleep cycle optimizations
- Add extraction config section (apiKey, model, baseUrl) to plugin schema
  with env-var fallback and Ollama/local LLM support (no API key required)
- Add category classification to extraction prompt; update memories from
  'other' to LLM-assigned category
- Reorder sleep phases: extraction before decay
- Parallelize extraction (3 concurrent via Promise.allSettled)
- Pre-compute effective scores once and reuse for promotion/demotion
- Replace O(n²) Cartesian dedup with per-memory HNSW vector index queries
- Use mentionCount for orphan entity detection instead of subquery
- Remove dead auto-capture code (evaluateAutoCapture, CaptureItem, etc.)
2026-02-16 17:56:38 +08:00
Tarun Sukhani
66f9f972b2 chore: remove unused imports in mid-session-refresh test
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
1e4ffdcec8 memory-neo4j: implement mid-session core memory refresh
Add `coreMemory.refreshAtContextPercent` config option to re-inject
core memories when context usage exceeds a threshold. This counters
the "lost in the middle" phenomenon documented by Liu et al. (2023).

Implementation:
- Extend before_agent_start hook event with context usage info
- Pass contextWindowTokens and estimatedUsedTokens to hooks
- Track mid-session refresh per session to prevent over-refreshing
- Clear refresh tracking on compaction
- Add comprehensive tests

Based on research: Liu et al., "Lost in the Middle: How Language
Models Use Long Contexts" (Stanford, 2023)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
007daf3c27 cli: show memory plugins in openclaw memory status
Detect configured memory plugins (memory-neo4j, memory-lancedb) and show
their status alongside core memory search. Provides helpful hints about
plugin-specific commands when plugins are enabled.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
e7ac300b7e memory-neo4j: add Pareto-based memory ecosystem with retrieval tracking
Implement retrieval tracking and Pareto-based memory consolidation:

- Track retrievalCount and lastRetrievedAt on every search
- Effective importance formula: importance × freq_boost × recency_factor
- Seven-phase sleep cycle: dedup, pareto scoring, promotion, demotion,
  decay/pruning, extraction, cleanup
- Bidirectional mobility between core (≤20%) and regular memory tiers
- Core memories ranked by pure usage (no importance multiplier)

Based on ACT-R memory model and Ebbinghaus forgetting curve research.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
e65d1deedd Sync adabot changes on top of origin/main
Includes:
- memory-neo4j: four-phase sleep cycle (dedup, decay, extraction, cleanup)
- memory-neo4j: full plugin implementation with hybrid search
- memory-lancedb: updates and benchmarks
- OpenSpec workflow skills and commands
- Session memory hooks
- Various CLI and config improvements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Peter Steinberger
7cfd0aed5f test: remove duplicate non-date negative-case assertion 2026-02-16 09:56:46 +00:00
Peter Steinberger
d611db8049 test: remove duplicate provider-prefix assertion variant 2026-02-16 09:55:44 +00:00
Peter Steinberger
3eb9c2105c test: remove duplicate date-suffix assertion variant 2026-02-16 09:54:56 +00:00
Peter Steinberger
9f6462bd56 test: trim duplicate latest-suffix assertion variant 2026-02-16 09:54:05 +00:00
Peter Steinberger
2d03473072 test: trim duplicate provider-prefix assertion in short-model tests 2026-02-16 09:52:16 +00:00
Peter Steinberger
dbcdcc5d19 test: remove duplicate positive template-variable detection case 2026-02-16 09:51:09 +00:00
Peter Steinberger
c4297a8d60 test: remove redundant no-provider short-model case 2026-02-16 09:49:58 +00:00
Peter Steinberger
deef9f91bf test: remove duplicate multi-variable template check case 2026-02-16 09:48:51 +00:00
Peter Steinberger
523193a91f test: remove duplicate static template-variable false case 2026-02-16 09:47:45 +00:00
Peter Steinberger
cd04385f9f test: remove redundant provider-plus-date model-name case 2026-02-16 09:46:44 +00:00
Peter Steinberger
82fa526bb0 test: remove duplicate undefined template-variable guard case 2026-02-16 09:45:51 +00:00
Peter Steinberger
3fb4a7eb53 test: remove duplicate hook-wake heartbeat empty-file case 2026-02-16 09:44:16 +00:00
Peter Steinberger
7a6928712b test: remove redundant explicit telegram heartbeat target case 2026-02-16 09:43:01 +00:00
Peter Steinberger
9b351fcbd8 test: remove duplicate whatsapp group heartbeat target case 2026-02-16 09:41:50 +00:00
Peter Steinberger
d3ddf893c2 test: remove redundant store-rotation integration prune case 2026-02-16 09:39:48 +00:00