Commit Graph

16 Commits

Author SHA1 Message Date
Tarun Sukhani
fee43d505d refactor(memory-neo4j): remove in-process auto sleep cycle, use system cron instead
Sleep cycle is now triggered by a system cron job (`0 3 * * *`) calling
`openclaw memory neo4j sleep` rather than an in-process 6-hour interval
timer with mutex. Simpler, more reliable, and easier to manage.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:39 +08:00
Tarun Sukhani
1a8e46b037 feat(memory-neo4j): add automatic sleep cycle + cleanup sleep/extraction pipeline
Auto-trigger sleep cycle (dedup, extraction, decay, cleanup) in the
background after agent_end when 6h+ have elapsed. Configurable via
sleepCycle.auto and sleepCycle.autoIntervalMs. Removes need for
external cron job with regular gateway usage.

Also includes: removal of Pareto promotion (replaced by manual core
promotion), entity dedup in sleep cycle, and sleep cycle pipeline
cleanup.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:39 +08:00
Tarun Sukhani
e9b9da5a1f memory-neo4j: add userPinned flag, remove demotion, add benchmarking, audit fixes
- Add userPinned boolean on Memory nodes: user-stored core memories are
  immune from importance recalculation, decay, and pruning. Only removable
  via memory_forget. Importance locked at 1.0.
- Add listCoreForInjection(): always injects ALL userPinned core memories
  plus top N non-pinned core memories by importance (no silent drop-off
  for user-pinned memories regardless of maxEntries cap).
- Remove core demotion entirely: promotion is now one-way. Bad core
  memories are handled manually via memory_forget.
- Add [bench] performance timing to auto-recall, auto-capture, core
  memory injection, core refresh, and hybridSearch.
- Audit fixes: remove dead entity/tag methods, dead test blocks, orphaned
  demoteFromCore docstring, unnecessary .slice() in graphSearch.
- Refactor attention gate into shared checks for user/assistant gates.
- Consolidate LLM client, message utils, and config helpers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:39 +08:00
Tarun Sukhani
e562ff4e31 memory-neo4j: tighten attention gate filters and add session skip patterns
Strip voice chat timestamps, conversation metadata blocks, and queued
message wrappers before the attention gate evaluates content. Expand
assistant narration patterns to catch UI interaction verbs, filler
responses ("I'm here", "Sure, tell me"), and page/step progress.
Add configurable autoCaptureSkipPattern and autoRecallSkipPattern
for bypassing memory on latency-sensitive sessions (e.g. voice).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:39 +08:00
Tarun Sukhani
806c5e2d13 memory-neo4j: fix high-severity review findings — security, concurrency, silent failures
- Add safety comment for RELATIONSHIP_TYPE_PATTERN Cypher interpolation
- Add concurrency batching (8) to findDuplicateClusters vector queries
- Bounds-validate memory_recall limit parameter (1-50)
- Fix maxRetries comment (default 2 = 3 attempts, not 1 = 2)
- Fix countByExtractionStatus passing undefined agentId to Cypher
- Fix assistant auto-capture silently disabled when extraction disabled
- Add agentId scoping to findSimilar (dedup + auto-capture)
- Fix BM25 single-result normalization (0.5 instead of inflated 1.0)
- Wrap pruneMemories in retryOnTransient for resilience
- Use UNWIND batch update in reindex instead of N individual queries
- Raise auto-delete threshold from 0.9 to 0.95 to reduce false positives

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
03e4768732 memory-neo4j: code review fixes — search, decay, dedup, retry, tests
Search: fix entity classification order (proper nouns before word count),
BM25 min-max normalization with floor, empty query guard.

Decay: retrieval-reinforced half-life with effective age anchored to
lastRetrievedAt, parameterized category curves (no string interpolation).

Dedup: transfer TAGGED relationships to survivor during merge.

Orphans: use EXISTS pattern instead of stale mentionCount.

Embeddings: Ollama retry with exponential backoff (2 retries, 1s base).

Config: resolve env vars in neo4j.uri, re-export MemoryCategory from schema.

Extractor: abort-aware batch delay, anonymize prompt examples.

Tests: add 80 tests for index.ts (attention gates, message extraction,
wrapper stripping). Full suite: 480 tests across 8 files, all passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
b70cecc307 memory-neo4j: long-term fixes — streaming, abort signals, configurable depth/decay
- Semantic dedup vector pre-screen: skip LLM calls when cosine similarity < 0.8
- Propagate abort signal into sleep cycle phases and extraction pipeline
- Configurable graph search depth (1-3 hops) via graphSearchDepth config
- Streaming extraction: SSE-based callOpenRouterStream with abort responsiveness
- Configurable per-category decay curves for memory consolidation
- Updated tests with SSE streaming mocks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
1ae3afbd6b memory-neo4j: code review quick wins — security, perf, docs fixes
- Fix initPromise retry: reset to null on failure so subsequent calls
  retry instead of returning cached rejected promise
- Remove dead code: findPromotionCandidates, findDemotionCandidates,
  calculateEffectiveImportance (~190 lines, never called)
- Add agentId filter to deleteMemory() to prevent cross-agent deletion
- Fix phase label swaps: 1b=Semantic Dedup, 1c=Conflict Detection
  (CLI banner, phaseNames map, SleepCycleResult/Options type comments)
- Add autoRecallMinScore and coreMemory config to plugin JSON schema
  so the UI can validate and display these options
- Add embedding LRU cache (200 entries, SHA-256 keyed) to eliminate
  redundant API calls across auto-recall, auto-capture, and tools
- Add Ollama concurrency limiter (chunks of 4) to prevent thundering
  herd on single-threaded embedding server

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
fff48a146d memory-neo4j: add auto-recall filtering, assistant capture, importance scoring, conflict detection
Five high-impact improvements to the memory system:

1. Min RRF score threshold on auto-recall (default 0.25) — filters low-relevance
   results before injecting into context
2. Deduplicate auto-recall against core memories already present in context
3. Capture assistant messages (decisions, recommendations, synthesized facts)
   with stricter attention gating and "auto-capture-assistant" source type
4. LLM-judged importance scoring at capture time (0.1-1.0) with 5s timeout
   fallback to 0.5, replacing the flat 0.5 default
5. Conflict detection in sleep cycle (Phase 1b) — finds contradictory memories
   sharing entities, uses LLM to resolve, invalidates the loser

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
ae1d35aab3 fix: remove unnecessary type assertion in neo4j config 2026-02-16 17:56:38 +08:00
Tarun Sukhani
bcbeba400e memory-neo4j: strip injected context blocks, add core category, widen embeddings context 2026-02-16 17:56:38 +08:00
Tarun Sukhani
8e5fe5fc14 memory-neo4j: add context-length-aware embedding truncation 2026-02-16 17:56:38 +08:00
Tarun Sukhani
3082c53a76 memory-neo4j: harden error handling, concurrency safety, config validation + add tests 2026-02-16 17:56:38 +08:00
Tarun Sukhani
c1371b639e memory-neo4j: configurable extraction model + sleep cycle optimizations
- Add extraction config section (apiKey, model, baseUrl) to plugin schema
  with env-var fallback and Ollama/local LLM support (no API key required)
- Add category classification to extraction prompt; update memories from
  'other' to LLM-assigned category
- Reorder sleep phases: extraction before decay
- Parallelize extraction (3 concurrent via Promise.allSettled)
- Pre-compute effective scores once and reuse for promotion/demotion
- Replace O(n²) Cartesian dedup with per-memory HNSW vector index queries
- Use mentionCount for orphan entity detection instead of subquery
- Remove dead auto-capture code (evaluateAutoCapture, CaptureItem, etc.)
2026-02-16 17:56:38 +08:00
Tarun Sukhani
1e4ffdcec8 memory-neo4j: implement mid-session core memory refresh
Add `coreMemory.refreshAtContextPercent` config option to re-inject
core memories when context usage exceeds a threshold. This counters
the "lost in the middle" phenomenon documented by Liu et al. (2023).

Implementation:
- Extend before_agent_start hook event with context usage info
- Pass contextWindowTokens and estimatedUsedTokens to hooks
- Track mid-session refresh per session to prevent over-refreshing
- Clear refresh tracking on compaction
- Add comprehensive tests

Based on research: Liu et al., "Lost in the Middle: How Language
Models Use Long Contexts" (Stanford, 2023)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00
Tarun Sukhani
e65d1deedd Sync adabot changes on top of origin/main
Includes:
- memory-neo4j: four-phase sleep cycle (dedup, decay, extraction, cleanup)
- memory-neo4j: full plugin implementation with hybrid search
- memory-lancedb: updates and benchmarks
- OpenSpec workflow skills and commands
- Session memory hooks
- Various CLI and config improvements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-16 17:56:38 +08:00