Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
64857f9d5e | ||
|
|
35c802c1b8 | ||
|
|
3b08f45618 |
No files matched your search
@@ -1,197 +1,176 @@
|
||||
# mcp-memory
|
||||
|
||||
A high-performance, persistent Knowledge Graph and Context daemon for Antigravity, implementing the Model Context Protocol (MCP).
|
||||
A high-performance, persistent Knowledge Graph, Code Intelligence, and Context daemon for Antigravity, implementing the Model Context Protocol (MCP).
|
||||
|
||||
## Overview
|
||||
|
||||
`mcp-memory` acts as the persistent "brain" for `agy` CLI agents. It tracks entities, relations, background tasks, engineering debt, architectural decisions, and error fixes across sessions.
|
||||
`mcp-memory` acts as the persistent "brain" for `agy` CLI agents and autonomous subagents. It tracks graph entities, relations, background tasks, milestones, engineering debt, architectural decisions, code modifications, terminal activity, and compiler error fixes across sessions.
|
||||
|
||||
To eliminate heavy Cross-OS I/O penalties when using WSL and Windows simultaneously, `mcp-memory` operates using a **Dual-Transport Leader/Stub Architecture**:
|
||||
* **The Server (`mcp-memory-server`)**: Runs natively on the Windows host. It binds to `0.0.0.0:3000`, serving standard stdio to the primary Windows `agy` instance while simultaneously hosting an Axum HTTP and WebSocket server for secondary clients.
|
||||
* **The Stub (`mcp-memory-stub`)**: An ultra-lightweight proxy binary. WSL `agy` instances run this native Linux stub, which transparently pipes stdio JSON-RPC traffic over the network to the Windows HTTP server (`http://127.0.0.1:3000`), completely bypassing WSL NTFS mounts. It features full MPSC queue buffering and a WebSocket reconnect handshake (`notifications/tools/list_changed`) so that tools automatically refresh seamlessly without disconnecting the CLI if the background server restarts.
|
||||
To eliminate cross-OS I/O penalties when developing across WSL and Windows simultaneously, `mcp-memory` operates using a **Dual-Transport Leader/Stub Architecture**:
|
||||
* **The Server (`mcp-memory-server`)**: Runs natively on the Windows host. It binds to `0.0.0.0:3000`, serving standard stdio to the primary Windows `agy` instance while simultaneously hosting Axum HTTP, WebSocket, and Zero-Latency UDP endpoints for secondary clients and UI dashboards.
|
||||
* **The Stub (`mcp-memory-stub`)**: An ultra-lightweight proxy binary. WSL `agy` instances run this native Linux stub, which transparently pipes stdio JSON-RPC traffic over the local network to the Windows HTTP server (`http://127.0.0.1:3000`), completely bypassing WSL NTFS cross-mounts. It features full MPSC queue buffering and a WebSocket reconnect handshake (`notifications/tools/list_changed`) so that tools automatically refresh seamlessly without disconnecting the CLI if the background server restarts.
|
||||
|
||||
> **Note for Users & LLMs**: Please read the [Strategic Guidelines](./instructions.md) and [Effective Discourse Guide](./EFFECTIVE_DISCOURSE.md) to learn how to perfectly trigger this server's advanced MCP tools.
|
||||
> [!NOTE]
|
||||
> For in-depth strategy, casing standards, and tool semantics, please refer to the [Strategic Guidelines (`instructions.md`)](./instructions.md) and [Effective Discourse Guide](./EFFECTIVE_DISCOURSE.md).
|
||||
|
||||
---
|
||||
|
||||
## Casing & Naming Standards
|
||||
## 🏛️ Architecture Status: 100% ADR Implementation
|
||||
|
||||
To prevent graph fragmentation and ensure optimal LLM tokenization and retrieval:
|
||||
* **Entity Types (`entity_type`)**: Standardized as **`PascalCase`** (e.g. `DatabaseTable`, `McpTool`, `ArchitectureComponent`, `File`).
|
||||
* **Relation Types (`relation_type`)**: Standardized as **`snake_case`** (e.g. `depends_on`, `calls`, `implements`, `uses`).
|
||||
* **Field Keys & Attributes**: Standardized as **`snake_case`** (e.g. `file_path`, `git_commit`, `created_at`).
|
||||
|
||||
*Note: The server automatically normalizes and migrates incoming types to these canonical conventions on every read and write operation.*
|
||||
All 25 Architectural Decision Records (**ADR-0080 through ADR-0104**) are **100% implemented, verified, and reconciled** in the persistent store:
|
||||
* **ADR-0080 – ADR-0093**: Enterprise persistence, AST intelligence, vector embeddings, cross-OS dual transports, and headless Neovim RPC.
|
||||
* **ADR-0094 – ADR-0101**: Zero-subprocess security invariants, pure native Rust clipboard (`arboard`), and bounded telemetry buffers.
|
||||
* **ADR-0102**: Dynamic Fastembed micro-batching with 16k character budget ceiling.
|
||||
* **ADR-0103**: Real-time Tantivy search reader auto-reloading upon background index commits.
|
||||
* **ADR-0104**: Automated Git post-commit ADR & Task status reconciliation engine (`scripts/git-reconcile.py`).
|
||||
|
||||
---
|
||||
|
||||
## 📡 Passive MCP Context Resources (`resources/list`)
|
||||
|
||||
Agents can passively read these 9 MCP resources for instant zero-turn context without incurring tool call latency:
|
||||
|
||||
| Resource URI | Resource Name | Description & Usage |
|
||||
|:---|:---|:---|
|
||||
| `memory://graph/entities` | Graph Entities | All nodes and entities currently stored in the knowledge graph. |
|
||||
| `memory://graph/relations` | Graph Relations | All relationship edges between entities in the knowledge graph. |
|
||||
| `memory://tasks/active` | Active Tasks | List of all currently pending or uncompleted tasks. |
|
||||
| `memory://decisions/active` | Active ADR Decisions | All accepted Architectural Decision Records (ADRs). |
|
||||
| `memory://tech_debt/unresolved` | Unresolved Tech Debt | All currently open engineering debt items. |
|
||||
| `memory://session/delta` | Session Delta | Code modifications, commits, active tasks, and notes created in the last 2 hours. |
|
||||
| `memory://terminal/recent` | Terminal History | Recent terminal commands, shell interpreters (`pwsh`, `bash`, `nu`), working dirs, and exit codes. |
|
||||
| `memory://activity/recent` | Recent Activity | Real-time IDE, editor, and developer activity logs. |
|
||||
| `memory://milestones` | Milestones | Project milestones, deliverables, target dates, and status. |
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Consolidated Smart MCP Tools
|
||||
## ⚡ MCP Workflow Prompts (`prompts/list`)
|
||||
|
||||
The server consolidates granular single-purpose tools into 11 concise, action-oriented smart domain handlers with zero prefix clutter:
|
||||
The server registers 5 high-signal workflow prompts:
|
||||
* **`context_warmup`**: Warm up session context by reading active tasks, recent deltas, and the git worktree.
|
||||
* **`analyze_tech_debt`**: Inspect open technical debt items and generate a prioritized remediation plan.
|
||||
* **`summarize_architecture`**: Synthesize active ADRs and knowledge graph entities into an architectural overview.
|
||||
* **`handoff_routine`**: Invoke the `DevOpsSRE` subagent at session end to generate a standup report and leave a handoff memo.
|
||||
* **`archive_routine`**: Compress older session summaries into dense milestone retrospectives.
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Complete MCP Tool Suite (53 Tools)
|
||||
|
||||
The server exposes 53 tools categorized into 7 functional domains:
|
||||
|
||||
### 1. Consolidated Smart Primary Tools (11 Domain Handlers)
|
||||
* **`tasks`**: Complete task lifecycle management (`add`, `update`, `delete`, `list`, `set_criteria`, `verify`).
|
||||
* **`milestones`**: Milestone tracking (`add`, `update`, `list`).
|
||||
* **`handoff_memos`**: Cross-session handoff notes (`leave`, `read`, `clear`).
|
||||
* **`snippets`**: Reusable code snippet vault with BM25+Vector search (`store`, `search`, `delete`, `tag`).
|
||||
* **`decisions`**: Architectural Decision Records (ADRs) (`log`, `query`, `delete`).
|
||||
* **`milestones`**: Project milestone tracking (`add`, `update`, `list`).
|
||||
* **`handoff_memos`**: Cross-session scratchpad and handoff memos (`leave`, `read`, `clear`).
|
||||
* **`snippets`**: Reusable code snippet vault with hybrid BM25 + dense vector search (`store`, `search`, `delete`, `tag`).
|
||||
* **`decisions`**: Architectural Decision Records (ADRs) (`log`, `update`, `query`, `delete`).
|
||||
* **`tech_debt`**: Engineering technical debt backlog (`log`, `resolve`, `list`).
|
||||
* **`environment`**: Infrastructure & tool fingerprints tracking (`update_fingerprint`, `read_fingerprint`, `log_requirement`, `register`, `get_details`).
|
||||
* **`clipboard`**: Cross-OS clipboard management (`read`, `write`).
|
||||
* **`hypotheses`**: Diagnostic hypothesis memory (`log`, `query`).
|
||||
* **`agent_signals`**: Inter-agent signal bus (`broadcast`, `query`).
|
||||
* **`clipboard`**: Pure native Rust OS clipboard interface (`read`, `write`).
|
||||
* **`hypotheses`**: Diagnostic hypothesis memory for root cause analysis (`log`, `query`).
|
||||
* **`agent_signals`**: Real-time inter-agent signal bus (`broadcast`, `query`).
|
||||
* **`process_logs`**: Process and daemon log management (`watch`, `get`, `clear`).
|
||||
---
|
||||
|
||||
## Key Features & Capabilities
|
||||
### 2. Knowledge Graph Core (18 Tools)
|
||||
* `create_entities`, `create_relations`, `add_observations`, `delete_entities`, `delete_relations`, `delete_observations`
|
||||
* `read_graph`, `search_nodes`, `open_nodes`, `visualize_graph`, `condense_entity`, `merge_entities`, `find_orphans`
|
||||
* `get_subgraph` (BFS $N$-hop neighborhood expansion)
|
||||
* `sweep_graph_health` (orphan detection, name similarity, automated merge recommendations)
|
||||
* `resolve_stale_symbols` (workspace AST cross-referencing to eliminate stale graph nodes)
|
||||
* `summarize_subgraph` (concise subgraph synthesis)
|
||||
* `query_graph_path` (BFS shortest path finding)
|
||||
|
||||
### 📜 VCS-Agnostic Code Change Ledger & Recent Deltas (`/api/ledger` & `memory://session/delta`)
|
||||
Maintains an audit ledger of all file modifications, commit hashes / SVN revisions (`vcs_revision`), repository branches, upstream URLs, and AI change summaries with deterministic length bounds. Fully agnostic across Git, Subversion (SVN), and Mercurial (Hg). Exposed via the Brain Monitor Web UI (`/api/ledger`) and accessible as a passive context resource (`memory://session/delta`).
|
||||
### 3. AST & Code Intelligence (8 Tools)
|
||||
* `read_file_skeleton`: Tree-sitter AST structural outline without implementation bodies.
|
||||
* `replace_ast_node`: Precise structural code replacement preserving comments and formatting.
|
||||
* `find_symbol_references`: Cross-file symbol reference lookup across snippets and disk source code.
|
||||
* `get_callers`: Call site and caller identification across the codebase.
|
||||
* `analyze_impact`: Blast-radius impact analysis of modifying a symbol or file.
|
||||
* `read_directory_architecture`: Recursive directory structure analysis capped at depth 10.
|
||||
* `semantic_code_search`: Dense vector semantic code search over indexed source code.
|
||||
* `manage_subagent_namespace`: Isolated memory namespaces for concurrent subagent workflows.
|
||||
|
||||
### 💻 Terminal & Process Telemetry (`/terminal/history`)
|
||||
Tracks active shell instances (PowerShell, Bash, Nushell, Zsh), command history, working directories, and exit codes in real time. Enables LLMs and the Brain Monitor UI to maintain total visibility over terminal execution contexts.
|
||||
### 4. Meta, Audit & Intelligence (15 Tools)
|
||||
* `decisions`, `tech_debt`, `log_error_fix`, `search_error_fixes`, `log_code_change`, `query_recent_changes`
|
||||
* `omni_search` (Reciprocal Rank Fusion hybrid BM25 + Vector search)
|
||||
* `get_project_health` (high-level system health dashboard)
|
||||
* `manage_checkpoint` (snapshot freeze and rollback)
|
||||
* `query_lineage` (causal lineage linking tasks, ADRs, commits, and error fixes)
|
||||
* `get_next_actionable_tasks` (topological unblocked task resolver)
|
||||
* `hypotheses`, `get_preflight_context`, `agent_signals`, `auto_session_checkpoint`
|
||||
|
||||
### 📋 Enriched Task Board, ADRs & Technical Debt Backlog
|
||||
Supports structured priorities (`low`, `medium`, `high`, `urgent`), assigned subagent roles, automated verification commands, architectural decision alternatives and consequences, and granular technical debt tracking (line ranges, workarounds, effort estimates).
|
||||
### 5. Task & Milestone Management (2 Tools)
|
||||
* `tasks`, `milestones`
|
||||
|
||||
### 🕸️ Multi-Hop Subgraph Expansion (`get_subgraph`)
|
||||
Performs a Breadth-First Search (BFS) around a target root entity node up to a specified depth ($N$ hops), returning all connected sub-entities and relationships in a single call.
|
||||
### 6. Notes, Handoffs & Reporting (4 Tools)
|
||||
* `handoff_memos`, `add_session_summary`, `generate_standup_report`, `promote_to_entity`
|
||||
|
||||
### ⚡ Automated Error Fix Auto-Matcher (`suggest_error_fix`)
|
||||
Compares build and test stack traces against historical error resolutions using dense vector embeddings and signature matching, returning past solutions, modified files, and git commits.
|
||||
|
||||
### 💾 Memory State Checkpoints & Rollbacks (`checkpoint_state` / `restore_state`)
|
||||
Saves point-in-time snapshots of graph entities, active tasks, and tech debt backlogs before risky operations, enabling seamless state restoration.
|
||||
|
||||
### 📊 Token Budgeting & RRF Search
|
||||
* **Token Budgeting**: Supports `summary_level` (`compact` | `detailed` | `full`) and `max_tokens` parameters on `tasks` (list) and `tech_debt` (list).
|
||||
* **Hybrid RRF Search**: `omni_search` combines Tantivy BM25 keyword matching with Dense Vector embeddings using Reciprocal Rank Fusion.
|
||||
* **Session Delta Resource (`memory://session/delta`)**: Delivers recent session changes in a compact context resource.
|
||||
|
||||
### 🏷️ Domain Tagging for Code Snippets (`snippets`)
|
||||
Supports categorization tags (`tags: Vec<String>`) on code snippets for category-filtered searches and domain organization.
|
||||
|
||||
### 🧹 Self-Healing Graph Sweeper (`sweep_graph_health`)
|
||||
Audits entity nodes for orphans and calculates name similarity to surface near-duplicate merge recommendations or auto-prune stale nodes.
|
||||
|
||||
### 🔗 Causal Lineage & Provenance Tracker (`query_lineage`)
|
||||
Traces the full causal chain linking tasks, ADRs, audit ledger entries, git commits, and error fixes for any query.
|
||||
|
||||
### 🎯 Topological Unblocked Task Resolver (`get_next_actionable_tasks`)
|
||||
Evaluates task dependency graphs and returns unblocked, ready-to-run tasks for subagent execution.
|
||||
|
||||
### 🧠 Chain-of-Thought & Diagnostic Hypothesis Memory (`hypotheses`)
|
||||
Records structured diagnostic hypotheses, test evidence, and verification statuses (actions: `log`, `query`) to preserve reasoning across sessions.
|
||||
|
||||
### 📡 Inter-Agent Signal Bus (`agent_signals`)
|
||||
Facilitates real-time peer-to-peer signal exchange between autonomous subagents (actions: `broadcast`, `query`) with TTL expiration and activity feeds.
|
||||
|
||||
### 🔒 Resilient Storage & Serde Parameter Tolerances
|
||||
### 📑 Process & Daemon Log Management (`process_logs`)
|
||||
Registers and tails live process and daemon log files with UTF-8 safe seeking (actions: `watch`, `get`, `clear`) to diagnose runtime behavior without reading multi-megabyte files into chat context.
|
||||
|
||||
* **Explicit Fail-Fast Persistence Safety**: Replaced unsafe silent fallback to temporary databases (`/tmp/mcp_store_fallback_*`) with an explicit open retry and fail-fast panic unless `MCP_ALLOW_TMP_FALLBACK=1` is explicitly set, preventing silent data loss.
|
||||
* **Store Write Lock Minimization**: Releases write lock immediately following in-memory mutation, serializing JSON payloads under read locks to allow non-blocking concurrent readers.
|
||||
* **Two-Phase Graph Condensation**: Employs a non-destructive 2-phase commit in `condense_graph_worker` (reading without clearing, inserting into the knowledge graph, and only pruning summarized records by timestamp/content upon verified success).
|
||||
* **Redb Transient Lock Backoff**: Added exponential backoff retry loop (3 attempts, 150ms delay) on Redb table lock acquisition to gracefully handle concurrent access contention.
|
||||
* **Offloaded Background Index Rebuilds**: Heavy graph cloning and Tantivy re-indexing in `rebuild_index` are offloaded to `tokio::task::spawn_blocking` to prevent starving Tokio async worker pools.
|
||||
* **Watch-Based Non-Destructive Shutdown**: Server cancellation signals utilize `tokio::sync::watch` rather than `mpsc` to allow multi-consumer broadcast notifications.
|
||||
* **Async Mutex Deadlock Elimination**: Converted shared state and Neovim connection locks (`shutdown_tx`, `NVIM_CONN`, `ACTIVE_SOCKET`, `HEADLESS_PROC`) to `tokio::sync::Mutex` to prevent worker thread pool starvation across `.await` points.
|
||||
* **Telemetry Session Deduplication & Channel Pruning**: Added `LAST_SESSION` in-memory state deduplication for UDP telemetry writes (eliminating disk I/O thrashing) and distinguished WebSocket `TrySendError::Full` backpressure vs `TrySendError::Closed` client pruning.
|
||||
* **Graph Adjacency Indexing**: Leverages `KnowledgeGraph::build_adjacency_map` to build $O(1)$ lookup adjacency lists for fast BFS shortest path graph queries.
|
||||
* **Serde Parameter & Enum Ergonomics**: Consolidated tools support flexible aliases (`source`/`from`, `target`/`to`, `relationType`/`relation_type`, `parent_id`/`parentId`, `camelCase`/`PascalCase`/`snake_case`) so LLM tool invocations never fail due to parameter discrepancies.
|
||||
* **Embedding Input Safeguards**: `generate_embedding_async` returns explicit errors for empty string inputs instead of 0-length fallback vectors, guaranteeing vector dimension compatibility in `cosine_similarity`.
|
||||
* **Path Traversal Security Guards**: Enforces path canonicalization (`validate_safe_path`) to reject parent relative directory traversal (`..`) across process and file log endpoints.
|
||||
* **Proactive Watcher Memory Eviction**: Caps file watcher `last_processed` map size at 1,000 items and purges items older than 10 minutes to prevent long-running memory leaks.
|
||||
* **Pre-cached Embedding Search**: Reuses pre-computed snippet embeddings (`snippet.embedding`), bypassing ONNX inference latency during in-memory semantic searches.
|
||||
### 📡 Real-time WebSocket Memory Sync (`ws://127.0.0.1:3000/ws`)
|
||||
Broadcasting event pipeline streams real-time graph, task, and activity mutations directly to the Brain Monitor UI.
|
||||
### 7. Git & Worktree Context (2 Tools)
|
||||
* `get_active_worktree_context`, `query_git_diffs`
|
||||
|
||||
---
|
||||
|
||||
## Quick Start & Usage
|
||||
## 🖥️ Brain Monitor Web UI (`http://127.0.0.1:3000/`)
|
||||
|
||||
### 1. Windows Installation (The Server & Stub)
|
||||
The server hosts a live, reactive Single Page Application (SPA) dashboard:
|
||||
* **Interactive Knowledge Graph:** Physics-simulated network graph with node-type coloring, drag-and-drop, and Inspector Panel.
|
||||
* **Universal Search & Filtering:** Instant debounced search across all tabs with keyboard shortcut (`/`) to jump to the active tab's search bar.
|
||||
* **Dynamic Pagination:** Configurable page sizes (`10`, `25`, `50`, `100`, `All`) preserving UI responsiveness across large datasets.
|
||||
* **Descending ADR Ordering:** ADRs are automatically sorted with newest IDs first (e.g., ADR-0104, ADR-0103) on Page 1.
|
||||
* **Live SSE & WebSocket Telemetry:** Real-time updates without manual browser refresh, wired to all domain mutations.
|
||||
* **Kanban Board & Audit Ledger:** Direct task state transitions and chronological code change logs.
|
||||
|
||||
To enforce strict process safety and eliminate file locks on Windows, the build, deploy, and execution lifecycle are entirely decoupled in the `justfile`.
|
||||
---
|
||||
|
||||
**The Golden Rule:** You must gracefully stop the server before deploying a new binary. Deploy recipes only copy files; they do not kill processes.
|
||||
## 🔒 Security & Concurrency Invariants
|
||||
|
||||
The easiest way to manage this end-to-end (Stop -> Build -> Deploy -> Start) is using the chaining commands:
|
||||
* **Zero Subprocess Policy**: Native system handlers (`clipboard`, `ast`, `search`, `db`) use pure native Rust crates (`arboard`, `tree-sitter`, `tantivy`, `psycopg`). Invoking external shell interpreters (`powershell.exe`, `wl-paste`, `xclip`, `cmd.exe`) is strictly prohibited.
|
||||
* **Automated Post-Commit Reconciliation**: `scripts/git-reconcile.py` (installed via `just install-git-hooks`) reconciles referenced ADR and Task statuses immediately upon commit.
|
||||
* **Atomic Store Write Lock Minimization**: Releases write lock immediately following in-memory mutation, serializing JSON payloads under read guards to prevent blocking concurrent readers.
|
||||
* **Async Mutex Deadlock Elimination**: Converted all shared state and Neovim locks to `tokio::sync::Mutex` to prevent worker thread pool starvation.
|
||||
* **Zero-Latency UDP Telemetry**: Bypasses disk I/O thrashing for high-frequency editor telemetry using deduplicated UDP streams (`MCP_UDP_PORT1`, `MCP_UDP_PORT2`).
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start & Lifecycle Management
|
||||
|
||||
### 1. Build and Deploy
|
||||
```powershell
|
||||
# For the main server:
|
||||
# Build and deploy everything across Windows and WSL
|
||||
just all
|
||||
|
||||
# Or deploy Windows server with graceful staged hot-swap
|
||||
just all-server-win
|
||||
|
||||
# For the lightweight stubs/nvim servers:
|
||||
just all-stub-win
|
||||
just all-nvim-win
|
||||
# Install Git post-commit reconciliation hook
|
||||
just install-git-hooks
|
||||
```
|
||||
|
||||
If you want to perform these steps manually, follow this exact order:
|
||||
### 2. Service Management
|
||||
```powershell
|
||||
just stop # 1. Gracefully shut down the background server (TCP 3000)
|
||||
just build-win # 2. Compile the binaries
|
||||
just deploy-win # 3. Move the executables into ~/.local/bin/
|
||||
just start # 4. Spawns the daemon completely detached in the background
|
||||
just verify # 5. Hits the /ping endpoint to ensure liveness
|
||||
just start # Start background server on port 3000
|
||||
just stop # Gracefully shut down server
|
||||
just restart # Graceful restart with health check verification
|
||||
just verify # Verify deployment health
|
||||
just version # Check running API version and CLI version
|
||||
```
|
||||
|
||||
**Auto-Start Configuration:** To ensure the background server is always available, add this to your PowerShell profile:
|
||||
### 3. Testing & Parity
|
||||
```powershell
|
||||
if ($host.Name -eq 'ConsoleHost' -and -not (Get-Process mcp-memory-server -ErrorAction SilentlyContinue)) {
|
||||
Start-Process -FilePath "C:\Users\reazul.ashraf\.local\bin\mcp-memory-server.exe" -WindowStyle Hidden -ErrorAction SilentlyContinue
|
||||
}
|
||||
just test # Run fast parallel tests via cargo-nextest & type-check UI
|
||||
just test-config # Verify eagerTools configuration parity
|
||||
just test-ui # Verify dashboard UI endpoint and HTML integrity
|
||||
```
|
||||
|
||||
**Shutting Down & Managing:**
|
||||
```powershell
|
||||
just start
|
||||
just stop
|
||||
just restart
|
||||
```
|
||||
|
||||
### 2. Config Setup (`mcp_config.json`)
|
||||
|
||||
Update your `~/.gemini/config/mcp_config.json`:
|
||||
### 4. Agent Configuration (`mcp_config.json`)
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"memory": {
|
||||
"command": "C:\\Users\\reazul.ashraf\\.local\\bin\\mcp-memory-stub.exe",
|
||||
"mcp-memory": {
|
||||
"command": "C:\\Users\\reazul.ashraf\\.gemini\\antigravity-cli\\mcp\\mcp-memory\\mcp-memory.exe",
|
||||
"args": []
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Brain Monitor Dashboard
|
||||
|
||||
The server hosts a live, real-time SPA dashboard called the **Brain Monitor**.
|
||||
|
||||
To view the dashboard, open your browser at:
|
||||
`http://127.0.0.1:3000/`
|
||||
|
||||
### Dashboard Features:
|
||||
* **Interactive Knowledge Graph:** Physics-simulated network graph with node-type coloring, drag-and-drop, and Inspector Panel.
|
||||
* **Kanban Board:** Track active Tasks and trigger status transitions directly from the browser.
|
||||
* **Clipboard Inspector:** Review OS-level clipboard image captures via `/api/clipboard/capture`.
|
||||
* **Live WebSocket Telemetry:** Real-time UI updates triggered by server state changes.
|
||||
* **Code Change Ledger:** Chronological audit trail of all code edits, commits, and summaries.
|
||||
|
||||
---
|
||||
|
||||
## High-Performance Concurrency & Resilience Guarantees
|
||||
|
||||
* **Atomic Store Write Lock Minimization**: `Store::modify` and `Store::modify_async` release write lock guards immediately after applying state mutations, performing JSON serialization under read guards to prevent blocking concurrent readers during state serialization.
|
||||
* **Async Commit Index Reader Auto-Reload**: `MemoryIndex::commit()` automatically reloads index searchers upon background commit completion, eliminating search latency and stale reader windows.
|
||||
* **Async Channel Backpressure (`push_async`)**: `Store::modify_async` uses `DbWriteQueue::push_async` with `tx.send(task).await` backpressure to guarantee database write persistence under heavy async write loads without dropping write transactions.
|
||||
* **Atomic Search Index Swaps**: `MemoryState::rebuild_index` constructs and populates a new `MemoryIndex` instance in isolation before performing an atomic pointer swap (`*self.search_index.write().await = new_idx`), eliminating transient empty search result windows.
|
||||
* **Dynamic Character Micro-Batching**: `generate_embeddings_async` dynamically batches text payloads up to a 16,000 character budget inside `spawn_blocking`, eliminating heap spikes during high-volume vector indexing while keeping SIMD pipelines saturated.
|
||||
* **Bounded Telemetry Detail Records**: Activity and terminal telemetry buffers enforce a 4,000 character truncation ceiling on log details (`ActivityRecord`, `TerminalHistory`) to prevent unbounded RAM growth under heavy RPC traffic.
|
||||
* **Zero-Allocation Stream Formatting**: Graph condensation loops (`condense_graph_worker`) use `std::fmt::Write` string stream buffers to format subgraphs without allocating temporary string intermediates.
|
||||
* **SIMD-Friendly Single-Pass Cosine Similarity**: `cosine_similarity` calculates dot product and Euclidean norm squares in a single iterator fold pass over float vectors, enabling SIMD compiler auto-vectorization.
|
||||
* **Safe Stream Decoding on Log Tails**: Process log tailing (`process_logs`, action: `get`) reads raw bytes and decodes using lossy UTF-8 conversion (`String::from_utf8_lossy`) to ensure resilience when seeking across multi-byte UTF-8 boundaries.
|
||||
* **Serde Parameter & Enum Tolerance**: All action enums (`HandoffMemoAction`, `HypothesisAction`, `AgentSignalAction`, `ProcessLogAction`, `SnippetSearchMode`, `Relation`) support case-insensitive variants and field aliases (`source`/`from`, `target`/`to`, `relationType`/`relation_type`) to ensure seamless execution when LLMs pass varied string formatting.
|
||||
@@ -12,26 +12,40 @@ description: Strict guidelines for interacting with the mcp-memory server, ensur
|
||||
## 2. Proactive "Central Brain" Usage
|
||||
|
||||
> [!NOTE] Two-Tier Memory Architecture
|
||||
> 1. **Tier 1 (Static Markdown)**: Repository rules, coding style, tech constraints, and architectural boundaries belong in static git-tracked markdown (`rules/*.md`, `instructions.md`) and system prompts for 0ms latency and deterministic turn-0 enforcement.
|
||||
> 1. **Tier 1 (Static Markdown)**: Repository rules, coding style, tech constraints, and architectural boundaries belong in static git-tracked markdown (`agent-rules/*.md`, `instructions.md`) and system prompts for 0ms latency and deterministic turn-0 enforcement.
|
||||
> 2. **Tier 2 (Structured DB & Telemetry)**: The MCP Memory server specializes in high-volume, dynamic data: file modification ledgers (`audit_ledger`), terminal command history, error resolutions (`log_error_fix`), active tasks, and preflight context aggregation.
|
||||
|
||||
The MCP Memory server is the central brain. You must be PROACTIVE, not reactive, in using it:
|
||||
- **Session Starts & Context Drops**: Always begin by calling `tasks` (action: "list"), `get_preflight_context`, and `omni_search` to regain context.
|
||||
|
||||
### Passive Resource Retrieval (Zero-Turn Latency)
|
||||
Before making expensive active tool calls, read available MCP resources:
|
||||
- **`memory://tasks/active`**: Currently uncompleted tasks, priorities, and criteria.
|
||||
- **`memory://decisions/active`**: Active architectural decisions in `accepted` status.
|
||||
- **`memory://tech_debt/unresolved`**: Open engineering tech debt items.
|
||||
- **`memory://session/delta`**: Recent changes, active tasks, code edits, and notes created in the last 2 hours.
|
||||
- **`memory://terminal/recent`**: Recent terminal commands, interpreters (`pwsh`, `bash`, `nu`), working dirs, and exit codes.
|
||||
- **`memory://activity/recent`**: Real-time IDE and developer activity event logs.
|
||||
- **`memory://milestones`**: Project milestones and deliverable tracking.
|
||||
- **`memory://graph/entities` & `memory://graph/relations`**: Knowledge graph snapshots.
|
||||
|
||||
### Active Tool Invocations
|
||||
- **Session Starts & Context Drops**: Begin by checking `memory://tasks/active` and `memory://session/delta` (or running `context_warmup` prompt), calling `get_preflight_context` and `omni_search` to regain operational context.
|
||||
- **Context Switching**: When switching tasks or branches, use `manage_checkpoint` (action: "create") to freeze state, and use `manage_checkpoint` (action: "restore") to restore state for the task.
|
||||
- **Error Fixes**: The moment a tricky, undocumented, or environment-specific bug is resolved (e.g., Bitbucket markdown rendering quirks, nuanced framework bugs), IMMEDIATELY call `log_error_fix`. Supply `repo_name`, `error_category`, and `stack_trace` so future searches can perform embedding-based match.
|
||||
- **Tech Debt**: If you notice an anti-pattern (e.g., nested `if` statements, arrow anti-pattern) but deliberately skip fixing it to focus on a feature, IMMEDIATELY call `tech_debt` (action: "log") with `description`, `file_path`, `line_range`, `workaround`, `effort_estimate`, and `severity`.
|
||||
- **Error Fixes**: The moment a tricky, undocumented, or environment-specific bug is resolved, IMMEDIATELY call `log_error_fix`. Supply `repo_name`, `error_category`, and `stack_trace` so future searches can perform embedding-based match via `search_error_fixes`.
|
||||
- **Tech Debt**: If you notice an anti-pattern but deliberately skip fixing it to focus on a feature, IMMEDIATELY call `tech_debt` (action: "log") with `description`, `file_path`, `line_range`, `workaround`, `effort_estimate`, and `severity`.
|
||||
- **Architectural Decisions (ADR) & Lifecycle Closure**:
|
||||
- When selecting design patterns, crate choices, or system structure, call `decisions` (action: "log") with `author`, `affected_components`, `alternatives_considered`, `decision`, and `consequence`.
|
||||
- **MANDATORY Definition of Done**: When code implementing an ADR is committed, you MUST IMMEDIATELY call `decisions` (action: "update", id: "ADR-XXXX", status: "implemented", git_commit: <commit_hash>, git_branch: <branch>). NEVER leave an ADR in `accepted` once the implementing code is committed. The user should NEVER have to manually flag or remind that an implemented ADR is still marked as 'accepted'.
|
||||
- **MANDATORY Definition of Done**: When code implementing an ADR is committed, you MUST IMMEDIATELY call `decisions` (action: "update", id: "ADR-XXXX", status: "implemented", git_commit: <commit_hash>, git_branch: <branch>). NEVER leave an ADR in `accepted` once the implementing code is committed. The repository also executes `scripts/git-reconcile.py` on post-commit hooks (`just install-git-hooks`) to reconcile commit references automatically.
|
||||
- **Task Management**: When creating tasks, supply `priority` ('low'|'medium'|'high'|'urgent'), `assigned_agent` (e.g. subagent role), `verification_command` (automated test command), and `acceptance_criteria`.
|
||||
- **VCS & SVN Agnosticism**: Supply `vcs_type` ('git'|'svn'|'hg'), `vcs_revision` (git hash or svn revision like 'r12345'), and `upstream_url` to `log_code_change` and workspace tools.
|
||||
- **Terminal & Shell Context**: Terminal sessions and commands are automatically tracked in the server. Query `/terminal/history` or recent logs when analyzing shell execution context.
|
||||
- **Terminal & Shell Context**: Terminal sessions and commands are automatically tracked in the server over zero-latency UDP. Query `/terminal/history` or `memory://terminal/recent` when analyzing shell execution context.
|
||||
- **Hypotheses & Root Cause Analysis**: When diagnosing complex bugs or race conditions, call `hypotheses` (action: "log" / "query") to record test evidence and maintain reasoning trails across sessions.
|
||||
- **Inter-Agent Coordination**: Autonomous subagents should call `agent_signals` (action: "broadcast" / "query") to publish events and discover peer agent status.
|
||||
- **Process & Daemon Logs**: Query or tail daemon logs with `process_logs` (action: "get", "watch", "clear") instead of dumping log files into context.
|
||||
- **End-of-Session Handoff**: When finishing work or logging off, trigger the `handoff_routine` prompt or call `generate_standup_report` and `handoff_memos` (action: "leave").
|
||||
|
||||
## 3. Delegation
|
||||
Continue to use the `MemoryLibrarian` subagent to log routine code changes (`log_code_change`) in the background to prevent cluttering the main conversation context.
|
||||
Continue to use the `MemoryLibrarian`, `PrePushAuditor`, `BugDiagnostician`, `ScrumMaster`, and `DevOpsSRE` subagents to offload graph curation, pre-push auditing, hypothesis testing, task tracking, and session handoffs.
|
||||
|
||||
## 4. Performance & Batching Rules
|
||||
- **Batch Mutating Operations**: When creating or updating multiple graph entities, code snippets, or observations, always batch items into a single tool call array (e.g. `create_entities` with multiple items) to leverage the server's single-pass transaction flush.
|
||||
|
||||
+144
-52
@@ -1,7 +1,7 @@
|
||||
# Memory MCP Strategic Guidelines
|
||||
|
||||
This document outlines the STRATEGY, SEMANTICS, and CASING STANDARDS for using the MCP Memory Server.
|
||||
You do not need to memorize JSON schemas for these tools; they are strictly defined and typed in the `tools/list` endpoint. Focus purely on WHEN and WHY to use them.
|
||||
This document outlines the STRATEGY, SEMANTICS, RESOURCE SCHEMAS, and CASING STANDARDS for using the MCP Memory Server.
|
||||
You do not need to memorize individual JSON schemas for every tool; they are strictly defined and typed in the `tools/list` endpoint. Focus on WHEN, WHY, and HOW to leverage them effectively.
|
||||
|
||||
---
|
||||
|
||||
@@ -19,12 +19,45 @@ To prevent graph fragmentation and ensure seamless LLM context retrieval:
|
||||
|
||||
---
|
||||
|
||||
## 2. Consolidated Smart Tools Architecture
|
||||
## 2. The Two-Tier Context Paradigm
|
||||
|
||||
> [!IMPORTANT] The Two-Tier Context Paradigm
|
||||
> * **Tier 1 (Static Markdown)**: Repository rules, constraints, architectural patterns, and developer preferences are maintained directly in static git-tracked files (`rules/*.md`, `instructions.md`) and system prompts. This guarantees 0ms turn-0 availability without relying on proactive agent tool retrieval.
|
||||
> [!IMPORTANT]
|
||||
> * **Tier 1 (Static Markdown)**: Repository rules, constraints, architectural patterns, and developer preferences are maintained directly in static git-tracked files (`agent-rules/*.md`, `instructions.md`) and system prompts. This guarantees 0ms turn-0 availability without relying on proactive agent tool retrieval.
|
||||
> * **Tier 2 (Telemetry & Ephemeral DB)**: High-frequency operational history—the Code Change Ledger (`audit_ledger`), terminal command history, compiler error fixes (`log_error_fix`), active tasks, and preflight context—is handled by the MCP Memory Server and surfaced via the Brain Monitor Web UI and MCP resources.
|
||||
|
||||
---
|
||||
|
||||
## 3. Passive MCP Resources (`resources/list`)
|
||||
|
||||
The server exposes 9 real-time, read-only MCP resources. Agents should read these resources directly to regain context without incurring tool call latency:
|
||||
|
||||
| Resource URI | Resource Name | Description & Usage |
|
||||
|:---|:---|:---|
|
||||
| `memory://graph/entities` | Graph Entities | All nodes and entities in the knowledge graph (top 100 with pagination guidance). |
|
||||
| `memory://graph/relations` | Graph Relations | All relationships between graph entities (top 200 with subgraph guidance). |
|
||||
| `memory://tasks/active` | Active Tasks | Current active tasks with status, priority, and assigned subagents. |
|
||||
| `memory://decisions/active` | Active ADR Decisions | Architectural decisions currently in `accepted` status. |
|
||||
| `memory://tech_debt/unresolved` | Unresolved Tech Debt | All open engineering debt items requiring future refactoring. |
|
||||
| `memory://session/delta` | Session Delta | Code modifications, commits, active tasks, and notes created in the last 2 hours. |
|
||||
| `memory://terminal/recent` | Terminal History | Recent terminal commands, interpreters (`pwsh`, `bash`, `nu`), working dirs, and exit codes. |
|
||||
| `memory://activity/recent` | Recent Activity | Real-time IDE and developer activity event stream. |
|
||||
| `memory://milestones` | Milestones | Project milestones, deliverables, target dates, and progress. |
|
||||
|
||||
---
|
||||
|
||||
## 4. MCP Workflow Prompts (`prompts/list`)
|
||||
|
||||
The server registers 5 high-signal workflow prompts to initiate standardized agent routines:
|
||||
|
||||
1. **`context_warmup`**: Executed at session startup. Prompts the agent to read `memory://tasks/active` and `memory://session/delta`, inspect the workspace worktree, and assemble immediate working context.
|
||||
2. **`analyze_tech_debt`**: Prompts the agent to review unresolved technical debt from `memory://tech_debt/unresolved` and generate a prioritized remediation plan.
|
||||
3. **`summarize_architecture`**: Synthesizes active ADRs from `memory://decisions/active` and graph entities from `memory://graph/entities` into an architectural overview.
|
||||
4. **`handoff_routine`**: Triggers the `DevOpsSRE` subagent at session end to generate a standup report, audit active tasks, and record a handoff memo for future sessions.
|
||||
5. **`archive_routine`**: Compresses historical session summaries into a dense milestone retrospective entity and purges pruned entries.
|
||||
|
||||
---
|
||||
|
||||
## 5. Consolidated Smart Tools Architecture (11 Primary Tools)
|
||||
|
||||
The server consolidates granular single-purpose tools into domain-named smart tools. Always prefer the consolidated tools over legacy aliases:
|
||||
|
||||
@@ -53,7 +86,8 @@ The server consolidates granular single-purpose tools into domain-named smart to
|
||||
- `action: "tag"`: Attach classification tags (requires `id`, `tags: Vec<String>`).
|
||||
|
||||
* **`decisions`**: Architectural Decision Records (ADRs).
|
||||
- `action: "log"`: Log ADR (requires `title`, optional `status: "accepted" | "proposed" | "deprecated" | "superseded"`, `context`, `decision`, `consequence`, `author`, `affected_components: Vec<String>`, `alternatives_considered: Vec<String>`, `supersedes`, `repo_name`).
|
||||
- `action: "log"`: Log ADR (requires `title`, optional `status: "accepted" | "proposed" | "deprecated" | "superseded" | "implemented"`, `context`, `decision`, `consequence`, `author`, `affected_components: Vec<String>`, `alternatives_considered: Vec<String>`, `supersedes`, `repo_name`).
|
||||
- `action: "update"`: Update ADR status and metadata (requires `id`, optional `status`, `git_commit`, `git_branch`).
|
||||
- `action: "query"`: Query ADRs (optional `query`).
|
||||
- `action: "delete"`: Delete ADR (requires `id`).
|
||||
|
||||
@@ -88,89 +122,155 @@ The server consolidates granular single-purpose tools into domain-named smart to
|
||||
|
||||
---
|
||||
|
||||
## 3. VCS & SVN Agnosticism & Multi-Repo Provenance
|
||||
## 6. Complete Tool Catalog (All 53 Tools)
|
||||
|
||||
The server exposes 53 specialized and smart MCP tools organized into 7 functional domains:
|
||||
|
||||
### 1. Knowledge Graph Core (18 Tools)
|
||||
1. `create_entities`: Batch-create entities with `name`, `entity_type`, and `observations`.
|
||||
2. `create_relations`: Batch-create relationships (`from`, `to`, `relation_type`).
|
||||
3. `add_observations`: Append observations to existing entities.
|
||||
4. `delete_entities`: Delete entity nodes and cascading relations.
|
||||
5. `delete_relations`: Delete specific relation edges between entities.
|
||||
6. `delete_observations`: Remove specific observations from an entity.
|
||||
7. `read_graph`: Return full or namespace-filtered knowledge graph.
|
||||
8. `search_nodes`: Search entity names and observations using Tantivy BM25.
|
||||
9. `open_nodes`: Inspect full details of specified entity nodes by name.
|
||||
10. `visualize_graph`: Generate Mermaid markdown or SVG diagram of the graph.
|
||||
11. `condense_entity`: Summarize entity observations into dense summaries.
|
||||
12. `merge_entities`: Merge source entity into target entity, re-pointing relations and pruning self-loops.
|
||||
13. `find_orphans`: Detect entities with zero relationships for pruning.
|
||||
14. `get_subgraph`: BFS graph traversal expanding $N$ hops from a root node.
|
||||
15. `sweep_graph_health`: Audit graph for orphans, calculate name similarity, and recommend merges.
|
||||
16. `resolve_stale_symbols`: Cross-reference graph symbols against the workspace AST to remove deleted code nodes.
|
||||
17. `summarize_subgraph`: LLM-ready concise synthesis of a localized subgraph.
|
||||
18. `query_graph_path`: BFS shortest path between two entities in the knowledge graph.
|
||||
|
||||
### 2. Task & Milestone Operations (2 Tools)
|
||||
19. `tasks`: Consolidated task board manager (`add`, `update`, `delete`, `list`, `set_criteria`, `verify`).
|
||||
20. `milestones`: Milestone lifecycle management (`add`, `update`, `list`).
|
||||
|
||||
### 3. Notes, Handoffs & Reporting (4 Tools)
|
||||
21. `handoff_memos`: Cross-session scratchpad and handoff memos (`leave`, `read`, `clear`).
|
||||
22. `add_session_summary`: Record session summary notes and highlights.
|
||||
23. `generate_standup_report`: Synthesize tasks, ledger changes, and session summaries into a standup report.
|
||||
24. `promote_to_entity`: Promote an ephemeral note or memo into a permanent knowledge graph entity.
|
||||
|
||||
### 4. Meta, Audit & Intelligence (15 Tools)
|
||||
25. `decisions`: Consolidated Architectural Decision Records (ADRs) manager (`log`, `update`, `query`, `delete`).
|
||||
26. `tech_debt`: Consolidated technical debt backlog manager (`log`, `resolve`, `list`).
|
||||
27. `log_error_fix`: Record an error resolution with stack trace, root cause, and git commit.
|
||||
28. `search_error_fixes`: Embedding-based and keyword search over past error resolutions.
|
||||
29. `log_code_change`: Record a file modification in the VCS-agnostic audit ledger.
|
||||
30. `query_recent_changes`: Retrieve recent code changes with lookback time filters.
|
||||
31. `omni_search`: Reciprocal Rank Fusion (RRF) search across all graph entities, snippets, ADRs, debt, and fixes.
|
||||
32. `get_project_health`: Health dashboard summarizing task completion, debt backlog, and graph consistency.
|
||||
33. `manage_checkpoint`: Create or restore named memory snapshots for safe rollback.
|
||||
34. `query_lineage`: Causal lineage tracker linking tasks, ADRs, commits, and error fixes.
|
||||
35. `get_next_actionable_tasks`: Topologically resolved list of unblocked tasks ready for execution.
|
||||
36. `hypotheses`: Structured diagnostic hypothesis tracker (`log`, `query`).
|
||||
37. `get_preflight_context`: Aggregated operational context at session start (tasks, debt, recent changes).
|
||||
38. `agent_signals`: Inter-agent signal bus (`broadcast`, `query`).
|
||||
39. `auto_session_checkpoint`: Automatic session boundary checkpointing.
|
||||
|
||||
### 5. System, Environment & Telemetry (4 Tools)
|
||||
40. `environment`: Tool fingerprinting, requirements, and environment registry (`update_fingerprint`, `read_fingerprint`, `log_requirement`, `register`, `get_details`).
|
||||
41. `snippets`: Reusable code snippet vault with hybrid search (`store`, `search`, `delete`, `tag`).
|
||||
42. `clipboard`: Pure native Rust OS clipboard interface (`read`, `write`).
|
||||
43. `process_logs`: Live process and daemon log watcher and tailer (`watch`, `get`, `clear`).
|
||||
|
||||
### 6. Git & Worktree Context (2 Tools)
|
||||
44. `get_active_worktree_context`: Inspect git status, modified files, diff summary, and current branch.
|
||||
45. `query_git_diffs`: Retrieve detailed git diffs for specific files or commit ranges.
|
||||
|
||||
### 7. AST & Code Intelligence (8 Tools)
|
||||
46. `read_file_skeleton`: Tree-sitter AST structural outline of functions, structs, and methods without implementation bodies.
|
||||
47. `replace_ast_node`: Precise AST node replacement preserving indentation and comments.
|
||||
48. `find_symbol_references`: Search for symbol references across snippets and disk source files.
|
||||
49. `get_callers`: Find call sites and callers of a specified function or method across the codebase.
|
||||
50. `analyze_impact`: Blast-radius impact analysis of modifying a symbol or file.
|
||||
51. `read_directory_architecture`: Recursive directory structure analysis capped at depth 10.
|
||||
52. `semantic_code_search`: Dense vector semantic code search over indexed source code.
|
||||
53. `manage_subagent_namespace`: Create, isolate, or merge subagent-scoped memory namespaces.
|
||||
|
||||
---
|
||||
|
||||
## 7. VCS & SVN Agnosticism & Multi-Repo Provenance
|
||||
|
||||
To support diverse enterprise repositories (Git, Subversion / SVN, Mercurial / Hg, Monorepos):
|
||||
* **`vcs_type`**: Designates the VCS engine (`"git"`, `"svn"`, `"hg"`, `"perforce"`, or `"none"`).
|
||||
* **`vcs_revision`**: Agnostic commit hash or SVN revision identifier (e.g., `"r12458"`, `"3e4f7a9"`).
|
||||
* **`upstream_url`**: Canonical remote repository URL (e.g. `https://svn.corp/repo/trunk`, `git@bitbucket.org:org/repo.git`).
|
||||
* **`repo_name`**: Logical project or repository identifier allowing multiple repositories to share or partition memory namespaces cleanly without collision.
|
||||
* **`repo_name`**: Logical project identifier allowing multiple repositories to share or partition memory namespaces cleanly without collision.
|
||||
* **Audit Ledger (`log_code_change`)**: Enriched with `vcs_type`, `vcs_revision`, `upstream_url`, `author`, `diff_summary`, and extensible `metadata: HashMap<String, String>`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Terminal & Process Telemetry
|
||||
## 8. Terminal & Process Telemetry
|
||||
|
||||
The server ingests and tracks active terminal commands and sessions:
|
||||
* **Active Terminals**: Tracks PIDs, shell interpreters (`pwsh`, `bash`, `nu`, `zsh`), current working directories (`cwd`), command exit codes, and timestamps.
|
||||
* **Terminal History Endpoint**: `/terminal/history` exposes recent shell commands and output streams to dashboard and LLMs to prevent lost shell context.
|
||||
* **Quality Gate Enforcement**: `GateRecord` captures pre-flight and pre-push validation passes with `gate_type`, `enforcer`, `status`, `validation_log`, and `repo_name`.
|
||||
* **Terminal History Endpoint & Resource**: `/terminal/history` and `memory://terminal/recent` expose recent shell commands and output streams to dashboard and LLMs to prevent lost shell context.
|
||||
* **Zero-Latency UDP Streams**: Terminal and IDE telemetry stream over UDP (`MCP_UDP_PORT1`, `MCP_UDP_PORT2`) with zero disk I/O bottlenecks.
|
||||
|
||||
---
|
||||
|
||||
## 4. High-Signal Tool Responses & Performance Guidelines
|
||||
|
||||
To optimize context usage, response times, and LLM reasoning efficiency:
|
||||
|
||||
* **High-Signal Feedback**:
|
||||
- MCP tool calls (such as `create_entities`, `create_relations`, `add_observations`, `pin_file`) return explicit, structured summaries containing created names, types, edge counts, and relation paths.
|
||||
- LLMs do NOT need to execute follow-up `open_nodes` calls merely to confirm successful creation.
|
||||
|
||||
* **Batch Operations**:
|
||||
- When creating or updating multiple entities, snippets, or observations, always batch items into a single tool call array (e.g. `create_entities` with multiple array items) rather than making separate calls.
|
||||
- The server handles batch store mutations serially in a single transaction pass with single-permit event-driven flushes.
|
||||
|
||||
* **Real-time Tantivy Search Indexing**:
|
||||
- The Tantivy search engine automatically checks pending commits and reloads search readers prior to executing `omni_search` or `search_nodes`. Search queries always return up-to-date document results immediately following mutations.
|
||||
- Single-item deletions use targeted document removal rather than global index wipes.
|
||||
|
||||
* **Real-Time AST & Workspace Source Code Symbol Scanning**:
|
||||
- `find_symbol_references`, `get_callers`, and `analyze_impact` scan both stored code snippets and physical workspace source code files on disk (`.rs`, `.ts`, `.py`, `.go`, `.java`, `.c`, `.cpp`), providing accurate AST symbol references and call site tracking.
|
||||
|
||||
* **Graph Entity Merge & Self-Loop Protection**:
|
||||
- `merge_entities` re-points all relations from `source_entity` to `target_entity` and automatically prunes cyclic self-loops (`target -> target`).
|
||||
|
||||
* **Safe UTF-8 Token Truncation**:
|
||||
- Large responses (e.g. `get_active_worktree_context`, `read_graph`, `summarize_subgraph`) are safely truncated along UTF-8 character boundaries (`floor_char_boundary`), ensuring response bounds without runtime panics.
|
||||
|
||||
## 4. Automated Error Fix Auto-Matcher
|
||||
## 9. Automated Error Fix Auto-Matcher
|
||||
- **Tools:** `log_error_fix`, `search_error_fixes` (and alias `suggest_error_fix`)
|
||||
- **When to use:** When encountering a build error, test failure, or stack trace. Call `search_error_fixes` with either a text `query` or `stack_trace` before attempting a fix from scratch.
|
||||
- **Behavior:** Computes cosine similarity between error trace embeddings and past resolution logs when `stack_trace` is provided, or keyword filtering when `query` is provided, returning top matched solutions, modified files, and git commits.
|
||||
|
||||
---
|
||||
|
||||
## 5. Memory State Checkpointing & Rollbacks
|
||||
- **Tool:** `checkpoint_state`, `restore_state` (or `create_snapshot`, `restore_snapshot`)
|
||||
## 10. Memory State Checkpointing & Rollbacks
|
||||
- **Tool:** `manage_checkpoint` (action: `"create"` | `"restore"`)
|
||||
- **When to use:** Before initiating a large refactor, running experimental subagent tasks, or executing destructive batch operations.
|
||||
- **Behavior:** Saves or restores a point-in-time snapshot of graph entities, active tasks, and tech debt backlogs.
|
||||
|
||||
---
|
||||
|
||||
## 6. Self-Healing Graph Health Sweeper
|
||||
## 11. Self-Healing Graph Health Sweeper
|
||||
- **Tool:** `sweep_graph_health`
|
||||
- **When to use:** Periodically or before committing major graph changes to audit entity consistency.
|
||||
- **Behavior:** Detects orphaned nodes (0 relations), computes name similarity using pre-computed lowercase keys to identify near-duplicates (e.g., `APIGateway` vs `ApiGateway`), and provides structured `merge_entities` recommendations or auto-prunes orphans.
|
||||
|
||||
---
|
||||
|
||||
## 7. Causal Lineage & Provenance Tracker
|
||||
## 12. Causal Lineage & Provenance Tracker
|
||||
- **Tool:** `query_lineage`
|
||||
- **When to use:** When asking *"Why was this component modified?"* or *"What task or ADR led to this code change?"*
|
||||
- **Behavior:** Searches across tasks, ADRs, audit ledger entries, and error fixes to assemble a unified chronological timeline explaining the provenance behind any file, symbol, or commit.
|
||||
|
||||
---
|
||||
|
||||
## 9. Native Rust Invariants & Subprocess Prohibition (CRITICAL)
|
||||
## 13. ADR Lifecycle & Automated Git Post-Commit Reconciliation
|
||||
|
||||
- **The Golden ADR Rule**: When code implementing an ADR is committed, you MUST IMMEDIATELY update the ADR status to `implemented`:
|
||||
```json
|
||||
{
|
||||
"action": "update",
|
||||
"id": "ADR-XXXX",
|
||||
"status": "implemented",
|
||||
"git_commit": "<commit_hash>",
|
||||
"git_branch": "<branch>"
|
||||
}
|
||||
```
|
||||
- **Automated Post-Commit Hook**: The repository provides an automated reconciliation script (`scripts/git-reconcile.py`) installed via `just install-git-hooks`. Upon every `git commit`, the hook scans the commit message for `ADR-XXXX` or task identifiers and reconciles their status in the persistent store.
|
||||
- **Current Architecture Status**: 100% of defined ADRs (ADR-0080 through ADR-0104) are fully implemented and reconciled in the store.
|
||||
|
||||
---
|
||||
|
||||
## 14. Native Rust Invariants & Subprocess Prohibition (CRITICAL)
|
||||
|
||||
To maintain maximum security, speed, and cross-platform reliability:
|
||||
* **Zero Subprocess Fallbacks**: System and server tools (`clipboard`, `ast`, `search`, `db`) MUST strictly use pure native Rust crates (`arboard`, `tree-sitter`, `tantivy`, `psycopg`). Invocations of external shell commands (`powershell.exe`, `wl-paste`, `xclip`, `cmd.exe`) are strictly prohibited in native handlers.
|
||||
* **Transient Lock Recovery**: Transient OS handle collisions (such as Win32 OLE `OpenClipboard` lock contention) must be handled using native retry loops with backoffs directly in Rust.
|
||||
* **Automated Static Regression Gates**: Automated AST/source audit tests (e.g. `test_no_subprocess_clipboard_regression`) verify at test time that forbidden subprocess patterns are absent from handler implementations.
|
||||
* **Automated Static Regression Gates**: Automated AST/source audit tests (`test_no_subprocess_clipboard_regression`) verify at test time that forbidden subprocess patterns are absent from handler implementations.
|
||||
|
||||
---
|
||||
|
||||
## 10. High-Performance Concurrency & Resilience Guarantees
|
||||
## 15. High-Performance Concurrency & Resilience Guarantees
|
||||
|
||||
* **Explicit Fail-Fast Persistence Safety**: Replaced silent fallback to temporary databases (`/tmp/mcp_store_fallback_*`) with an explicit open retry and fail-fast panic unless `MCP_ALLOW_TMP_FALLBACK=1` is explicitly set, preventing silent data loss.
|
||||
* **Async Mutex Deadlock Elimination**: Converted shared state and Neovim connection locks (`shutdown_tx`, `NVIM_CONN`, `ACTIVE_SOCKET`, `HEADLESS_PROC`) to `tokio::sync::Mutex` to prevent worker thread pool starvation across `.await` points.
|
||||
* **Telemetry Session Deduplication & Channel Pruning**: Added `LAST_SESSION` in-memory state deduplication for UDP telemetry writes (eliminating disk I/O thrashing) and distinguished WebSocket `TrySendError::Full` backpressure vs `TrySendError::Closed` client pruning.
|
||||
@@ -191,7 +291,6 @@ To maintain maximum security, speed, and cross-platform reliability:
|
||||
* **AST Recursion Depth Safeguard & Zero-Copy Borrowing**: Tree-sitter AST traversal caps recursion depth at 100 to prevent thread stack overflows and borrows string slices (`&str`) during AST node walking.
|
||||
* **Strongly-Typed SearchResult & Pre-Allocated Search Vectors**: `search.rs` uses a strongly-typed `SearchResult` struct with named fields and pre-allocates result vector capacity (`Vec::with_capacity(top_docs.len())`).
|
||||
* **BFS Graph Traversal Pre-allocation & Visited Node Upper Bound**: `GraphQueryBuilder::find_shortest_path` pre-allocates adjacency map capacity (`HashMap::with_capacity(relations.len() * 2)`) and enforces a visited node upper bound (10,000 max) to guarantee deterministic BFS runtime.
|
||||
* **LLM Tool Schema Ergonomics & Context Guidance**: `ReadGraphHandler` schema explicitly instructs LLMs on `namespace` filtering and `search_nodes` / `get_subgraph` tools for large graph discovery.
|
||||
* **Filesystem Event Debouncing & Proactive State Refresh**: `spawn_watcher` implements a sliding 250ms debouncing window per file path, ignores `.git`, `target`, `.gemini`, and `node_modules`, and broadcasts activity events to `MemoryState`.
|
||||
* **Buffered Line-by-Line AST Workspace Symbol Scanning**: `scan_workspace_for_symbol` reads workspace files via `BufReader` line streams instead of loading entire files into heap strings, preventing memory spikes when traversing source trees.
|
||||
* **AST Node Type Aliasing & Skeleton Preallocation**: `replace_ast_node` documents friendly node aliases (`function`, `fn`, `method`, `struct`, `class`, `enum`, `trait`, `type`), and `read_file_skeleton` preallocates string buffer capacity (`code.len() / 2`).
|
||||
@@ -200,20 +299,13 @@ To maintain maximum security, speed, and cross-platform reliability:
|
||||
* **SIMD-Friendly Single-Pass Cosine Similarity**: `cosine_similarity` calculates dot product and Euclidean norm squares in a single linear pass over float vectors, enabling SIMD compiler auto-vectorization.
|
||||
* **Safe Stream Decoding on Log Tails**: Log tail operations (`process_logs`, action: "get") read raw bytes and decode using lossy UTF-8 conversion (`String::from_utf8_lossy`) to ensure resilience when seeking across multi-byte UTF-8 boundaries.
|
||||
* **Task Summary UTF-8 Truncation Safety**: `tasks` tool (`action = "list"`) truncates serialized task text strictly along UTF-8 character boundaries using `floor_char_boundary` when enforcing `max_tokens`.
|
||||
* **Sequential Snapshot Lock Scope Flattening**: `GenerateStandupReportHandler` reads `tasks`, `ledger`, and `session_summaries` sequentially rather than nesting read locks, preventing multi-lock deadlocks during concurrent store modifications.
|
||||
* **Directory Tree Depth Safeguard**: `ReadDirectoryArchitectureHandler` caps directory recursion at depth 10 to prevent stack overflow on deep or cyclic directory structures.
|
||||
* **Deterministic Total-Order Score Ranking**: `OmniSearchHandler` uses `f64::total_cmp` for Reciprocal Rank Fusion (RRF) score sorting, guaranteeing deterministic NaN-safe search result ordering.
|
||||
* **RPC Timeout Memory Hygiene**: `nvim-core` maintains request hygiene by removing pending request entries from static RPC maps upon timeout or channel drop, eliminating orphan memory leaks.
|
||||
* **Embedding Input Safeguard**: `generate_embedding_async` returns explicit errors for empty/0-length text inputs instead of returning empty vectors, preventing downstream vector dimension mismatches during cosine similarity calculations.
|
||||
* **Path Traversal Security Guards**: `validate_safe_path` enforces path canonicalization and rejects relative parent traversal components (`..`) across file and process log handlers (`process_logs` / `ProcessLogsTool`).
|
||||
* **Watcher Map Memory Eviction**: Proactive daemon file watcher in `watcher.rs` caps `last_processed` map size at 1,000 entries and purges entries older than 10 minutes to prevent monotonic memory leakage.
|
||||
* **Comprehensive Serde Casing Aliases**: All 11 consolidated tool action enums (TaskAction, MilestoneAction, SnippetAction, DecisionAction, TechDebtAction, EnvAction, ClipboardAction, HandoffMemoAction, HypothesisAction, AgentSignalAction, ProcessLogAction) include serde alias attributes supporting `snake_case`, `camelCase`, `PascalCase`, and uppercase variants for maximum LLM casing resilience.
|
||||
* **Two-Phase Graph Condensation**: `condense_graph_worker` uses a 2-phase commit (non-destructive `read_with` -> graph insert -> prune by timestamp/content) to prevent data loss if summarization or graph insertion fails.
|
||||
* **Store Write Lock Minimization**: `Store::modify` and `Store::modify_async` unblock concurrent readers during JSON serialization by releasing the write lock immediately after mutating memory state.
|
||||
* **Redb Database Lock Retry Backoff**: `init_db` retries transient Redb lock contention with exponential backoff (3 attempts, 150ms delay) before falling back.
|
||||
* **Offloaded Background Index Rebuilds**: `MemoryState::rebuild_index` offloads graph snapshot cloning and Tantivy document re-indexing into `tokio::task::spawn_blocking` to avoid stalling async event loops.
|
||||
* **Broadcast Watch-Based Shutdown Channels**: Background workers utilize `tokio::sync::watch` for broadcast shutdown notifications without consuming cancellation signals.
|
||||
* **Consolidated Neovim Tool Suite (v2)**: The Neovim server exposes 7 consolidated domain tools (`nvim_buffer`, `nvim_window`, `nvim_view`, `nvim_diagnostics`, `nvim_visual`, `nvim_execute_lua`, `nvim_system`) with comprehensive action dispatching.
|
||||
* **Fallback Vector Search Parity**: In-memory vector search fallback indexes Knowledge Graph entities, observations, and error fixes when external vector databases are unavailable.
|
||||
|
||||
|
||||
@@ -305,7 +305,7 @@ pub fn create_router(app_state: Arc<AppState>) -> Router {
|
||||
let state_clone = app_state.handler.state.clone();
|
||||
move || async move {
|
||||
let json = state_clone.code.tech_debts.read_with(|items| serde_json::to_string(items).unwrap_or_else(|_| "[]".to_string()));
|
||||
([(axum::http::header::CONTENT_TYPE, "application/json")], json)
|
||||
([(axum::http::header::CONTENT_TYPE, "application/json"), (axum::http::header::CACHE_CONTROL, "no-cache, no-store, must-revalidate")], json)
|
||||
}
|
||||
}),
|
||||
)
|
||||
@@ -315,7 +315,7 @@ pub fn create_router(app_state: Arc<AppState>) -> Router {
|
||||
let state_clone = app_state.handler.state.clone();
|
||||
move || async move {
|
||||
let json = state_clone.code.adrs.read_with(|items| serde_json::to_string(items).unwrap_or_else(|_| "[]".to_string()));
|
||||
([(axum::http::header::CONTENT_TYPE, "application/json")], json)
|
||||
([(axum::http::header::CONTENT_TYPE, "application/json"), (axum::http::header::CACHE_CONTROL, "no-cache, no-store, must-revalidate")], json)
|
||||
}
|
||||
}),
|
||||
)
|
||||
|
||||
+2
-15
@@ -105,8 +105,8 @@ pub async fn handle_socket(socket: WebSocket, state: Arc<AppState>, _client_type
|
||||
});
|
||||
|
||||
let handler = Arc::clone(&state.handler);
|
||||
let state_clone = Arc::clone(&state);
|
||||
let session_id_clone = session_id.clone();
|
||||
let response_tx = tx.clone();
|
||||
|
||||
let recv_task = tokio::spawn(async move {
|
||||
while let Some(msg_result) = receiver.next().await {
|
||||
@@ -121,26 +121,13 @@ pub async fn handle_socket(socket: WebSocket, state: Arc<AppState>, _client_type
|
||||
// Process MCP request
|
||||
if let Some(response) = handler.handle_request(payload).await {
|
||||
let res_str = serde_json::to_string(&response).unwrap_or_else(|e| format!(r#"{{\"jsonrpc\":\"2.0\",\"id\":null,\"error\":{{\"code\":-32603,\"message\":\"{}\"}}}}"#, e));
|
||||
let tx_opt = state_clone
|
||||
.clients
|
||||
.read()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.get(&session_id_clone)
|
||||
.cloned();
|
||||
if let Some(client_tx) = tx_opt {
|
||||
if let Err(e) = client_tx.send(res_str).await {
|
||||
if let Err(e) = response_tx.send(res_str).await {
|
||||
tracing::error!(
|
||||
"Failed to send response to client channel for session {}: {}",
|
||||
session_id_clone,
|
||||
e
|
||||
);
|
||||
}
|
||||
} else {
|
||||
tracing::warn!(
|
||||
"Could not find client_tx for session_id {} when trying to send response",
|
||||
session_id_clone
|
||||
);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
tracing::warn!(
|
||||
|
||||
@@ -72,7 +72,9 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
if instructions_src.exists() {
|
||||
let _ = fs::copy(&instructions_src, win_dir.join("instructions.md"));
|
||||
let _ = fs::copy(&instructions_src, wsl_dir.join("instructions.md"));
|
||||
println!(" [OK] Synchronized instructions.md to Win and WSL");
|
||||
let server_instructions = PathBuf::from(r"C:\Users\reazul.ashraf\workspace\rust\mcp-memory\server\src\instructions.md");
|
||||
let _ = fs::copy(&instructions_src, server_instructions);
|
||||
println!(" [OK] Synchronized instructions.md to Win, WSL, and server/src/instructions.md");
|
||||
}
|
||||
|
||||
let _ = fs::remove_dir_all(&temp_dir);
|
||||
|
||||
+11
-4
@@ -216,7 +216,7 @@ Type: ${entity.entity_type}`,
|
||||
}
|
||||
async function loadGraph() {
|
||||
try {
|
||||
const res = await fetch("/api/graph");
|
||||
const res = await fetch("/api/graph", { cache: "no-store" });
|
||||
const dataText = await res.text();
|
||||
if (dataText === lastGraphJson && network) {
|
||||
return;
|
||||
@@ -459,7 +459,7 @@ function applyTaskFilters() {
|
||||
}
|
||||
async function loadTasks() {
|
||||
try {
|
||||
const res = await fetch("/api/tasks");
|
||||
const res = await fetch("/api/tasks", { cache: "no-store" });
|
||||
cachedTasks = await res.json();
|
||||
applyTaskFilters();
|
||||
} catch (err) {
|
||||
@@ -689,6 +689,7 @@ function requestDomainRefresh(domain) {
|
||||
graph: "graph-tab",
|
||||
task: "task-tab",
|
||||
techdebt: "techdebt-tab",
|
||||
adrs: "adrs-tab",
|
||||
snippets: "snippets-tab",
|
||||
terminal: "terminal-tab",
|
||||
ledger: "ledger-tab",
|
||||
@@ -713,6 +714,9 @@ function requestDomainRefresh(domain) {
|
||||
case "techdebt":
|
||||
loadTechDebt();
|
||||
break;
|
||||
case "adrs":
|
||||
loadADRs();
|
||||
break;
|
||||
case "snippets":
|
||||
loadSnippets();
|
||||
break;
|
||||
@@ -739,7 +743,10 @@ function handleIncomingActivity(payload) {
|
||||
category = (payload.category || payload.type || "").toUpperCase();
|
||||
}
|
||||
}
|
||||
if (method === "notifications/resources/updated" || category === "GRAPH" || category === "DECISION") {
|
||||
if (method === "notifications/resources/updated" || category === "GRAPH") {
|
||||
requestDomainRefresh("graph");
|
||||
} else if (category === "DECISION" || category === "ADR") {
|
||||
requestDomainRefresh("adrs");
|
||||
requestDomainRefresh("graph");
|
||||
} else if (method === "notifications/task/completed" || category.startsWith("TASK")) {
|
||||
requestDomainRefresh("task");
|
||||
@@ -1124,7 +1131,7 @@ async function loadGenericList(endpoint, containerId, formatter, options) {
|
||||
state.formatter = formatter;
|
||||
listControllers.set(containerId, state);
|
||||
try {
|
||||
const res = await fetch(endpoint);
|
||||
const res = await fetch(endpoint, { cache: "no-store" });
|
||||
if (!res.ok) {
|
||||
container.innerHTML = `<div style="color:var(--error-color); padding:20px; text-align:center;">⚠️ Failed to load ${escapeHtml(endpoint)} (${res.status} ${escapeHtml(res.statusText)})</div>`;
|
||||
return;
|
||||
|
||||
+43
-24
@@ -113,7 +113,9 @@ function switchTab(tabId: string, btn?: HTMLElement | null): void {
|
||||
|
||||
const targetTab = document.getElementById(tabId);
|
||||
if (targetTab) targetTab.classList.add("active");
|
||||
const targetBtn = btn || document.querySelector<HTMLElement>(`.tab-button[onclick*="'${tabId}'"]`);
|
||||
const targetBtn =
|
||||
btn ||
|
||||
document.querySelector<HTMLElement>(`.tab-button[onclick*="'${tabId}'"]`);
|
||||
if (targetBtn) targetBtn.classList.add("active");
|
||||
|
||||
switch (tabId) {
|
||||
@@ -236,10 +238,16 @@ function showInspector(nodeId: string): void {
|
||||
const metaEl = document.querySelector<HTMLElement>(".inspector-meta");
|
||||
if (metaEl) {
|
||||
const createdStr = entity.created_at
|
||||
? new Date(entity.created_at * 1000).toISOString().replace("T", " ").substring(0, 19)
|
||||
? new Date(entity.created_at * 1000)
|
||||
.toISOString()
|
||||
.replace("T", " ")
|
||||
.substring(0, 19)
|
||||
: "";
|
||||
const updatedStr = entity.updated_at
|
||||
? new Date(entity.updated_at * 1000).toISOString().replace("T", " ").substring(0, 19)
|
||||
? new Date(entity.updated_at * 1000)
|
||||
.toISOString()
|
||||
.replace("T", " ")
|
||||
.substring(0, 19)
|
||||
: "";
|
||||
metaEl.innerHTML = `
|
||||
<div><strong>Type:</strong> <span id="inspector-type">${escapeHtml(entity.entity_type)}</span></div>
|
||||
@@ -365,7 +373,7 @@ function updateGraphData(): void {
|
||||
|
||||
async function loadGraph(): Promise<void> {
|
||||
try {
|
||||
const res = await fetch("/api/graph");
|
||||
const res = await fetch("/api/graph", { cache: "no-store" });
|
||||
const dataText = await res.text();
|
||||
if (dataText === lastGraphJson && network) {
|
||||
return;
|
||||
@@ -584,8 +592,10 @@ function buildTaskTreeHTML(
|
||||
|
||||
if (t.git_branch || t.repo_name) {
|
||||
html += `<div style="margin-top:8px; font-size:0.75em; display:flex; gap:6px; flex-wrap:wrap; color:var(--text-secondary);">`;
|
||||
if (t.repo_name) html += `<span style="background:var(--canvas-bg); padding:1px 6px; border-radius:3px; border:1px solid var(--border-color);">📦 ${escapeHtml(t.repo_name)}</span>`;
|
||||
if (t.git_branch) html += `<span style="background:var(--canvas-bg); padding:1px 6px; border-radius:3px; border:1px solid var(--border-color);">🌿 ${escapeHtml(t.git_branch)}</span>`;
|
||||
if (t.repo_name)
|
||||
html += `<span style="background:var(--canvas-bg); padding:1px 6px; border-radius:3px; border:1px solid var(--border-color);">📦 ${escapeHtml(t.repo_name)}</span>`;
|
||||
if (t.git_branch)
|
||||
html += `<span style="background:var(--canvas-bg); padding:1px 6px; border-radius:3px; border:1px solid var(--border-color);">🌿 ${escapeHtml(t.git_branch)}</span>`;
|
||||
html += `</div>`;
|
||||
}
|
||||
|
||||
@@ -625,9 +635,11 @@ function applyTaskFilters(): void {
|
||||
filtered = filtered.filter(
|
||||
(t) =>
|
||||
(t.title && t.title.toLowerCase().includes(taskFilterQuery)) ||
|
||||
(t.description && t.description.toLowerCase().includes(taskFilterQuery)) ||
|
||||
(t.description &&
|
||||
t.description.toLowerCase().includes(taskFilterQuery)) ||
|
||||
(t.id && t.id.toLowerCase().includes(taskFilterQuery)) ||
|
||||
(t.assigned_agent && t.assigned_agent.toLowerCase().includes(taskFilterQuery)),
|
||||
(t.assigned_agent &&
|
||||
t.assigned_agent.toLowerCase().includes(taskFilterQuery)),
|
||||
);
|
||||
}
|
||||
|
||||
@@ -664,7 +676,7 @@ function applyTaskFilters(): void {
|
||||
|
||||
async function loadTasks(): Promise<void> {
|
||||
try {
|
||||
const res = await fetch("/api/tasks");
|
||||
const res = await fetch("/api/tasks", { cache: "no-store" });
|
||||
cachedTasks = await res.json();
|
||||
applyTaskFilters();
|
||||
} catch (err) {
|
||||
@@ -922,6 +934,7 @@ function requestDomainRefresh(domain: string): void {
|
||||
graph: "graph-tab",
|
||||
task: "task-tab",
|
||||
techdebt: "techdebt-tab",
|
||||
adrs: "adrs-tab",
|
||||
snippets: "snippets-tab",
|
||||
terminal: "terminal-tab",
|
||||
ledger: "ledger-tab",
|
||||
@@ -949,6 +962,9 @@ function requestDomainRefresh(domain: string): void {
|
||||
case "techdebt":
|
||||
loadTechDebt();
|
||||
break;
|
||||
case "adrs":
|
||||
loadADRs();
|
||||
break;
|
||||
case "snippets":
|
||||
loadSnippets();
|
||||
break;
|
||||
@@ -981,11 +997,10 @@ function handleIncomingActivity(payload: any): void {
|
||||
}
|
||||
}
|
||||
|
||||
if (
|
||||
method === "notifications/resources/updated" ||
|
||||
category === "GRAPH" ||
|
||||
category === "DECISION"
|
||||
) {
|
||||
if (method === "notifications/resources/updated" || category === "GRAPH") {
|
||||
requestDomainRefresh("graph");
|
||||
} else if (category === "DECISION" || category === "ADR") {
|
||||
requestDomainRefresh("adrs");
|
||||
requestDomainRefresh("graph");
|
||||
} else if (
|
||||
method === "notifications/task/completed" ||
|
||||
@@ -1011,7 +1026,6 @@ function handleIncomingActivity(payload: any): void {
|
||||
) {
|
||||
requestDomainRefresh("memos");
|
||||
}
|
||||
|
||||
const feed = document.getElementById("activity-feed");
|
||||
if (feed) {
|
||||
if (feed.querySelector(".feed-entry") === null) {
|
||||
@@ -1506,7 +1520,7 @@ async function loadGenericList(
|
||||
listControllers.set(containerId, state);
|
||||
|
||||
try {
|
||||
const res = await fetch(endpoint);
|
||||
const res = await fetch(endpoint, { cache: "no-store" });
|
||||
if (!res.ok) {
|
||||
container.innerHTML = `<div style="color:var(--error-color); padding:20px; text-align:center;">⚠️ Failed to load ${escapeHtml(endpoint)} (${res.status} ${escapeHtml(res.statusText)})</div>`;
|
||||
return;
|
||||
@@ -1679,12 +1693,16 @@ function loadTerminal(): void {
|
||||
</div>
|
||||
${item.status_reason ? `<div style="margin-top:6px; font-size:0.85em; color:var(--error-color);"><strong>Reason:</strong> ${escapeHtml(item.status_reason)}</div>` : ""}
|
||||
${item.stdout_summary ? `<div style="margin-top:6px; font-size:0.85em; color:var(--text-secondary);"><strong>Output:</strong> ${escapeHtml(item.stdout_summary)}</div>` : ""}
|
||||
${item.error_output ? `
|
||||
${
|
||||
item.error_output
|
||||
? `
|
||||
<details style="margin-top:8px;" ${!isSuccess ? "open" : ""}>
|
||||
<summary style="font-size:0.8em; color:var(--error-color); cursor:pointer; font-weight:600;">Error Output</summary>
|
||||
<pre style="background:#1a0f0f; color:#ff6b6b; padding:8px; border-radius:4px; border:1px solid rgba(231,76,60,0.3); font-size:0.8em; overflow-x:auto; margin-top:4px; white-space:pre-wrap;">${escapeHtml(item.error_output)}</pre>
|
||||
</details>
|
||||
` : ""}
|
||||
`
|
||||
: ""
|
||||
}
|
||||
`;
|
||||
},
|
||||
{
|
||||
@@ -1832,12 +1850,16 @@ function loadTechDebt(): void {
|
||||
<span style="color:var(--success-color); font-weight:bold; background: rgba(46, 204, 113, 0.15); padding: 2px 6px; border-radius: 4px; border: 1px solid var(--success-color); flex-shrink:0;">✓ Resolved</span>
|
||||
</div>
|
||||
<div style="margin-top:8px; font-size:0.9em; line-height:1.4;"><strong>Solution:</strong> ${escapeHtml(item.solution || "")}</div>
|
||||
${item.stack_trace ? `
|
||||
${
|
||||
item.stack_trace
|
||||
? `
|
||||
<details style="margin-top:8px;">
|
||||
<summary style="font-size:0.8em; color:var(--text-secondary); cursor:pointer; font-weight:600;">Stack Trace</summary>
|
||||
<pre style="background:#111417; color:#ff7b72; padding:8px; border-radius:4px; border:1px solid var(--border-color); font-size:0.8em; overflow-x:auto; margin-top:4px; white-space:pre-wrap;">${escapeHtml(item.stack_trace)}</pre>
|
||||
</details>
|
||||
` : ""}
|
||||
`
|
||||
: ""
|
||||
}
|
||||
<div style="margin-top:8px; font-size:0.8em; display:flex; gap:8px; flex-wrap:wrap; align-items:center;">
|
||||
${item.repo_name ? `<span style="background:var(--canvas-bg); padding:2px 6px; border-radius:4px; border:1px solid var(--border-color);">📦 ${escapeHtml(item.repo_name)}</span>` : ""}
|
||||
<span style="background:var(--canvas-bg); padding:2px 6px; border-radius:4px; font-family:monospace; border:1px solid var(--border-color);">Commit: ${item.git_commit ? escapeHtml(item.git_commit.substring(0, 8)) : "None"}</span>
|
||||
@@ -1914,10 +1936,7 @@ function loadADRs(): void {
|
||||
<span style="background:var(--canvas-bg); padding:2px 8px; border-radius:4px; font-size:0.8em; border:1px solid var(--border-color); ${badgeStyle}">${badgeText}</span>
|
||||
</div>
|
||||
${
|
||||
item.git_commit ||
|
||||
item.git_branch ||
|
||||
item.task_id ||
|
||||
item.resolved_at
|
||||
item.git_commit || item.git_branch || item.task_id || item.resolved_at
|
||||
? `
|
||||
<div style="font-size:0.8em; color:var(--text-secondary); margin-top:4px; font-family:monospace; display:flex; gap:12px; flex-wrap:wrap;">
|
||||
${item.git_branch || item.git_commit ? `<span>📦 ${escapeHtml(item.git_branch || "repo")} @ ${escapeHtml((item.git_commit || "").substring(0, 8))}</span>` : ""}
|
||||
|
||||
+10
-61
@@ -287,8 +287,8 @@ impl McpTool for LogCodeChangeHandler {
|
||||
author: req.author,
|
||||
session_id: req.session_id,
|
||||
vcs_type: detected_vcs,
|
||||
revision: effective_rev.clone(),
|
||||
branch: effective_branch.clone(),
|
||||
revision: effective_rev,
|
||||
branch: effective_branch,
|
||||
repository_root: req.repository_root,
|
||||
});
|
||||
if ledger.len() > 500 {
|
||||
@@ -300,42 +300,9 @@ impl McpTool for LogCodeChangeHandler {
|
||||
&format!("Modified {}", req.file_path),
|
||||
Some(&description),
|
||||
);
|
||||
|
||||
let recon = crate::handlers::reconciliation::reconcile_commit_or_code_change(
|
||||
&state,
|
||||
&description,
|
||||
Some(&req.file_path),
|
||||
effective_rev.as_deref(),
|
||||
effective_branch.as_deref(),
|
||||
)
|
||||
.await;
|
||||
|
||||
let mut recon_notes = Vec::new();
|
||||
if !recon.implemented_adrs.is_empty() {
|
||||
recon_notes.push(format!("Implemented ADRs: {}", recon.implemented_adrs.join(", ")));
|
||||
}
|
||||
if !recon.resolved_tech_debts.is_empty() {
|
||||
recon_notes.push(format!("Resolved TechDebt: {}", recon.resolved_tech_debts.join(", ")));
|
||||
}
|
||||
if !recon.completed_tasks.is_empty() {
|
||||
recon_notes.push(format!("Completed Tasks: {}", recon.completed_tasks.join(", ")));
|
||||
}
|
||||
if !recon.unblocked_tasks.is_empty() {
|
||||
recon_notes.push(format!("Unblocked Tasks: {}", recon.unblocked_tasks.join(", ")));
|
||||
}
|
||||
if !recon.updated_milestones.is_empty() {
|
||||
recon_notes.push(format!("Updated Milestones: {}", recon.updated_milestones.join(", ")));
|
||||
}
|
||||
|
||||
let recon_suffix = if recon_notes.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
format!(" [{}]", recon_notes.join(" | "))
|
||||
};
|
||||
|
||||
Ok(format!(
|
||||
"Logged code change for {}: {}{}",
|
||||
req.file_path, description, recon_suffix
|
||||
"Logged code change for {}: {}",
|
||||
req.file_path, description
|
||||
))
|
||||
}
|
||||
}
|
||||
@@ -811,16 +778,7 @@ impl McpTool for OmniSearchHandler {
|
||||
.unwrap_or_default();
|
||||
|
||||
// Reciprocal Rank Fusion (RRF) algorithm
|
||||
#[allow(dead_code)]
|
||||
#[derive(Clone)]
|
||||
struct MatchItem {
|
||||
id: String,
|
||||
doc_type: String,
|
||||
title: String,
|
||||
body: String,
|
||||
}
|
||||
|
||||
let mut rrf_scores: std::collections::HashMap<String, (f64, MatchItem)> =
|
||||
let mut rrf_scores: std::collections::HashMap<String, (f64, crate::search::SearchResult)> =
|
||||
std::collections::HashMap::new();
|
||||
|
||||
for (rank, (id, doc_type, title, body, _score)) in keyword_matches.into_iter().enumerate() {
|
||||
@@ -829,11 +787,12 @@ impl McpTool for OmniSearchHandler {
|
||||
id.clone(),
|
||||
(
|
||||
score,
|
||||
MatchItem {
|
||||
crate::search::SearchResult {
|
||||
id,
|
||||
doc_type,
|
||||
title,
|
||||
body,
|
||||
score: 0.0,
|
||||
},
|
||||
),
|
||||
);
|
||||
@@ -845,20 +804,14 @@ impl McpTool for OmniSearchHandler {
|
||||
if let Some(existing) = rrf_scores.get_mut(&item_id) {
|
||||
existing.0 += score;
|
||||
} else {
|
||||
let item = MatchItem {
|
||||
id: v_match.id.clone(),
|
||||
doc_type: v_match.doc_type,
|
||||
title: v_match.title,
|
||||
body: v_match.body,
|
||||
};
|
||||
rrf_scores.insert(item_id, (score, item));
|
||||
rrf_scores.insert(item_id, (score, v_match));
|
||||
}
|
||||
}
|
||||
|
||||
let mut ranked_items: Vec<_> = rrf_scores.into_values().collect();
|
||||
ranked_items.sort_by(|a, b| b.0.total_cmp(&a.0));
|
||||
|
||||
let matches: Vec<MatchItem> = ranked_items.into_iter().map(|(_, item)| item).collect();
|
||||
let matches: Vec<crate::search::SearchResult> = ranked_items.into_iter().map(|(_, item)| item).collect();
|
||||
|
||||
let kg_json = state.read_graph(|full| {
|
||||
let mut kg_results = serde_json::Map::new();
|
||||
@@ -1203,11 +1156,7 @@ impl McpTool for GetProjectHealthHandler {
|
||||
let active_milestones = state.project.milestones.read_with(|milestones| {
|
||||
milestones
|
||||
.iter()
|
||||
.filter(|m| {
|
||||
m.namespace == req.namespace
|
||||
&& !m.status.eq_ignore_ascii_case("done")
|
||||
&& !m.status.eq_ignore_ascii_case("completed")
|
||||
})
|
||||
.filter(|m| m.namespace == req.namespace && m.status != "done")
|
||||
.count()
|
||||
});
|
||||
let report = serde_json::json!({
|
||||
|
||||
@@ -9,7 +9,10 @@ use std::borrow::Cow;
|
||||
use std::sync::Arc;
|
||||
|
||||
|
||||
static CLIPBOARD_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
|
||||
|
||||
pub fn get_native_clipboard_text() -> Option<String> {
|
||||
let _guard = CLIPBOARD_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
for _ in 0..3 {
|
||||
if let Ok(mut clipboard) = arboard::Clipboard::new()
|
||||
&& let Ok(text) = clipboard.get_text()
|
||||
@@ -65,6 +68,7 @@ fn get_windows_png_clipboard_image() -> Option<image::DynamicImage> {
|
||||
None
|
||||
}
|
||||
pub fn get_native_clipboard_image() -> Option<image::DynamicImage> {
|
||||
let _guard = CLIPBOARD_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
#[cfg(target_os = "windows")]
|
||||
if let Some(img) = get_windows_png_clipboard_image() {
|
||||
return Some(img);
|
||||
@@ -104,9 +108,10 @@ impl McpTool for ClipboardHandler {
|
||||
let req: ClipboardTool = serde_json::from_value(args).map_err(|e| e.to_string())?;
|
||||
match req.action {
|
||||
ClipboardAction::Read => {
|
||||
let result =
|
||||
tokio::task::spawn_blocking(move || -> crate::error::Result<serde_json::Value> {
|
||||
let (mut out, b64_opt) =
|
||||
tokio::task::spawn_blocking(move || -> crate::error::Result<(serde_json::Map<String, Value>, Option<String>)> {
|
||||
let mut out = serde_json::Map::new();
|
||||
let mut b64_opt = None;
|
||||
|
||||
if let Some(text) = get_native_clipboard_text() {
|
||||
out.insert("text".into(), json!(text));
|
||||
@@ -125,7 +130,7 @@ impl McpTool for ClipboardHandler {
|
||||
let bytes = jpeg_bytes.into_inner();
|
||||
use base64::Engine;
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(&bytes);
|
||||
out.insert("image_base64".into(), json!(b64));
|
||||
b64_opt = Some(b64);
|
||||
|
||||
let cache_dir = dirs::home_dir()
|
||||
.unwrap_or_default()
|
||||
@@ -146,17 +151,12 @@ impl McpTool for ClipboardHandler {
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(Value::Object(out))
|
||||
Ok((out, b64_opt))
|
||||
})
|
||||
.await
|
||||
.map_err(|e| crate::error::AppError::Internal(format!("Task panic: {}", e)))??;
|
||||
|
||||
let mut final_obj = result;
|
||||
if let Some(b64) = final_obj.get("image_base64").and_then(|v| v.as_str()) {
|
||||
let b64_str = b64.to_string();
|
||||
if let Some(obj) = final_obj.as_object_mut() {
|
||||
obj.remove("image_base64");
|
||||
}
|
||||
if let Some(b64_str) = b64_opt {
|
||||
if state.ollama.is_available().await
|
||||
&& let Ok(analysis) = state
|
||||
.ollama
|
||||
@@ -165,19 +165,19 @@ impl McpTool for ClipboardHandler {
|
||||
&b64_str,
|
||||
)
|
||||
.await
|
||||
&& let Some(obj) = final_obj.as_object_mut()
|
||||
{
|
||||
obj.insert("image_analysis".to_string(), json!(analysis.trim()));
|
||||
out.insert("image_analysis".to_string(), json!(analysis.trim()));
|
||||
}
|
||||
}
|
||||
|
||||
state.record_activity("clipboard", "Read contents from OS clipboard", None);
|
||||
Ok::<String, crate::error::AppError>(serde_json::to_string_pretty(&final_obj)?)
|
||||
Ok::<String, crate::error::AppError>(serde_json::to_string_pretty(&Value::Object(out))?)
|
||||
}
|
||||
ClipboardAction::Write => {
|
||||
let text_opt = req.text;
|
||||
let image_path_opt = req.image_path;
|
||||
let res = tokio::task::spawn_blocking(move || {
|
||||
let _guard = CLIPBOARD_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let mut msgs = Vec::new();
|
||||
|
||||
if let Some(text) = &text_opt {
|
||||
|
||||
+147
-50
@@ -1,7 +1,7 @@
|
||||
# Memory MCP Strategic Guidelines
|
||||
|
||||
This document outlines the STRATEGY, SEMANTICS, and CASING STANDARDS for using the MCP Memory Server.
|
||||
You do not need to memorize JSON schemas for these tools; they are strictly defined and typed in the `tools/list` endpoint. Focus purely on WHEN and WHY to use them.
|
||||
This document outlines the STRATEGY, SEMANTICS, RESOURCE SCHEMAS, and CASING STANDARDS for using the MCP Memory Server.
|
||||
You do not need to memorize individual JSON schemas for every tool; they are strictly defined and typed in the `tools/list` endpoint. Focus on WHEN, WHY, and HOW to leverage them effectively.
|
||||
|
||||
---
|
||||
|
||||
@@ -19,7 +19,45 @@ To prevent graph fragmentation and ensure seamless LLM context retrieval:
|
||||
|
||||
---
|
||||
|
||||
## 2. Consolidated Smart Tools Architecture
|
||||
## 2. The Two-Tier Context Paradigm
|
||||
|
||||
> [!IMPORTANT]
|
||||
> * **Tier 1 (Static Markdown)**: Repository rules, constraints, architectural patterns, and developer preferences are maintained directly in static git-tracked files (`agent-rules/*.md`, `instructions.md`) and system prompts. This guarantees 0ms turn-0 availability without relying on proactive agent tool retrieval.
|
||||
> * **Tier 2 (Telemetry & Ephemeral DB)**: High-frequency operational history—the Code Change Ledger (`audit_ledger`), terminal command history, compiler error fixes (`log_error_fix`), active tasks, and preflight context—is handled by the MCP Memory Server and surfaced via the Brain Monitor Web UI and MCP resources.
|
||||
|
||||
---
|
||||
|
||||
## 3. Passive MCP Resources (`resources/list`)
|
||||
|
||||
The server exposes 9 real-time, read-only MCP resources. Agents should read these resources directly to regain context without incurring tool call latency:
|
||||
|
||||
| Resource URI | Resource Name | Description & Usage |
|
||||
|:---|:---|:---|
|
||||
| `memory://graph/entities` | Graph Entities | All nodes and entities in the knowledge graph (top 100 with pagination guidance). |
|
||||
| `memory://graph/relations` | Graph Relations | All relationships between graph entities (top 200 with subgraph guidance). |
|
||||
| `memory://tasks/active` | Active Tasks | Current active tasks with status, priority, and assigned subagents. |
|
||||
| `memory://decisions/active` | Active ADR Decisions | Architectural decisions currently in `accepted` status. |
|
||||
| `memory://tech_debt/unresolved` | Unresolved Tech Debt | All open engineering debt items requiring future refactoring. |
|
||||
| `memory://session/delta` | Session Delta | Code modifications, commits, active tasks, and notes created in the last 2 hours. |
|
||||
| `memory://terminal/recent` | Terminal History | Recent terminal commands, interpreters (`pwsh`, `bash`, `nu`), working dirs, and exit codes. |
|
||||
| `memory://activity/recent` | Recent Activity | Real-time IDE and developer activity event stream. |
|
||||
| `memory://milestones` | Milestones | Project milestones, deliverables, target dates, and progress. |
|
||||
|
||||
---
|
||||
|
||||
## 4. MCP Workflow Prompts (`prompts/list`)
|
||||
|
||||
The server registers 5 high-signal workflow prompts to initiate standardized agent routines:
|
||||
|
||||
1. **`context_warmup`**: Executed at session startup. Prompts the agent to read `memory://tasks/active` and `memory://session/delta`, inspect the workspace worktree, and assemble immediate working context.
|
||||
2. **`analyze_tech_debt`**: Prompts the agent to review unresolved technical debt from `memory://tech_debt/unresolved` and generate a prioritized remediation plan.
|
||||
3. **`summarize_architecture`**: Synthesizes active ADRs from `memory://decisions/active` and graph entities from `memory://graph/entities` into an architectural overview.
|
||||
4. **`handoff_routine`**: Triggers the `DevOpsSRE` subagent at session end to generate a standup report, audit active tasks, and record a handoff memo for future sessions.
|
||||
5. **`archive_routine`**: Compresses historical session summaries into a dense milestone retrospective entity and purges pruned entries.
|
||||
|
||||
---
|
||||
|
||||
## 5. Consolidated Smart Tools Architecture (11 Primary Tools)
|
||||
|
||||
The server consolidates granular single-purpose tools into domain-named smart tools. Always prefer the consolidated tools over legacy aliases:
|
||||
|
||||
@@ -48,7 +86,8 @@ The server consolidates granular single-purpose tools into domain-named smart to
|
||||
- `action: "tag"`: Attach classification tags (requires `id`, `tags: Vec<String>`).
|
||||
|
||||
* **`decisions`**: Architectural Decision Records (ADRs).
|
||||
- `action: "log"`: Log ADR (requires `title`, optional `status: "accepted" | "proposed" | "deprecated" | "superseded"`, `context`, `decision`, `consequence`, `author`, `affected_components: Vec<String>`, `alternatives_considered: Vec<String>`, `supersedes`, `repo_name`).
|
||||
- `action: "log"`: Log ADR (requires `title`, optional `status: "accepted" | "proposed" | "deprecated" | "superseded" | "implemented"`, `context`, `decision`, `consequence`, `author`, `affected_components: Vec<String>`, `alternatives_considered: Vec<String>`, `supersedes`, `repo_name`).
|
||||
- `action: "update"`: Update ADR status and metadata (requires `id`, optional `status`, `git_commit`, `git_branch`).
|
||||
- `action: "query"`: Query ADRs (optional `query`).
|
||||
- `action: "delete"`: Delete ADR (requires `id`).
|
||||
|
||||
@@ -83,89 +122,155 @@ The server consolidates granular single-purpose tools into domain-named smart to
|
||||
|
||||
---
|
||||
|
||||
## 3. VCS & SVN Agnosticism & Multi-Repo Provenance
|
||||
## 6. Complete Tool Catalog (All 53 Tools)
|
||||
|
||||
The server exposes 53 specialized and smart MCP tools organized into 7 functional domains:
|
||||
|
||||
### 1. Knowledge Graph Core (18 Tools)
|
||||
1. `create_entities`: Batch-create entities with `name`, `entity_type`, and `observations`.
|
||||
2. `create_relations`: Batch-create relationships (`from`, `to`, `relation_type`).
|
||||
3. `add_observations`: Append observations to existing entities.
|
||||
4. `delete_entities`: Delete entity nodes and cascading relations.
|
||||
5. `delete_relations`: Delete specific relation edges between entities.
|
||||
6. `delete_observations`: Remove specific observations from an entity.
|
||||
7. `read_graph`: Return full or namespace-filtered knowledge graph.
|
||||
8. `search_nodes`: Search entity names and observations using Tantivy BM25.
|
||||
9. `open_nodes`: Inspect full details of specified entity nodes by name.
|
||||
10. `visualize_graph`: Generate Mermaid markdown or SVG diagram of the graph.
|
||||
11. `condense_entity`: Summarize entity observations into dense summaries.
|
||||
12. `merge_entities`: Merge source entity into target entity, re-pointing relations and pruning self-loops.
|
||||
13. `find_orphans`: Detect entities with zero relationships for pruning.
|
||||
14. `get_subgraph`: BFS graph traversal expanding $N$ hops from a root node.
|
||||
15. `sweep_graph_health`: Audit graph for orphans, calculate name similarity, and recommend merges.
|
||||
16. `resolve_stale_symbols`: Cross-reference graph symbols against the workspace AST to remove deleted code nodes.
|
||||
17. `summarize_subgraph`: LLM-ready concise synthesis of a localized subgraph.
|
||||
18. `query_graph_path`: BFS shortest path between two entities in the knowledge graph.
|
||||
|
||||
### 2. Task & Milestone Operations (2 Tools)
|
||||
19. `tasks`: Consolidated task board manager (`add`, `update`, `delete`, `list`, `set_criteria`, `verify`).
|
||||
20. `milestones`: Milestone lifecycle management (`add`, `update`, `list`).
|
||||
|
||||
### 3. Notes, Handoffs & Reporting (4 Tools)
|
||||
21. `handoff_memos`: Cross-session scratchpad and handoff memos (`leave`, `read`, `clear`).
|
||||
22. `add_session_summary`: Record session summary notes and highlights.
|
||||
23. `generate_standup_report`: Synthesize tasks, ledger changes, and session summaries into a standup report.
|
||||
24. `promote_to_entity`: Promote an ephemeral note or memo into a permanent knowledge graph entity.
|
||||
|
||||
### 4. Meta, Audit & Intelligence (15 Tools)
|
||||
25. `decisions`: Consolidated Architectural Decision Records (ADRs) manager (`log`, `update`, `query`, `delete`).
|
||||
26. `tech_debt`: Consolidated technical debt backlog manager (`log`, `resolve`, `list`).
|
||||
27. `log_error_fix`: Record an error resolution with stack trace, root cause, and git commit.
|
||||
28. `search_error_fixes`: Embedding-based and keyword search over past error resolutions.
|
||||
29. `log_code_change`: Record a file modification in the VCS-agnostic audit ledger.
|
||||
30. `query_recent_changes`: Retrieve recent code changes with lookback time filters.
|
||||
31. `omni_search`: Reciprocal Rank Fusion (RRF) search across all graph entities, snippets, ADRs, debt, and fixes.
|
||||
32. `get_project_health`: Health dashboard summarizing task completion, debt backlog, and graph consistency.
|
||||
33. `manage_checkpoint`: Create or restore named memory snapshots for safe rollback.
|
||||
34. `query_lineage`: Causal lineage tracker linking tasks, ADRs, commits, and error fixes.
|
||||
35. `get_next_actionable_tasks`: Topologically resolved list of unblocked tasks ready for execution.
|
||||
36. `hypotheses`: Structured diagnostic hypothesis tracker (`log`, `query`).
|
||||
37. `get_preflight_context`: Aggregated operational context at session start (tasks, debt, recent changes).
|
||||
38. `agent_signals`: Inter-agent signal bus (`broadcast`, `query`).
|
||||
39. `auto_session_checkpoint`: Automatic session boundary checkpointing.
|
||||
|
||||
### 5. System, Environment & Telemetry (4 Tools)
|
||||
40. `environment`: Tool fingerprinting, requirements, and environment registry (`update_fingerprint`, `read_fingerprint`, `log_requirement`, `register`, `get_details`).
|
||||
41. `snippets`: Reusable code snippet vault with hybrid search (`store`, `search`, `delete`, `tag`).
|
||||
42. `clipboard`: Pure native Rust OS clipboard interface (`read`, `write`).
|
||||
43. `process_logs`: Live process and daemon log watcher and tailer (`watch`, `get`, `clear`).
|
||||
|
||||
### 6. Git & Worktree Context (2 Tools)
|
||||
44. `get_active_worktree_context`: Inspect git status, modified files, diff summary, and current branch.
|
||||
45. `query_git_diffs`: Retrieve detailed git diffs for specific files or commit ranges.
|
||||
|
||||
### 7. AST & Code Intelligence (8 Tools)
|
||||
46. `read_file_skeleton`: Tree-sitter AST structural outline of functions, structs, and methods without implementation bodies.
|
||||
47. `replace_ast_node`: Precise AST node replacement preserving indentation and comments.
|
||||
48. `find_symbol_references`: Search for symbol references across snippets and disk source files.
|
||||
49. `get_callers`: Find call sites and callers of a specified function or method across the codebase.
|
||||
50. `analyze_impact`: Blast-radius impact analysis of modifying a symbol or file.
|
||||
51. `read_directory_architecture`: Recursive directory structure analysis capped at depth 10.
|
||||
52. `semantic_code_search`: Dense vector semantic code search over indexed source code.
|
||||
53. `manage_subagent_namespace`: Create, isolate, or merge subagent-scoped memory namespaces.
|
||||
|
||||
---
|
||||
|
||||
## 7. VCS & SVN Agnosticism & Multi-Repo Provenance
|
||||
|
||||
To support diverse enterprise repositories (Git, Subversion / SVN, Mercurial / Hg, Monorepos):
|
||||
* **`vcs_type`**: Designates the VCS engine (`"git"`, `"svn"`, `"hg"`, `"perforce"`, or `"none"`).
|
||||
* **`vcs_revision`**: Agnostic commit hash or SVN revision identifier (e.g., `"r12458"`, `"3e4f7a9"`).
|
||||
* **`upstream_url`**: Canonical remote repository URL (e.g. `https://svn.corp/repo/trunk`, `git@bitbucket.org:org/repo.git`).
|
||||
* **`repo_name`**: Logical project or repository identifier allowing multiple repositories to share or partition memory namespaces cleanly without collision.
|
||||
* **`repo_name`**: Logical project identifier allowing multiple repositories to share or partition memory namespaces cleanly without collision.
|
||||
* **Audit Ledger (`log_code_change`)**: Enriched with `vcs_type`, `vcs_revision`, `upstream_url`, `author`, `diff_summary`, and extensible `metadata: HashMap<String, String>`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Terminal & Process Telemetry
|
||||
## 8. Terminal & Process Telemetry
|
||||
|
||||
The server ingests and tracks active terminal commands and sessions:
|
||||
* **Active Terminals**: Tracks PIDs, shell interpreters (`pwsh`, `bash`, `nu`, `zsh`), current working directories (`cwd`), command exit codes, and timestamps.
|
||||
* **Terminal History Endpoint**: `/terminal/history` exposes recent shell commands and output streams to dashboard and LLMs to prevent lost shell context.
|
||||
* **Quality Gate Enforcement**: `GateRecord` captures pre-flight and pre-push validation passes with `gate_type`, `enforcer`, `status`, `validation_log`, and `repo_name`.
|
||||
* **Terminal History Endpoint & Resource**: `/terminal/history` and `memory://terminal/recent` expose recent shell commands and output streams to dashboard and LLMs to prevent lost shell context.
|
||||
* **Zero-Latency UDP Streams**: Terminal and IDE telemetry stream over UDP (`MCP_UDP_PORT1`, `MCP_UDP_PORT2`) with zero disk I/O bottlenecks.
|
||||
|
||||
---
|
||||
|
||||
## 4. High-Signal Tool Responses & Performance Guidelines
|
||||
|
||||
To optimize context usage, response times, and LLM reasoning efficiency:
|
||||
|
||||
* **High-Signal Feedback**:
|
||||
- MCP tool calls (such as `create_entities`, `create_relations`, `add_observations`, `pin_file`) return explicit, structured summaries containing created names, types, edge counts, and relation paths.
|
||||
- LLMs do NOT need to execute follow-up `open_nodes` calls merely to confirm successful creation.
|
||||
|
||||
* **Batch Operations**:
|
||||
- When creating or updating multiple entities, snippets, or observations, always batch items into a single tool call array (e.g. `create_entities` with multiple array items) rather than making separate calls.
|
||||
- The server handles batch store mutations serially in a single transaction pass with single-permit event-driven flushes.
|
||||
|
||||
* **Real-time Tantivy Search Indexing**:
|
||||
- The Tantivy search engine automatically checks pending commits and reloads search readers prior to executing `omni_search` or `search_nodes`. Search queries always return up-to-date document results immediately following mutations.
|
||||
- Single-item deletions use targeted document removal rather than global index wipes.
|
||||
|
||||
* **Real-Time AST & Workspace Source Code Symbol Scanning**:
|
||||
- `find_symbol_references`, `get_callers`, and `analyze_impact` scan both stored code snippets and physical workspace source code files on disk (`.rs`, `.ts`, `.py`, `.go`, `.java`, `.c`, `.cpp`), providing accurate AST symbol references and call site tracking.
|
||||
|
||||
* **Graph Entity Merge & Self-Loop Protection**:
|
||||
- `merge_entities` re-points all relations from `source_entity` to `target_entity` and automatically prunes cyclic self-loops (`target -> target`).
|
||||
|
||||
* **Safe UTF-8 Token Truncation**:
|
||||
- Large responses (e.g. `get_active_worktree_context`, `read_graph`, `summarize_subgraph`) are safely truncated along UTF-8 character boundaries (`floor_char_boundary`), ensuring response bounds without runtime panics.
|
||||
|
||||
## 4. Automated Error Fix Auto-Matcher
|
||||
## 9. Automated Error Fix Auto-Matcher
|
||||
- **Tools:** `log_error_fix`, `search_error_fixes` (and alias `suggest_error_fix`)
|
||||
- **When to use:** When encountering a build error, test failure, or stack trace. Call `search_error_fixes` with either a text `query` or `stack_trace` before attempting a fix from scratch.
|
||||
- **Behavior:** Computes cosine similarity between error trace embeddings and past resolution logs when `stack_trace` is provided, or keyword filtering when `query` is provided, returning top matched solutions, modified files, and git commits.
|
||||
|
||||
---
|
||||
|
||||
## 5. Memory State Checkpointing & Rollbacks
|
||||
- **Tool:** `checkpoint_state`, `restore_state` (or `create_snapshot`, `restore_snapshot`)
|
||||
## 10. Memory State Checkpointing & Rollbacks
|
||||
- **Tool:** `manage_checkpoint` (action: `"create"` | `"restore"`)
|
||||
- **When to use:** Before initiating a large refactor, running experimental subagent tasks, or executing destructive batch operations.
|
||||
- **Behavior:** Saves or restores a point-in-time snapshot of graph entities, active tasks, and tech debt backlogs.
|
||||
|
||||
---
|
||||
|
||||
## 6. Self-Healing Graph Health Sweeper
|
||||
## 11. Self-Healing Graph Health Sweeper
|
||||
- **Tool:** `sweep_graph_health`
|
||||
- **When to use:** Periodically or before committing major graph changes to audit entity consistency.
|
||||
- **Behavior:** Detects orphaned nodes (0 relations), computes name similarity using pre-computed lowercase keys to identify near-duplicates (e.g., `APIGateway` vs `ApiGateway`), and provides structured `merge_entities` recommendations or auto-prunes orphans.
|
||||
|
||||
---
|
||||
|
||||
## 7. Causal Lineage & Provenance Tracker
|
||||
## 12. Causal Lineage & Provenance Tracker
|
||||
- **Tool:** `query_lineage`
|
||||
- **When to use:** When asking *"Why was this component modified?"* or *"What task or ADR led to this code change?"*
|
||||
- **Behavior:** Searches across tasks, ADRs, audit ledger entries, and error fixes to assemble a unified chronological timeline explaining the provenance behind any file, symbol, or commit.
|
||||
|
||||
---
|
||||
|
||||
## 9. Native Rust Invariants & Subprocess Prohibition (CRITICAL)
|
||||
## 13. ADR Lifecycle & Automated Git Post-Commit Reconciliation
|
||||
|
||||
- **The Golden ADR Rule**: When code implementing an ADR is committed, you MUST IMMEDIATELY update the ADR status to `implemented`:
|
||||
```json
|
||||
{
|
||||
"action": "update",
|
||||
"id": "ADR-XXXX",
|
||||
"status": "implemented",
|
||||
"git_commit": "<commit_hash>",
|
||||
"git_branch": "<branch>"
|
||||
}
|
||||
```
|
||||
- **Automated Post-Commit Hook**: The repository provides an automated reconciliation script (`scripts/git-reconcile.py`) installed via `just install-git-hooks`. Upon every `git commit`, the hook scans the commit message for `ADR-XXXX` or task identifiers and reconciles their status in the persistent store.
|
||||
- **Current Architecture Status**: 100% of defined ADRs (ADR-0080 through ADR-0104) are fully implemented and reconciled in the store.
|
||||
|
||||
---
|
||||
|
||||
## 14. Native Rust Invariants & Subprocess Prohibition (CRITICAL)
|
||||
|
||||
To maintain maximum security, speed, and cross-platform reliability:
|
||||
* **Zero Subprocess Fallbacks**: System and server tools (`clipboard`, `ast`, `search`, `db`) MUST strictly use pure native Rust crates (`arboard`, `tree-sitter`, `tantivy`, `psycopg`). Invocations of external shell commands (`powershell.exe`, `wl-paste`, `xclip`, `cmd.exe`) are strictly prohibited in native handlers.
|
||||
* **Transient Lock Recovery**: Transient OS handle collisions (such as Win32 OLE `OpenClipboard` lock contention) must be handled using native retry loops with backoffs directly in Rust.
|
||||
* **Automated Static Regression Gates**: Automated AST/source audit tests (e.g. `test_no_subprocess_clipboard_regression`) verify at test time that forbidden subprocess patterns are absent from handler implementations.
|
||||
* **Automated Static Regression Gates**: Automated AST/source audit tests (`test_no_subprocess_clipboard_regression`) verify at test time that forbidden subprocess patterns are absent from handler implementations.
|
||||
|
||||
---
|
||||
|
||||
## 10. High-Performance Concurrency & Resilience Guarantees
|
||||
## 15. High-Performance Concurrency & Resilience Guarantees
|
||||
|
||||
* **Explicit Fail-Fast Persistence Safety**: Replaced silent fallback to temporary databases (`/tmp/mcp_store_fallback_*`) with an explicit open retry and fail-fast panic unless `MCP_ALLOW_TMP_FALLBACK=1` is explicitly set, preventing silent data loss.
|
||||
* **Async Mutex Deadlock Elimination**: Converted shared state and Neovim connection locks (`shutdown_tx`, `NVIM_CONN`, `ACTIVE_SOCKET`, `HEADLESS_PROC`) to `tokio::sync::Mutex` to prevent worker thread pool starvation across `.await` points.
|
||||
* **Telemetry Session Deduplication & Channel Pruning**: Added `LAST_SESSION` in-memory state deduplication for UDP telemetry writes (eliminating disk I/O thrashing) and distinguished WebSocket `TrySendError::Full` backpressure vs `TrySendError::Closed` client pruning.
|
||||
@@ -186,7 +291,6 @@ To maintain maximum security, speed, and cross-platform reliability:
|
||||
* **AST Recursion Depth Safeguard & Zero-Copy Borrowing**: Tree-sitter AST traversal caps recursion depth at 100 to prevent thread stack overflows and borrows string slices (`&str`) during AST node walking.
|
||||
* **Strongly-Typed SearchResult & Pre-Allocated Search Vectors**: `search.rs` uses a strongly-typed `SearchResult` struct with named fields and pre-allocates result vector capacity (`Vec::with_capacity(top_docs.len())`).
|
||||
* **BFS Graph Traversal Pre-allocation & Visited Node Upper Bound**: `GraphQueryBuilder::find_shortest_path` pre-allocates adjacency map capacity (`HashMap::with_capacity(relations.len() * 2)`) and enforces a visited node upper bound (10,000 max) to guarantee deterministic BFS runtime.
|
||||
* **LLM Tool Schema Ergonomics & Context Guidance**: `ReadGraphHandler` schema explicitly instructs LLMs on `namespace` filtering and `search_nodes` / `get_subgraph` tools for large graph discovery.
|
||||
* **Filesystem Event Debouncing & Proactive State Refresh**: `spawn_watcher` implements a sliding 250ms debouncing window per file path, ignores `.git`, `target`, `.gemini`, and `node_modules`, and broadcasts activity events to `MemoryState`.
|
||||
* **Buffered Line-by-Line AST Workspace Symbol Scanning**: `scan_workspace_for_symbol` reads workspace files via `BufReader` line streams instead of loading entire files into heap strings, preventing memory spikes when traversing source trees.
|
||||
* **AST Node Type Aliasing & Skeleton Preallocation**: `replace_ast_node` documents friendly node aliases (`function`, `fn`, `method`, `struct`, `class`, `enum`, `trait`, `type`), and `read_file_skeleton` preallocates string buffer capacity (`code.len() / 2`).
|
||||
@@ -195,20 +299,13 @@ To maintain maximum security, speed, and cross-platform reliability:
|
||||
* **SIMD-Friendly Single-Pass Cosine Similarity**: `cosine_similarity` calculates dot product and Euclidean norm squares in a single linear pass over float vectors, enabling SIMD compiler auto-vectorization.
|
||||
* **Safe Stream Decoding on Log Tails**: Log tail operations (`process_logs`, action: "get") read raw bytes and decode using lossy UTF-8 conversion (`String::from_utf8_lossy`) to ensure resilience when seeking across multi-byte UTF-8 boundaries.
|
||||
* **Task Summary UTF-8 Truncation Safety**: `tasks` tool (`action = "list"`) truncates serialized task text strictly along UTF-8 character boundaries using `floor_char_boundary` when enforcing `max_tokens`.
|
||||
* **Sequential Snapshot Lock Scope Flattening**: `GenerateStandupReportHandler` reads `tasks`, `ledger`, and `session_summaries` sequentially rather than nesting read locks, preventing multi-lock deadlocks during concurrent store modifications.
|
||||
* **Directory Tree Depth Safeguard**: `ReadDirectoryArchitectureHandler` caps directory recursion at depth 10 to prevent stack overflow on deep or cyclic directory structures.
|
||||
* **Deterministic Total-Order Score Ranking**: `OmniSearchHandler` uses `f64::total_cmp` for Reciprocal Rank Fusion (RRF) score sorting, guaranteeing deterministic NaN-safe search result ordering.
|
||||
* **RPC Timeout Memory Hygiene**: `nvim-core` maintains request hygiene by removing pending request entries from static RPC maps upon timeout or channel drop, eliminating orphan memory leaks.
|
||||
* **Embedding Input Safeguard**: `generate_embedding_async` returns explicit errors for empty/0-length text inputs instead of returning empty vectors, preventing downstream vector dimension mismatches during cosine similarity calculations.
|
||||
* **Path Traversal Security Guards**: `validate_safe_path` enforces path canonicalization and rejects relative parent traversal components (`..`) across file and process log handlers (`process_logs` / `ProcessLogsTool`).
|
||||
* **Watcher Map Memory Eviction**: Proactive daemon file watcher in `watcher.rs` caps `last_processed` map size at 1,000 entries and purges entries older than 10 minutes to prevent monotonic memory leakage.
|
||||
* **Comprehensive Serde Casing Aliases**: All 11 consolidated tool action enums (TaskAction, MilestoneAction, SnippetAction, DecisionAction, TechDebtAction, EnvAction, ClipboardAction, HandoffMemoAction, HypothesisAction, AgentSignalAction, ProcessLogAction) include serde alias attributes supporting `snake_case`, `camelCase`, `PascalCase`, and uppercase variants for maximum LLM casing resilience.
|
||||
* **Two-Phase Graph Condensation**: `condense_graph_worker` uses a 2-phase commit (non-destructive `read_with` -> graph insert -> prune by timestamp/content) to prevent data loss if summarization or graph insertion fails.
|
||||
* **Store Write Lock Minimization**: `Store::modify` and `Store::modify_async` unblock concurrent readers during JSON serialization by releasing the write lock immediately after mutating memory state.
|
||||
* **Redb Database Lock Retry Backoff**: `init_db` retries transient Redb lock contention with exponential backoff (3 attempts, 150ms delay) before falling back.
|
||||
* **Offloaded Background Index Rebuilds**: `MemoryState::rebuild_index` offloads graph snapshot cloning and Tantivy document re-indexing into `tokio::task::spawn_blocking` to avoid stalling async event loops.
|
||||
* **Broadcast Watch-Based Shutdown Channels**: Background workers utilize `tokio::sync::watch` for broadcast shutdown notifications without consuming cancellation signals.
|
||||
* **Consolidated Neovim Tool Suite (v2)**: The Neovim server exposes 7 consolidated domain tools (`nvim_buffer`, `nvim_window`, `nvim_view`, `nvim_diagnostics`, `nvim_visual`, `nvim_execute_lua`, `nvim_system`) with comprehensive action dispatching.
|
||||
* **Fallback Vector Search Parity**: In-memory vector search fallback indexes Knowledge Graph entities, observations, and error fixes when external vector databases are unavailable.
|
||||
|
||||
|
||||
+8
-1
@@ -332,7 +332,10 @@ pub async fn run_server(state: Arc<MemoryState>) -> Result<(), Box<dyn std::erro
|
||||
}
|
||||
|
||||
let (tech_debts, adrs) = if let Some(ref f) = payload.file {
|
||||
crate::api::telemetry::find_projected_knowledge(&nvim_udp_state.handler.state, f)
|
||||
crate::api::telemetry::find_projected_knowledge(
|
||||
&nvim_udp_state.handler.state,
|
||||
f,
|
||||
)
|
||||
} else {
|
||||
(Vec::new(), Vec::new())
|
||||
};
|
||||
@@ -678,8 +681,11 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
static ENV_LOCK: std::sync::Mutex<()> = std::sync::Mutex::new(());
|
||||
|
||||
#[test]
|
||||
fn test_init_logging_helper() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
unsafe {
|
||||
std::env::set_var("MCP_MEMORY_STORE_DIR", temp_dir.path().to_str().unwrap());
|
||||
@@ -711,6 +717,7 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_run_server_graceful_shutdown() {
|
||||
let _lock = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
|
||||
let temp_dir = tempfile::tempdir().unwrap();
|
||||
let state = Arc::new(MemoryState::new(temp_dir.path().to_str().unwrap()));
|
||||
|
||||
|
||||
@@ -1,4 +1,6 @@
|
||||
use crate::embedding::{cosine_similarity, generate_embedding_async, generate_embeddings_async};
|
||||
use crate::models::{Adr, Entity, Snippet, Task};
|
||||
use crate::state::MemoryState;
|
||||
use std::sync::{Arc, Mutex};
|
||||
use tantivy::schema::*;
|
||||
use tantivy::{Index, IndexReader, IndexWriter, ReloadPolicy, doc};
|
||||
@@ -432,6 +434,154 @@ impl MemoryIndex {
|
||||
}
|
||||
}
|
||||
|
||||
pub struct SearchService {
|
||||
state: Arc<MemoryState>,
|
||||
}
|
||||
|
||||
impl SearchService {
|
||||
pub fn new(state: Arc<MemoryState>) -> Self {
|
||||
Self { state }
|
||||
}
|
||||
|
||||
pub async fn semantic_search(
|
||||
&self,
|
||||
query: &str,
|
||||
filter_namespace: Option<&str>,
|
||||
limit: usize,
|
||||
) -> crate::error::Result<Vec<SearchResult>> {
|
||||
let query_emb = generate_embedding_async(query.to_string())
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
if query_emb.is_empty() {
|
||||
return Ok(Vec::new());
|
||||
}
|
||||
|
||||
let mut results = Vec::new();
|
||||
let mut uncached_texts = Vec::new();
|
||||
let mut uncached_meta = Vec::new();
|
||||
|
||||
self.state.code.snippets.read_with(|snips| {
|
||||
for snippet in snips.iter() {
|
||||
if let Some(ref emb) = snippet.embedding {
|
||||
let sim = cosine_similarity(&query_emb, emb);
|
||||
results.push(SearchResult {
|
||||
id: snippet.name.clone(),
|
||||
doc_type: "snippet".to_string(),
|
||||
title: snippet.name.clone(),
|
||||
body: snippet.description.clone(),
|
||||
score: sim,
|
||||
});
|
||||
} else if uncached_texts.len() < 50 {
|
||||
uncached_texts.push(format!(
|
||||
"{} {} {}",
|
||||
snippet.name, snippet.description, snippet.code
|
||||
));
|
||||
uncached_meta.push((
|
||||
snippet.name.clone(),
|
||||
"snippet".to_string(),
|
||||
snippet.description.clone(),
|
||||
));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
self.state.read_graph(|graph| {
|
||||
for entity in graph.entities.values() {
|
||||
if let Some(ns) = filter_namespace {
|
||||
if entity.namespace != ns {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
let obs = entity.observations.join("; ");
|
||||
let desc = format!("{}: {}", entity.entity_type, obs);
|
||||
if let Some(ref emb) = entity.embedding {
|
||||
let sim = cosine_similarity(&query_emb, emb);
|
||||
results.push(SearchResult {
|
||||
id: entity.name.clone(),
|
||||
doc_type: "entity".to_string(),
|
||||
title: entity.name.clone(),
|
||||
body: desc,
|
||||
score: sim,
|
||||
});
|
||||
} else if uncached_texts.len() < 50 {
|
||||
uncached_texts.push(format!("{} {} {}", entity.name, entity.entity_type, obs));
|
||||
uncached_meta.push((entity.name.clone(), "entity".to_string(), desc));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
self.state.code.error_fixes.read_with(|fixes| {
|
||||
for fix in fixes.iter() {
|
||||
if let Some(ref emb) = fix.embedding {
|
||||
let sim = cosine_similarity(&query_emb, emb);
|
||||
results.push(SearchResult {
|
||||
id: fix.signature.clone(),
|
||||
doc_type: "error_fix".to_string(),
|
||||
title: fix.signature.clone(),
|
||||
body: fix.solution.clone(),
|
||||
score: sim,
|
||||
});
|
||||
} else if uncached_texts.len() < 50 {
|
||||
uncached_texts.push(format!("{} {}", fix.signature, fix.solution));
|
||||
uncached_meta.push((
|
||||
fix.signature.clone(),
|
||||
"error_fix".to_string(),
|
||||
fix.solution.clone(),
|
||||
));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if !uncached_texts.is_empty()
|
||||
&& let Ok(embeddings) = generate_embeddings_async(uncached_texts).await
|
||||
{
|
||||
for (emb, (title, doc_type, body)) in embeddings.into_iter().zip(uncached_meta) {
|
||||
let sim = cosine_similarity(&query_emb, &emb);
|
||||
results.push(SearchResult {
|
||||
id: title.clone(),
|
||||
doc_type,
|
||||
title,
|
||||
body,
|
||||
score: sim,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
results.sort_by(|a, b| {
|
||||
b.score
|
||||
.partial_cmp(&a.score)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
});
|
||||
results.truncate(limit);
|
||||
|
||||
Ok(results)
|
||||
}
|
||||
|
||||
pub async fn keyword_search(
|
||||
&self,
|
||||
query: &str,
|
||||
filter_namespace: Option<&str>,
|
||||
limit: usize,
|
||||
) -> crate::error::Result<Vec<SearchResult>> {
|
||||
let idx = self.state.get_search_index().await;
|
||||
let matches = idx
|
||||
.search(query, filter_namespace)
|
||||
.map_err(|e| crate::error::AppError::Internal(e.to_string()))?;
|
||||
|
||||
let mut results = Vec::new();
|
||||
for (id, doc_type, title, body, score) in matches.into_iter().take(limit) {
|
||||
results.push(SearchResult {
|
||||
id,
|
||||
doc_type,
|
||||
title,
|
||||
body,
|
||||
score,
|
||||
});
|
||||
}
|
||||
Ok(results)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
+1
-140
@@ -469,143 +469,4 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
use crate::embedding::{cosine_similarity, generate_embedding_async, generate_embeddings_async};
|
||||
|
||||
pub struct UnifiedSearchResult {
|
||||
pub id: String,
|
||||
pub doc_type: String,
|
||||
pub title: String,
|
||||
pub body: String,
|
||||
pub score: f32,
|
||||
}
|
||||
|
||||
pub struct SearchService {
|
||||
state: Arc<MemoryState>,
|
||||
}
|
||||
|
||||
impl SearchService {
|
||||
pub fn new(state: Arc<MemoryState>) -> Self {
|
||||
Self { state }
|
||||
}
|
||||
|
||||
pub async fn semantic_search(
|
||||
&self,
|
||||
query: &str,
|
||||
filter_namespace: Option<&str>,
|
||||
limit: usize,
|
||||
) -> crate::error::Result<Vec<UnifiedSearchResult>> {
|
||||
let query_emb = generate_embedding_async(query.to_string())
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
let mut results = Vec::new();
|
||||
let mut cached_items = Vec::new();
|
||||
let mut uncached_texts = Vec::new();
|
||||
let mut uncached_meta = Vec::new();
|
||||
|
||||
self.state.code.snippets.read_with(|snips| {
|
||||
for snippet in snips.iter() {
|
||||
let title = snippet.name.clone();
|
||||
let desc = snippet.description.clone();
|
||||
if let Some(ref emb) = snippet.embedding {
|
||||
cached_items.push((title, "snippet".to_string(), desc, emb.clone()));
|
||||
} else if uncached_texts.len() < 50 {
|
||||
uncached_texts.push(format!(
|
||||
"{} {} {}",
|
||||
snippet.name, snippet.description, snippet.code
|
||||
));
|
||||
uncached_meta.push((title, "snippet".to_string(), desc));
|
||||
}
|
||||
}
|
||||
});
|
||||
self.state.read_graph(|graph| {
|
||||
for entity in graph.entities.values() {
|
||||
if let Some(ns) = filter_namespace {
|
||||
if entity.namespace != ns {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
let title = entity.name.clone();
|
||||
let obs = entity.observations.join("; ");
|
||||
let desc = format!("{}: {}", entity.entity_type, obs);
|
||||
if let Some(ref emb) = entity.embedding {
|
||||
cached_items.push((title, "entity".to_string(), desc, emb.clone()));
|
||||
} else if uncached_texts.len() < 50 {
|
||||
uncached_texts.push(format!("{} {} {}", entity.name, entity.entity_type, obs));
|
||||
uncached_meta.push((title, "entity".to_string(), desc));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
self.state.code.error_fixes.read_with(|fixes| {
|
||||
for fix in fixes.iter() {
|
||||
let title = fix.signature.clone();
|
||||
let desc = fix.solution.clone();
|
||||
if let Some(ref emb) = fix.embedding {
|
||||
cached_items.push((title, "error_fix".to_string(), desc, emb.clone()));
|
||||
} else if uncached_texts.len() < 50 {
|
||||
uncached_texts.push(format!("{} {}", fix.signature, fix.solution));
|
||||
uncached_meta.push((title, "error_fix".to_string(), desc));
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
for (title, doc_type, body, emb) in cached_items {
|
||||
let sim = cosine_similarity(&query_emb, &emb);
|
||||
results.push(UnifiedSearchResult {
|
||||
id: title.clone(),
|
||||
doc_type,
|
||||
title,
|
||||
body,
|
||||
score: sim,
|
||||
});
|
||||
}
|
||||
|
||||
if !uncached_texts.is_empty()
|
||||
&& let Ok(embeddings) = generate_embeddings_async(uncached_texts).await
|
||||
{
|
||||
for (emb, meta) in embeddings.into_iter().zip(uncached_meta) {
|
||||
let sim = cosine_similarity(&query_emb, &emb);
|
||||
results.push(UnifiedSearchResult {
|
||||
id: meta.0.clone(),
|
||||
doc_type: meta.1.clone(),
|
||||
title: meta.0,
|
||||
body: meta.2,
|
||||
score: sim,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
results.sort_by(|a, b| {
|
||||
b.score
|
||||
.partial_cmp(&a.score)
|
||||
.unwrap_or(std::cmp::Ordering::Equal)
|
||||
});
|
||||
results.truncate(limit);
|
||||
|
||||
Ok(results)
|
||||
}
|
||||
|
||||
pub async fn keyword_search(
|
||||
&self,
|
||||
query: &str,
|
||||
filter_namespace: Option<&str>,
|
||||
limit: usize,
|
||||
) -> crate::error::Result<Vec<UnifiedSearchResult>> {
|
||||
let idx = self.state.get_search_index().await;
|
||||
let matches = idx
|
||||
.search(query, filter_namespace)
|
||||
.map_err(|e| crate::error::AppError::Internal(e.to_string()))?;
|
||||
|
||||
let mut results = Vec::new();
|
||||
for (id, doc_type, title, body, score) in matches.into_iter().take(limit) {
|
||||
results.push(UnifiedSearchResult {
|
||||
id,
|
||||
doc_type,
|
||||
title,
|
||||
body,
|
||||
score,
|
||||
});
|
||||
}
|
||||
Ok(results)
|
||||
}
|
||||
}
|
||||
pub use crate::search::{SearchResult as UnifiedSearchResult, SearchService};
|
||||
+34
-80
@@ -444,14 +444,35 @@ impl<T: DeserializeOwned + Default + Serialize + Send + Sync + 'static> Store<T>
|
||||
(*lock).clone()
|
||||
};
|
||||
|
||||
// Expensive serialization and granular extraction run completely unblocked outside the lock
|
||||
let full_bytes_res = serde_json::to_vec(&new_snapshot);
|
||||
let granular_entries = full_bytes_res
|
||||
.as_ref()
|
||||
.ok()
|
||||
.and_then(|bytes| serde_json::from_slice::<serde_json::Value>(bytes).ok())
|
||||
.map(|val| Self::extract_granular_entries(&self.key, &val))
|
||||
.unwrap_or_default();
|
||||
match self.prepare_batch(&new_snapshot) {
|
||||
Ok((batch_inserts, removed_keys)) => {
|
||||
self.queue.push_batch(
|
||||
self.key.clone(),
|
||||
batch_inserts,
|
||||
removed_keys,
|
||||
self.flushed.clone(),
|
||||
);
|
||||
}
|
||||
Err(e) => tracing::error!(
|
||||
"Failed to serialize memory store for key '{}': {}",
|
||||
self.key,
|
||||
e
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
fn prepare_batch(
|
||||
&self,
|
||||
new_snapshot: &T,
|
||||
) -> Result<(Vec<(String, Vec<u8>)>, Vec<String>), serde_json::Error>
|
||||
where
|
||||
T: Serialize,
|
||||
{
|
||||
let full_bytes = serde_json::to_vec(new_snapshot)?;
|
||||
let granular_entries = match serde_json::from_slice::<serde_json::Value>(&full_bytes) {
|
||||
Ok(val) => Self::extract_granular_entries(&self.key, &val),
|
||||
Err(_) => Vec::new(),
|
||||
};
|
||||
|
||||
let new_keys: std::collections::HashSet<String> =
|
||||
granular_entries.iter().map(|(k, _)| k.clone()).collect();
|
||||
@@ -469,48 +490,11 @@ impl<T: DeserializeOwned + Default + Serialize + Send + Sync + 'static> Store<T>
|
||||
*known = new_keys;
|
||||
}
|
||||
|
||||
match full_bytes_res {
|
||||
Ok(data) => {
|
||||
let mut batch_inserts = Vec::with_capacity(granular_entries.len() + 1);
|
||||
for (g_key, g_bytes) in granular_entries {
|
||||
batch_inserts.push((g_key, g_bytes));
|
||||
}
|
||||
batch_inserts.push((self.key.clone(), data.clone()));
|
||||
batch_inserts.extend(granular_entries);
|
||||
batch_inserts.push((self.key.clone(), full_bytes));
|
||||
|
||||
if self
|
||||
.queue
|
||||
.push_batch(
|
||||
self.key.clone(),
|
||||
batch_inserts.clone(),
|
||||
removed_keys.clone(),
|
||||
self.flushed.clone(),
|
||||
)
|
||||
.is_none()
|
||||
{
|
||||
tracing::warn!(
|
||||
"DbWriteQueue channel full for key '{}'. Applying backpressure fallback.",
|
||||
self.key
|
||||
);
|
||||
let queue = self.queue.clone();
|
||||
let key = self.key.clone();
|
||||
let flushed = self.flushed.clone();
|
||||
if let Ok(handle) = tokio::runtime::Handle::try_current() {
|
||||
handle.spawn(async move {
|
||||
let _ = tokio::time::timeout(
|
||||
std::time::Duration::from_secs(10),
|
||||
queue.push_batch_async(key, batch_inserts, removed_keys, flushed),
|
||||
)
|
||||
.await;
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => tracing::error!(
|
||||
"Failed to serialize memory store for key '{}': {}",
|
||||
self.key,
|
||||
e
|
||||
),
|
||||
}
|
||||
Ok((batch_inserts, removed_keys))
|
||||
}
|
||||
|
||||
pub async fn modify_async<F: FnOnce(&mut T)>(&self, f: F)
|
||||
@@ -531,38 +515,8 @@ impl<T: DeserializeOwned + Default + Serialize + Send + Sync + 'static> Store<T>
|
||||
(*lock).clone()
|
||||
};
|
||||
|
||||
let full_bytes_res = serde_json::to_vec(&new_snapshot);
|
||||
let granular_entries = full_bytes_res
|
||||
.as_ref()
|
||||
.ok()
|
||||
.and_then(|bytes| serde_json::from_slice::<serde_json::Value>(bytes).ok())
|
||||
.map(|val| Self::extract_granular_entries(&self.key, &val))
|
||||
.unwrap_or_default();
|
||||
|
||||
let new_keys: std::collections::HashSet<String> =
|
||||
granular_entries.iter().map(|(k, _)| k.clone()).collect();
|
||||
let mut removed_keys = Vec::new();
|
||||
{
|
||||
let mut known = self
|
||||
.known_granular_keys
|
||||
.write()
|
||||
.unwrap_or_else(|e| e.into_inner());
|
||||
for old_k in known.iter() {
|
||||
if !new_keys.contains(old_k) {
|
||||
removed_keys.push(old_k.clone());
|
||||
}
|
||||
}
|
||||
*known = new_keys;
|
||||
}
|
||||
|
||||
match full_bytes_res {
|
||||
Ok(data) => {
|
||||
let mut batch_inserts = Vec::with_capacity(granular_entries.len() + 1);
|
||||
for (g_key, g_bytes) in granular_entries {
|
||||
batch_inserts.push((g_key, g_bytes));
|
||||
}
|
||||
batch_inserts.push((self.key.clone(), data));
|
||||
|
||||
match self.prepare_batch(&new_snapshot) {
|
||||
Ok((batch_inserts, removed_keys)) => {
|
||||
if let Some(rx) = self
|
||||
.queue
|
||||
.push_batch_async(
|
||||
|
||||
Reference in new issue
Block a user