0 Fork 0

Files

T

Riz Ashraf 0c2fc96ff8 chore: housekeeping

2026-10-09 09:33:54 +01:00

28 KiB

Raw Permalink Blame History

Memory MCP Strategic Guidelines

This document outlines the STRATEGY, SEMANTICS, RESOURCE SCHEMAS, and CASING STANDARDS for using the MCP Memory Server. You do not need to memorize individual JSON schemas for every tool; they are strictly defined and typed in the tools/list endpoint. Focus on WHEN, WHY, and HOW to leverage them effectively.

1. Casing & Naming Standards (CRITICAL)

To prevent graph fragmentation and ensure seamless LLM context retrieval:

Entity Types (entity_type): MUST ALWAYS be PascalCase (e.g. DatabaseTable, McpTool, ArchitectureComponent, File, DataStructure).
Relation Types (relation_type): MUST ALWAYS be snake_case (e.g. depends_on, calls, implements, uses, contains).
Field Keys & Properties: MUST ALWAYS be snake_case (e.g. file_path, git_commit, created_at).
Relation Tool Parameters: create_relations supports Serde field aliases (source -> from, target -> to, relationType/type -> relation_type) so LLM tool calls succeed seamlessly regardless of parameter naming.

Note

The server automatically enforces and migrates incoming entity and relation types to these canonical casing rules on every read and write operation. Store operations enforce atomic lock scope for serialization/deserialization and enter Quarantine Mode upon database corruption to prevent data overwrite hazards.

2. The Two-Tier Context Paradigm

Important

Tier 1 (Static Markdown): Repository rules, constraints, architectural patterns, and developer preferences are maintained directly in static git-tracked files (agent-rules/*.md, instructions.md) and system prompts. This guarantees 0ms turn-0 availability without relying on proactive agent tool retrieval.

Tier 2 (Telemetry & Ephemeral DB): High-frequency operational history—the Code Change Ledger (audit_ledger), terminal command history, compiler error fixes (log_error_fix), active tasks, and preflight context—is handled by the MCP Memory Server and surfaced via the Brain Monitor Web UI and MCP resources.

3. Passive MCP Resources (`resources/list`)

The server exposes 9 real-time, read-only MCP resources. Agents should read these resources directly to regain context without incurring tool call latency:

Resource URI	Resource Name	Description & Usage
`memory://graph/entities`	Graph Entities	All nodes and entities in the knowledge graph (top 100 with pagination guidance).
`memory://graph/relations`	Graph Relations	All relationships between graph entities (top 200 with subgraph guidance).
`memory://tasks/active`	Active Tasks	Current active tasks with status, priority, and assigned subagents.
`memory://decisions/active`	Active ADR Decisions	Architectural decisions currently in `accepted` status.
`memory://tech_debt/unresolved`	Unresolved Tech Debt	All open engineering debt items requiring future refactoring.
`memory://session/delta`	Session Delta	Code modifications, commits, active tasks, and notes created in the last 2 hours.
`memory://terminal/recent`	Terminal History	Recent terminal commands, interpreters (`pwsh`, `bash`, `nu`), working dirs, and exit codes.
`memory://activity/recent`	Recent Activity	Real-time IDE and developer activity event stream.
`memory://milestones`	Milestones	Project milestones, deliverables, target dates, and progress.

4. MCP Workflow Prompts (`prompts/list`)

The server registers 5 high-signal workflow prompts to initiate standardized agent routines:

context_warmup: Executed at session startup. Prompts the agent to read memory://tasks/active and memory://session/delta, inspect the workspace worktree, and assemble immediate working context.
analyze_tech_debt: Prompts the agent to review unresolved technical debt from memory://tech_debt/unresolved and generate a prioritized remediation plan.
summarize_architecture: Synthesizes active ADRs from memory://decisions/active and graph entities from memory://graph/entities into an architectural overview.
handoff_routine: Triggers the DevOpsSRE subagent at session end to generate a standup report, audit active tasks, and record a handoff memo for future sessions.
archive_routine: Compresses historical session summaries into a dense milestone retrospective entity and purges pruned entries.

5. Consolidated Smart Tools Architecture (11 Primary Tools)

The server consolidates granular single-purpose tools into domain-named smart tools.

tasks: Complete task lifecycle management.
- action: "add": Create a new task (requires title, optional description, git_branch, repo_name, priority: "low" | "medium" | "high" | "urgent", assigned_agent, verification_command, parent_id, dependencies).
- action: "update": Update task status (requires id, status: "pending" | "completed" | "cancelled").
- action: "delete": Delete task and child tasks (requires id).
- action: "list": List active tasks (optional git_branch, summary_level: "compact" | "detailed" | "full", max_tokens).
- action: "set_criteria": Set acceptance criteria (requires id, criteria: Vec<String>).
- action: "verify": Verify criteria met (requires id, optional proof).
milestones: Milestone tracking.
- action: "add": Create milestone (requires title, optional namespace, target_date, description, deliverables: Vec<String>, repo_name).
- action: "update": Update milestone status (requires id, status: "active" | "completed" | "cancelled").
- action: "list": List milestones (optional namespace).
handoff_memos: Session handoff notes for future agents.
- action: "leave": Leave a memo (requires content, optional vcs_revision, repo_name, git_branch, blockers: Vec<String>, action_items: Vec<String>, expires_at).
- action: "read": Read active handoff memos.
- action: "clear": Clear memos.
snippets: Reusable code snippet vault.
- action: "store": Store snippet (requires query as name, optional language, code, description, tags, origin_file, line_range, repo_name).
- action: "search": Search snippet vault (optional query, tags, hybrid: true).
- action: "delete": Delete snippet (requires id).
- action: "tag": Attach classification tags (requires id, tags: Vec<String>).
decisions: Architectural Decision Records (ADRs).
- action: "log": Log ADR (requires title, optional status: "accepted" | "proposed" | "deprecated" | "superseded" | "implemented", context, decision, consequence, author, affected_components: Vec<String>, alternatives_considered: Vec<String>, supersedes, repo_name).
- action: "update": Update ADR status and metadata (requires id, optional status, git_commit, git_branch).
- action: "query": Query ADRs (optional query).
- action: "delete": Delete ADR (requires id).
tech_debt: Engineering debt backlog.
- action: "log": Log debt item (requires description, optional ideal_solution, file_path, line_range, workaround, effort_estimate, severity: "low" | "medium" | "high" | "critical", git_commit, git_branch, symbol_references, repo_name).
- action: "resolve": Resolve debt item (requires id).
- action: "list": List debt items (optional include_resolved).
environment: Infrastructure and requirements tracking.
- action: "update_fingerprint": Update tool versions (requires tool_versions: Map<String, String>, optional repo_name).
- action: "read_fingerprint": Read tool versions fingerprint.
- action: "log_requirement": Log environment variable requirement (requires key, description, is_secret, optional default_value, validation_regex, repo_name).
- action: "register": Register target environment (requires name, url, optional description, requires_vpn, env_type: "dev" | "staging" | "qa" | "prod", healthcheck_endpoint, ssh_host, repo_name).
- action: "get_details": Read full environment details.
clipboard: OS Clipboard management.
- action: "read": Read OS clipboard.
- action: "write": Write text/html/files/image to clipboard.
hypotheses: Diagnostic hypothesis memory.
- action: "log": Record diagnostic hypothesis (requires hypothesis, optional status: "open" | "verified" | "disproven", evidence: Vec<String>, test_command, git_branch, repo_name).
- action: "query": Query hypotheses (optional status, git_branch, repo_name).
agent_signals: Inter-agent signal bus.
- action: "broadcast": Broadcast signal to other agents (requires signal_type, payload, optional target_agent, ttl_seconds).
- action: "query": Query active signals (optional signal_type, include_expired: bool).
process_logs: Process and daemon log management.
- action: "watch": Register or update log file watcher (requires label, file_path, optional description).
- action: "get": Retrieve recent log lines from watched process log (requires label, optional tail_lines: usize).
- action: "clear": Clear or truncate watched process log file (requires label).

6. Complete Tool Catalog (All 53 Tools)

The server exposes 53 specialized and smart MCP tools organized into 7 functional domains:

1. Knowledge Graph Core (18 Tools)

create_entities: Batch-create entities with name, entity_type, and observations.
create_relations: Batch-create relationships (from, to, relation_type).
add_observations: Append observations to existing entities.
delete_entities: Delete entity nodes and cascading relations.
delete_relations: Delete specific relation edges between entities.
delete_observations: Remove specific observations from an entity.
read_graph: Return full or namespace-filtered knowledge graph.
search_nodes: Search entity names and observations using Tantivy BM25.
open_nodes: Inspect full details of specified entity nodes by name.
visualize_graph: Generate Mermaid markdown or SVG diagram of the graph.
condense_entity: Summarize entity observations into dense summaries.
merge_entities: Merge source entity into target entity, re-pointing relations and pruning self-loops.
find_orphans: Detect entities with zero relationships for pruning.
get_subgraph: BFS graph traversal expanding N hops from a root node.
sweep_graph_health: Audit graph for orphans, calculate name similarity, and recommend merges.
resolve_stale_symbols: Cross-reference graph symbols against the workspace AST to remove deleted code nodes.
summarize_subgraph: LLM-ready concise synthesis of a localized subgraph.
query_graph_path: BFS shortest path between two entities in the knowledge graph.

2. Task & Milestone Operations (2 Tools)

tasks: Consolidated task board manager (add, update, delete, list, set_criteria, verify).
milestones: Milestone lifecycle management (add, update, list).

3. Notes, Handoffs & Reporting (4 Tools)

handoff_memos: Cross-session scratchpad and handoff memos (leave, read, clear).
add_session_summary: Record session summary notes and highlights.
generate_standup_report: Synthesize tasks, ledger changes, and session summaries into a standup report.
promote_to_entity: Promote an ephemeral note or memo into a permanent knowledge graph entity.

4. Meta, Audit & Intelligence (15 Tools)

decisions: Consolidated Architectural Decision Records (ADRs) manager (log, update, query, delete).
tech_debt: Consolidated technical debt backlog manager (log, resolve, list).
log_error_fix: Record an error resolution with stack trace, root cause, and git commit.
search_error_fixes: Embedding-based and keyword search over past error resolutions.
log_code_change: Record a file modification in the VCS-agnostic audit ledger.
query_recent_changes: Retrieve recent code changes with lookback time filters.
omni_search: Reciprocal Rank Fusion (RRF) search across all graph entities, snippets, ADRs, debt, and fixes.
get_project_health: Health dashboard summarizing task completion, debt backlog, and graph consistency.
manage_checkpoint: Create or restore named memory snapshots for safe rollback.
query_lineage: Causal lineage tracker linking tasks, ADRs, commits, and error fixes.
get_next_actionable_tasks: Topologically resolved list of unblocked tasks ready for execution.
hypotheses: Structured diagnostic hypothesis tracker (log, query).
get_preflight_context: Aggregated operational context at session start (tasks, debt, recent changes).
agent_signals: Inter-agent signal bus (broadcast, query).
auto_session_checkpoint: Automatic session boundary checkpointing.

5. System, Environment & Telemetry (4 Tools)

environment: Tool fingerprinting, requirements, and environment registry (update_fingerprint, read_fingerprint, log_requirement, register, get_details).
snippets: Reusable code snippet vault with hybrid search (store, search, delete, tag).
clipboard: Pure native Rust OS clipboard interface (read, write).
process_logs: Live process and daemon log watcher and tailer (watch, get, clear).

6. Git & Worktree Context (2 Tools)

get_active_worktree_context: Inspect git status, modified files, diff summary, and current branch.
query_git_diffs: Retrieve detailed git diffs for specific files or commit ranges.

7. AST & Code Intelligence (8 Tools)

read_file_skeleton: Tree-sitter AST structural outline of functions, structs, and methods without implementation bodies.
replace_ast_node: Precise AST node replacement preserving indentation and comments.
find_symbol_references: Search for symbol references across snippets and disk source files.
get_callers: Find call sites and callers of a specified function or method across the codebase.
analyze_impact: Blast-radius impact analysis of modifying a symbol or file.
read_directory_architecture: Recursive directory structure analysis capped at depth 10.
semantic_code_search: Dense vector semantic code search over indexed source code.
manage_subagent_namespace: Create, isolate, or merge subagent-scoped memory namespaces.

7. VCS & SVN Agnosticism & Multi-Repo Provenance

To support diverse enterprise repositories (Git, Subversion / SVN, Mercurial / Hg, Monorepos):

vcs_type: Designates the VCS engine ("git", "svn", "hg", "perforce", or "none").
vcs_revision: Agnostic commit hash or SVN revision identifier (e.g., "r12458", "3e4f7a9").
upstream_url: Canonical remote repository URL (e.g. https://svn.corp/repo/trunk, git@bitbucket.org:org/repo.git).
repo_name: Logical project identifier allowing multiple repositories to share or partition memory namespaces cleanly without collision.
Audit Ledger (log_code_change): Enriched with vcs_type, vcs_revision, upstream_url, author, diff_summary, and extensible metadata: HashMap<String, String>.

8. Terminal & Process Telemetry

The server ingests and tracks active terminal commands and sessions:

Active Terminals: Tracks PIDs, shell interpreters (pwsh, bash, nu, zsh), current working directories (cwd), command exit codes, and timestamps.
Terminal History Endpoint & Resource: /terminal/history and memory://terminal/recent expose recent shell commands and output streams to dashboard and LLMs to prevent lost shell context.
Zero-Latency UDP Streams: Terminal and IDE telemetry stream over UDP (MCP_UDP_PORT1, MCP_UDP_PORT2) with zero disk I/O bottlenecks.

9. Automated Error Fix Auto-Matcher

Tools: log_error_fix, search_error_fixes (and alias suggest_error_fix)
When to use: When encountering a build error, test failure, or stack trace. Call search_error_fixes with either a text query or stack_trace before attempting a fix from scratch.
Behavior: Computes cosine similarity between error trace embeddings and past resolution logs when stack_trace is provided, or keyword filtering when query is provided, returning top matched solutions, modified files, and git commits.

10. Memory State Checkpointing & Rollbacks

Tool: manage_checkpoint (action: "create" | "restore")
When to use: Before initiating a large refactor, running experimental subagent tasks, or executing destructive batch operations.
Behavior: Saves or restores a point-in-time snapshot of graph entities, active tasks, and tech debt backlogs.

11. Self-Healing Graph Health Sweeper

Tool: sweep_graph_health
When to use: Periodically or before committing major graph changes to audit entity consistency.
Behavior: Detects orphaned nodes (0 relations), computes name similarity using pre-computed lowercase keys to identify near-duplicates (e.g., APIGateway vs ApiGateway), and provides structured merge_entities recommendations or auto-prunes orphans.

12. Causal Lineage & Provenance Tracker

Tool: query_lineage
When to use: When asking "Why was this component modified?" or "What task or ADR led to this code change?"
Behavior: Searches across tasks, ADRs, audit ledger entries, and error fixes to assemble a unified chronological timeline explaining the provenance behind any file, symbol, or commit.

13. ADR Lifecycle & Automated Git Post-Commit Reconciliation

The Golden ADR Rule: When code implementing an ADR is committed, you MUST IMMEDIATELY update the ADR status to implemented:

{
  "action": "update",
  "id": "ADR-XXXX",
  "status": "implemented",
  "git_commit": "<commit_hash>",
  "git_branch": "<branch>"
}

Automated Post-Commit Hook: The repository provides an automated reconciliation script (scripts/git-reconcile.py) installed via just install-git-hooks. Upon every git commit, the hook scans the commit message for ADR-XXXX or task identifiers and reconciles their status in the persistent store.
Current Architecture Status: 100% of defined ADRs (ADR-0080 through ADR-0104) are fully implemented and reconciled in the store.

14. Native Rust Invariants & Subprocess Prohibition (CRITICAL)

To maintain maximum security, speed, and cross-platform reliability:

Zero Subprocess Fallbacks: System and server tools (clipboard, ast, search, db) MUST strictly use pure native Rust crates (arboard, tree-sitter, tantivy, psycopg). Invocations of external shell commands (powershell.exe, wl-paste, xclip, cmd.exe) are strictly prohibited in native handlers.
Transient Lock Recovery: Transient OS handle collisions (such as Win32 OLE OpenClipboard lock contention) must be handled using native retry loops with backoffs directly in Rust.
Automated Static Regression Gates: Automated AST/source audit tests (test_no_subprocess_clipboard_regression) verify at test time that forbidden subprocess patterns are absent from handler implementations.

15. High-Performance Concurrency & Resilience Guarantees

Explicit Fail-Fast Persistence Safety: Replaced silent fallback to temporary databases (/tmp/mcp_store_fallback_*) with an explicit open retry and fail-fast panic unless MCP_ALLOW_TMP_FALLBACK=1 is explicitly set, preventing silent data loss.
Async Mutex Deadlock Elimination: Converted shared state and Neovim connection locks (shutdown_tx, NVIM_CONN, ACTIVE_SOCKET, HEADLESS_PROC) to tokio::sync::Mutex to prevent worker thread pool starvation across .await points.
Telemetry Session Deduplication & Channel Pruning: Added LAST_SESSION in-memory state deduplication for UDP telemetry writes (eliminating disk I/O thrashing) and distinguished WebSocket TrySendError::Full backpressure vs TrySendError::Closed client pruning.
Graph Adjacency Indexing: Leverages KnowledgeGraph::build_adjacency_map to build O(1) lookup adjacency lists for fast BFS shortest path graph queries.
Atomic Store Write Lock Minimization: Store::modify and Store::modify_async release write guards immediately after in-memory state mutations, serializing JSON payloads under read locks to allow non-blocking concurrent readers.
Async Commit Index Reader Auto-Reload: MemoryIndex::commit() automatically triggers reader.reload() upon completing background commits, guaranteeing immediate visibility of newly indexed document terms.
Zero-Allocation NDJSON Reader: mcp-stdio reclaims line buffers via std::mem::take and in-place trimming to eliminate heap allocations during high-frequency NDJSON message parsing.
Dynamic Character Micro-Batched Fastembed Inference: generate_embeddings_async dynamically batches text arrays using a 16,000 character budget ceiling inside spawn_blocking, preventing heap spikes during vector indexing while keeping ONNX SIMD execution saturated.
Bounded Telemetry Detail Records: Activity and terminal telemetry buffers enforce a 4,000 character truncation ceiling on log details (ActivityRecord, TerminalHistory) to prevent unbounded RAM growth under high RPC throughput.
Zero-Allocation Stream Formatting: Graph condensation loops (condense_graph_worker) format node/relation subgraphs into stream buffers via std::fmt::Write, eliminating intermediate String allocation overhead.
Single-Handle OS Clipboard Retries & Image Downsampling: WriteClipboardHandler initializes OS clipboard handles once per operation and downsamples images exceeding 2048 \times 2048 resolution before writing raw RGBA bytes.
Zero Transaction Drop Persistence Guarantee: Store::modify automatically spawns an async task to execute push_async with channel backpressure if push encounters queue saturation, ensuring zero data loss under spike write loads.
Token-Budgeted Query Projections: Decision queries (query_decisions) support limit caps and compact include_body: false projections for token budget optimization.
Serde Parameter & Enum Ergonomics: Action enums (SnippetSearchMode, Relation) support case-insensitive variants and common synonyms (create/add, remove/delete, list/read, source/from, target/to, relationType/relation_type) ensuring seamless LLM tool execution.
Atomic Search Index Swaps: MemoryState::rebuild_index constructs and populates a new MemoryIndex instance in isolation before performing an atomic pointer swap (*self.search_index.write().await = new_idx), eliminating transient empty search result windows.
Non-Blocking Tantivy Search Queries: MemoryIndex::search queries current index searcher snapshots without executing synchronous disk commit locks, preventing query stalls during heavy background indexing.
Zero-Allocation HashSet<&str> Snippet Deduplication: indexer.rs utilizes borrowed HashSet<&str> name lookups during snippet batch modifications, eliminating heap string re-allocations inside the store write lock.
AST Recursion Depth Safeguard & Zero-Copy Borrowing: Tree-sitter AST traversal caps recursion depth at 100 to prevent thread stack overflows and borrows string slices (&str) during AST node walking.
Strongly-Typed SearchResult & Pre-Allocated Search Vectors: search.rs uses a strongly-typed SearchResult struct with named fields and pre-allocates result vector capacity (Vec::with_capacity(top_docs.len())).
BFS Graph Traversal Pre-allocation & Visited Node Upper Bound: GraphQueryBuilder::find_shortest_path pre-allocates adjacency map capacity (HashMap::with_capacity(relations.len() * 2)) and enforces a visited node upper bound (10,000 max) to guarantee deterministic BFS runtime.
Filesystem Event Debouncing & Proactive State Refresh: spawn_watcher implements a sliding 250ms debouncing window per file path, ignores .git, target, .gemini, and node_modules, and broadcasts activity events to MemoryState.
Buffered Line-by-Line AST Workspace Symbol Scanning: scan_workspace_for_symbol reads workspace files via BufReader line streams instead of loading entire files into heap strings, preventing memory spikes when traversing source trees.
AST Node Type Aliasing & Skeleton Preallocation: replace_ast_node documents friendly node aliases (function, fn, method, struct, class, enum, trait, type), and read_file_skeleton preallocates string buffer capacity (code.len() / 2).
Batch Vector Indexing & Similarity Score Guidance: VectorDB provides index_documents_batch for single-request multi-point vector upserts and explicit score calibration notes (\ge 0.75 high confidence match).
Compact JSON MCP Resources & UTF-8 Activity Truncation: MCP resources serialize using compact JSON (to_string), TerminalHistoryResource / MilestonesResource enforce output bounds, and format_tool_activity_description uses floor_char_boundary for guaranteed UTF-8 safety.
SIMD-Friendly Single-Pass Cosine Similarity: cosine_similarity calculates dot product and Euclidean norm squares in a single linear pass over float vectors, enabling SIMD compiler auto-vectorization.
Safe Stream Decoding on Log Tails: Log tail operations (process_logs, action: "get") read raw bytes and decode using lossy UTF-8 conversion (String::from_utf8_lossy) to ensure resilience when seeking across multi-byte UTF-8 boundaries.
Task Summary UTF-8 Truncation Safety: tasks tool (action = "list") truncates serialized task text strictly along UTF-8 character boundaries using floor_char_boundary when enforcing max_tokens.
Directory Tree Depth Safeguard: ReadDirectoryArchitectureHandler caps directory recursion at depth 10 to prevent stack overflow on deep or cyclic directory structures.
Deterministic Total-Order Score Ranking: OmniSearchHandler uses f64::total_cmp for Reciprocal Rank Fusion (RRF) score sorting, guaranteeing deterministic NaN-safe search result ordering.
RPC Timeout Memory Hygiene: nvim-core maintains request hygiene by removing pending request entries from static RPC maps upon timeout or channel drop, eliminating orphan memory leaks.
Path Traversal Security Guards: validate_safe_path enforces path canonicalization and rejects relative parent traversal components (..) across file and process log handlers (process_logs / ProcessLogsTool).
Watcher Map Memory Eviction: Proactive daemon file watcher in watcher.rs caps last_processed map size at 1,000 entries and purges entries older than 10 minutes to prevent monotonic memory leakage.
Comprehensive Serde Casing Aliases: All 11 consolidated tool action enums (TaskAction, MilestoneAction, SnippetAction, DecisionAction, TechDebtAction, EnvAction, ClipboardAction, HandoffMemoAction, HypothesisAction, AgentSignalAction, ProcessLogAction) include serde alias attributes supporting snake_case, camelCase, PascalCase, and uppercase variants for maximum LLM casing resilience.
Two-Phase Graph Condensation: condense_graph_worker uses a 2-phase commit (non-destructive read_with -> graph insert -> prune by timestamp/content) to prevent data loss if summarization or graph insertion fails.
Redb Database Lock Retry Backoff: init_db retries transient Redb lock contention with exponential backoff (3 attempts, 150ms delay) before falling back.
Offloaded Background Index Rebuilds: MemoryState::rebuild_index offloads graph snapshot cloning and Tantivy document re-indexing into tokio::task::spawn_blocking to avoid stalling async event loops.
Broadcast Watch-Based Shutdown Channels: Background workers utilize tokio::sync::watch for broadcast shutdown notifications without consuming cancellation signals.

28 KiB Raw Permalink Blame History