feat: Implement lightweight UDP interactive UI for Neovim

- Rewrote gemini-ui.lua to send native Neovim prompt selections, Ghost Text diff accept/reject actions, and smart context highlighting directly to the backend via zero-latency UDP port 3002, entirely replacing the old xecute_agent terminal subprocesses.
- Added prompt_user and preview_diff extmark lua functions for non-destructive diff review.
- Updated the Rust backend mcp-memory-server UDP listener to ingest the new gent_ask and interactive schema payloads and route them to the EventBus.
- Updated README.md and instructions.md with UDP documentation.
This commit is contained in:
Riz Ashraf committed 2026-09-29 08:59:23 +01:00
1 parent b3c64c1ad8
commit 4e23043f5d
5 files changed
+147 -27

No files matched your search

+8
View File
@@ -107,6 +107,14 @@ Whenever you alt-tab into a Neovim window (FocusGained) or switch files (BufEnte
2. **WebSocket Telemetry:** Neovim pushes a JSON payload containing the active filename, cursor row, and column via a connectionless UDP datagram to the Rust server's port 3002 listener.
3. **UI Broadcast:** The Rust server updates the global state and broadcasts this over WebSockets (/ws) so that the Brain Monitor Dashboard can animate your active file live in the UI!
### Interactive UDP UI (New)
The UI script (`gemini-ui.lua`) provides a deeply integrated, non-blocking pair-programming experience using zero-latency UDP:
* **Interactive Prompts:** The agent can trigger native `vim.ui.select` or `vim.ui.input` dialogs in your editor. Your responses are instantly routed back to the agent via UDP.
* **Smart Context (`<leader>ai`):** Highlighting code and pressing `<leader>ai` will package your prompt, file, and exact cursor/selection coordinates into a UDP packet and send it directly to the agent without spawning any subprocesses.
* **Ghost Text Diffs (Non-Destructive Review):** Instead of modifying your buffers directly, the agent uses `extmarks` to overlay proposed code changes as grayed-out "Ghost Text".
* Press `<leader>aa` (**A**gent **A**ccept) to apply the change and notify the agent.
* Press `<leader>ar` (**A**gent **R**eject) to dismiss the change and notify the agent.
### Neovim MCP Tools
The LLM agent interacts with your active Neovim session using a dedicated set of MCP tools. *(Note: /nvim/telemetry is strictly a one-way webhook for Neovim; the LLM uses the tools below to interact).*
* **vim_goto_line**: Open files and jump cursors directly from the LLM.
+117 -25
View File
@@ -55,35 +55,118 @@ function M.close_progress()
end
end
local function execute_agent(prompt)
local cmd = { "agy", "--new-project", "-i", prompt }
M.show_progress("Starting Antigravity...")
-- termopen requires an empty and unmodified buffer
vim.api.nvim_buf_set_lines(progress_buf, 0, -1, false, {})
vim.bo[progress_buf].modified = false
-- termopen will attach to the current buffer (which show_progress just focused)
job_id = vim.fn.termopen(cmd, {
on_exit = function(_, code)
job_id = nil
local function send_udp(payload)
local uv = vim.uv or vim.loop
local client = uv.new_udp()
if client then
local ok, data = pcall(vim.fn.json_encode, payload)
if ok then
client:send(data, "127.0.0.1", 3002, function(err)
client:close()
end)
else
client:close()
end
end
end
function M.prompt_user(prompt_text, options, request_id)
if options and #options > 0 then
vim.ui.select(options, { prompt = prompt_text }, function(choice, idx)
send_udp({
event = "agent_prompt_response",
request_id = request_id,
response = choice,
index = idx,
session_id = vim.v.servername
})
vim.cmd('startinsert')
end)
else
vim.ui.input({ prompt = prompt_text }, function(input)
send_udp({
event = "agent_prompt_response",
request_id = request_id,
response = input,
session_id = vim.v.servername
})
end)
end
end
local ns_id = vim.api.nvim_create_namespace("gemini_ghost_text")
local active_diffs = {}
function M.preview_diff(diff_id, bufnr, start_row, end_row, new_lines)
bufnr = (bufnr == 0 or bufnr == nil) and vim.api.nvim_get_current_buf() or bufnr
local virt_lines = {}
for _, line in ipairs(new_lines) do
table.insert(virt_lines, {{line, "DiffAdd"}})
end
local extmark_id = vim.api.nvim_buf_set_extmark(bufnr, ns_id, start_row, 0, {
virt_lines = virt_lines,
virt_lines_above = true,
})
active_diffs[diff_id] = {
bufnr = bufnr,
start_row = start_row,
end_row = end_row,
new_lines = new_lines,
extmark_id = extmark_id
}
if pcall(require, "notify") then
require("notify")("Ghost text diff added. Use <leader>aa to accept or <leader>ar to reject.", "info", { title = "Antigravity" })
else
print("Ghost text diff added. Use <leader>aa to accept or <leader>ar to reject.")
end
end
function M.accept_diff()
local diff_id, diff = next(active_diffs)
if not diff then
print("No active diffs to accept.")
return
end
vim.api.nvim_buf_set_lines(diff.bufnr, diff.start_row, diff.end_row, false, diff.new_lines)
vim.api.nvim_buf_del_extmark(diff.bufnr, ns_id, diff.extmark_id)
active_diffs[diff_id] = nil
send_udp({ event = "agent_diff_accepted", diff_id = diff_id, session_id = vim.v.servername })
print("Diff accepted.")
end
function M.reject_diff()
local diff_id, diff = next(active_diffs)
if not diff then
print("No active diffs to reject.")
return
end
vim.api.nvim_buf_del_extmark(diff.bufnr, ns_id, diff.extmark_id)
active_diffs[diff_id] = nil
send_udp({ event = "agent_diff_rejected", diff_id = diff_id, session_id = vim.v.servername })
print("Diff rejected.")
end
function M.ask_agent()
local file = vim.api.nvim_buf_get_name(0)
vim.ui.input({ prompt = 'Ask Agent: ' }, function(input)
if not input or input == '' then return end
local final_input = input
if file and file ~= "" then
final_input = input .. string.format(" [Context file: %s]", file)
end
execute_agent(final_input)
local cursor = vim.api.nvim_win_get_cursor(0)
send_udp({
event = "agent_ask",
prompt = input,
file = file,
line = cursor[1],
col = cursor[2],
session_id = vim.v.servername
})
M.show_progress("Sent to Antigravity via UDP...")
vim.defer_fn(M.close_progress, 1500)
end)
end
@@ -97,12 +180,21 @@ function M.setup()
if not input or input == '' then return end
local file = vim.api.nvim_buf_get_name(0)
local context = string.format(" [Context: %s lines %d-%d]", file, csrow, cerow)
local final_input = input .. context
execute_agent(final_input)
send_udp({
event = "agent_ask_selection",
prompt = input,
file = file,
start_row = csrow,
end_row = cerow,
session_id = vim.v.servername
})
M.show_progress("Sent selection to Antigravity via UDP...")
vim.defer_fn(M.close_progress, 1500)
end)
end, { desc = "Ask Antigravity Agent about selection", noremap = true, silent = true })
vim.keymap.set('n', '<leader>aa', M.accept_diff, { desc = "Accept Agent Diff", noremap = true, silent = true })
vim.keymap.set('n', '<leader>ar', M.reject_diff, { desc = "Reject Agent Diff", noremap = true, silent = true })
end
return M
+3 -2
View File
@@ -112,8 +112,9 @@ To maintain absolute traceability, we link memory items directly to the exact gi
## 15. Neovim Integration & God Mode
The project contains two MCP binaries (win-nvim and linux-nvim) that bridge JSON-RPC over stdio directly to the active Neovim instance (using `active_nvim.txt` for Last Focused Wins telemetry). If no interactive instance is open, they will automatically spawn and connect to a persistent headless Neovim background instance. **Note on Visual Engagement:** While the headless instance is perfect for autonomous/background tasks, if the task involves collaborative "pair programming" or requires the user's visual review, you should pause and request the user to open a Neovim UI before proceeding.
- These binaries expose basic tools (`nvim_get_cursor`, `nvim_get_active_buffer`, `nvim_list_buffers`, etc.).
The project contains two MCP binaries (win-nvim and linux-nvim) that bridge JSON-RPC over stdio directly to the active Neovim instance.
- **Interactive UI (UDP):** The active editor connects to the backend via UDP (port 3002). The UI script (`gemini-ui.lua`) supports non-destructive **Ghost Text Diff Reviews** and native `vim.ui` prompts without spawning subprocesses. You can trigger these interactively using the backend server.
- **Tools:** These binaries expose basic tools (`nvim_get_cursor`, `nvim_get_active_buffer`, `nvim_list_buffers`, etc.).
- **God Mode**: They also expose `nvim_execute_lua`. This is the ultimate fallback tool. If you need to access *any* Neovim API that does not have a dedicated Rust tool (e.g., getting LSP diagnostics, evaluating a visual selection block based on modes, setting registers), you MUST write a short Lua script and pass it to `nvim_execute_lua`. Do not attempt to recompile the Rust server to add new basic tools; use the Lua escape hatch dynamically.
## 16. Tool Schema Discovery (Lazy Loading)
+8
View File
@@ -9,6 +9,14 @@ pub struct NvimTelemetry {
pub file: Option<String>,
pub line: Option<i64>,
pub col: Option<i64>,
// New interactive fields
pub prompt: Option<String>,
pub response: Option<String>,
pub index: Option<i64>,
pub request_id: Option<String>,
pub diff_id: Option<String>,
pub start_row: Option<i64>,
pub end_row: Option<i64>,
}
pub async fn nvim_telemetry_handler(
+11
View File
@@ -247,6 +247,17 @@ async fn run_server(state: Arc<MemoryState>) -> Result<(), Box<dyn std::error::E
};
let _ = nvim_udp_state.handler.state.event_bus_tx.send(event);
}
// 4. Interactive Agent UI Events
if payload.event.starts_with("agent_") {
let topic = format!("nvim:ui:{}", payload.event);
let event = crate::state::GenericEvent {
topic,
session_id: Some(payload.session_id.clone()),
payload: serde_json::json!(&payload),
};
let _ = nvim_udp_state.handler.state.event_bus_tx.send(event);
}
}
}
}