# Network MCP - Project Summary

## Overview
This project is a long-running Network MCP service that merges OPNsense discovery data, Nmap scans, and static inventory into Elasticsearch, then exposes both a minimal web UI and a full MCP JSON-RPC interface for LLM agents. It runs via Docker Compose and is now located at `/var/core/network-mcp`.

## What We Built
- **Collectors**
  - OPNsense collector ingests DHCP/ARP/DNS and overlays inventory targets.
  - Nmap collector performs discovery and port scans.
  - Data lands in Elasticsearch: `network-hosts` (current state) and `network-events-*` (historical events).
- **Inventory merge**
  - Inventory data from `inventory_targets.yml` is merged onto live hosts by IP when a MAC is known (so live MAC-based records carry inventory notes/expected ports).
- **Frontend**
  - Flask UI + JSON API, containerized with Gunicorn and exposed on port `5001` for LAN access.
- **MCP server**
  - JSON-RPC endpoint at `/.well-known/mcp.json` (and `/api/mcp`) supports:
    - `initialize`, `ping`, `tools/list`, `tools/call`
    - `resources/list`, `resources/read`, `resources/templates/list`
  - Tool schemas include titles, descriptions, input/output schemas, and annotations (read-only hints).
  - Resource templates provide snapshot + query access (e.g. `network://hosts?q=...`).
- **Search behavior**
  - Host search is case-insensitive across name/hostname/IP/MAC.
- **Tests**
  - Unit tests for REST and MCP search by hostname/IP/MAC, MCP resource reads, and MCP notifications.

## Key Endpoints
- UI: `http://<host>:5001/`
- REST:
  - `GET /api/hosts` (supports `q`, `source`, `limit`)
  - `GET /api/hosts/<host_id>`
  - `GET /api/events`
  - `GET /api/hosts/<host_id>/events`
  - `GET /api/map`
- MCP JSON-RPC: `POST /.well-known/mcp.json`

## MCP Tools (JSON-RPC)
- `list_hosts` (search by hostname/IP/MAC; case-insensitive)
- `get_host` (optional events)
- `list_events`
- `host_events`
- `network_map`

## MCP Resources
- `resources/list` -> `network://hosts`, `network://map`, `network://events`
- `resources/templates/list` -> query templates such as:
  - `network://hosts{?q,source,limit}`
  - `network://host/{host_id}{?include_events,events_limit}`
  - `network://events{?host_id,type,since,limit}`

## Docker & Repo State
- Repo path: `/var/core/network-mcp`
- `inventory_targets.yml` lives in the repo and is mounted via compose.
- Services run via `docker-compose up -d`.
- Git repo initialized and initial commit created.

## Gotchas / Pitfalls We Hit
- **MCP handshake**: Codex sent `notifications/initialized` without `id` (notification). Returning a response caused the transport to close. Fixed by treating notifications as no-response.
- **Case-sensitive search**: Elasticsearch wildcard on `.keyword` fields was case-sensitive, so `seele` didn’t match `SEELE`. Fixed via `case_insensitive: true` in wildcard queries.
- **Inventory merge duplication**: Initial inventory-only docs were `ip:*` and live docs were `mac:*`, so both existed. Merge now attaches inventory to live MAC records by IP. Legacy `ip:*` docs may remain stale unless cleaned.
- **MCP errors**: Tool errors are now returned as `CallToolResult` with `isError: true` (instead of JSON-RPC errors), so LLMs can see and correct issues.
- **Service move**: Repo moved from `/var/core/ansible/network-mcp` to `/var/core/network-mcp`. Compose mount paths updated.

## Verification Performed
- REST search works for hostname/IP/MAC.
- MCP `initialize`, `tools/list`, `tools/call` work.
- MCP resource list/templates/read work.
- Services verified running via `docker-compose up -d`.

## Future Work Ideas
- **Cleanup**: Add a cleanup job to remove stale `ip:*` docs after successful MAC merge.
- **Resource subscriptions**: Implement `resources/subscribe` if clients need push updates.
- **Auth**: Optional token on the MCP endpoint for shared LAN exposure.
- **More UI**: Add filters/alerts for stale hosts or missing expected ports.
- **Metrics**: Export collector stats to detect scan/ingest failures.
- **Schema mapping**: Improve Elasticsearch mappings for search (e.g., lowercase normalizers for names/hostnames).