Mcp As A Judge
by OtherVibescommunityv0.3.20
17
9
MCP as a Judge acts as a validation layer between AI coding assistants and LLMs, helping ensure safer and higher-quality code.
From the registry: MCP as a Judge: a behavioral MCP that strengthens AI coding assistants via explicit LLM evaluations
Install & configure
Prerequisites
- requiredDockerDocker must be installed
- requiredMCP client with sampling+elicitationClient must support MCP Sampling and Elicitation features
Install in your client
Paste into the AI you're chatting with — it will figure out the rest
Please install the `mcp-as-a-judge` MCP server into my current AI client (that's you).
Required prerequisites (do these first if not already done):
- **Docker** — Docker must be installed
- **MCP client with sampling+elicitation** — Client must support MCP Sampling and Elicitation features
Canonical MCP server config (stdio transport):
- command: `docker`
- args: ["run","--rm","-i","--pull=always","ghcr.io/othervibes/mcp-as-a-judge:latest"]
- optional environment variables:
- `LLM_API_KEY`: LLM API key (optional with GitHub Copilot + VS Code which provides MCP sampling) (example: `<your-api-key>`)
- `LLM_MODEL_NAME`: LLM model name override (example: `gpt-4o-mini`)
Note: Also installable via uvx (mcp-as-a-judge).
Add this MCP server to my current client's config in the correct format for you. If you need secrets or credentials I haven't provided, ASK me — do not invent values or leave raw placeholders. After adding it, tell me how to verify the server is connected.Note: Also installable via uvx (mcp-as-a-judge).
Tools (8)
set_coding_task
Creates/updates task metadata; classifies task_size; returns next-step workflow guidance.
get_current_coding_task
Recovers the latest task_id and metadata to resume work safely.
judge_coding_plan
Validates plan/design; requires library selection and internal reuse maps; flags risks.
judge_code_change
Reviews unified Git diffs for correctness, reuse, security, and code quality.
judge_testing_implementation
Validates tests using real runner output and optional coverage.
judge_coding_task_completion
Final gate ensuring plan, code, and tests approvals before completion.
raise_missing_requirements
Elicits missing details and decisions to unblock progress.
raise_obstacle
Engages the user on trade‑offs, constraints, and enforced changes.
Environment variables
LLM_API_KEYrequiredRequired for AI assistants without full MCP sampling support.Prerequisites
- • Python 3.13+
Packages
mcp-as-a-judge
pypiv0.3.20transport: stdio
Details
- Language
- Python
- License
- MIT
- Released
- Sep 20, 2025
- Last commit
- Dec 15, 2025
- Registry name
- io.github.OtherVibes/mcp-as-a-judge
Categories
Related Servers
Netdata
Real-time infrastructure monitoring with metrics, logs, alerts, and ML-based anomaly detection.
78.7k
Oh My Posh Validator
Validate oh-my-posh configurations and segment snippets against the official schema.
22.5k
Mcp
Real-time Kubernetes network traffic visibility and API analysis for HTTP, gRPC, Redis, Kafka, DNS.
11.9k