Forgejudge

Name: Forgejudge
Author: ahmedEid1

by ahmedEid1communityv0.1.1

ForgeJudge is an MCP server and related toolkit for evaluating autonomous coding agents with a deterministic sandbox harness, public traces, and a multi-seed CI regression gate. It loads a golden task set, runs solve/grade workflows, and exposes a leaderboard and self-test flow for the harness.

From the registry: Open eval leaderboard + CI gate for autonomous coding agents (solve, score, trace).

GitHub Repository Website

Install & configure

Prerequisites

required
Install uv
Install uv, which is used for the zero-install MCP launch path.
```
$ curl -LsSf https://astral.sh/uv/install.sh | sh
```
https://docs.astral.sh/uv/
optional
Install forgejudge MCP extra
The MCP server requires the package with the [mcp] extra if not using the uvx zero-install path.
```
$ pip install "forgejudge[mcp]"
```

Install in your client

Paste into the AI you're chatting with — it will figure out the rest

Please install the `forgejudge` MCP server into my current AI client (that's you).

Required prerequisites (do these first if not already done):
- **Install uv** — Install uv, which is used for the zero-install MCP launch path. Run: `curl -LsSf https://astral.sh/uv/install.sh | sh` (https://docs.astral.sh/uv/)

Optional prerequisites:
- Install forgejudge MCP extra — The MCP server requires the package with the [mcp] extra if not using the uvx zero-install path. Run: `pip install "forgejudge[mcp]"`

Canonical MCP server config (stdio transport):
- command: `uvx`
- args: ["--from","forgejudge[mcp]","forgejudge","mcp"]

Note: The README presents both a local install path and a zero-install MCP path. The simplest portable MCP configuration is the zero-install stdio launcher: `uvx --from "forgejudge[mcp]" forgejudge mcp`. No MCP-specific environment variables are documented.

Add this MCP server to my current client's config in the correct format for you. If you need secrets or credentials I haven't provided, ASK me — do not invent values or leave raw placeholders. After adding it, tell me how to verify the server is connected.

Note: The README presents both a local install path and a zero-install MCP path. The simplest portable MCP configuration is the zero-install stdio launcher: `uvx --from "forgejudge[mcp]" forgejudge mcp`. No MCP-specific environment variables are documented.