Skill Lab — the evaluation layer for agent skills

Judge Skill#

POST/v1/repos/{owner}/{repo}/judge

Runs an LLM-as-judge quality review on a SKILL.md file. Scores the skill across 9 criteria on two axes: Activation Quality and Instruction Quality. Returns per-criterion scores (0-4), axis scores, an overall judge score (0-100), a verdict band, and improvement suggestions. Rate limited to 5 requests per minute.

Path Parameters#

Parameter	Type	Description
owner	string	GitHub username or organization
repo	string	Repository name

Query Parameters#

Parameter	Type	Description
path	string	Path to the skill directory or SKILL.md file

Request Body#

Optional JSON body:

Field	Type	Description
model	string?	LLM model override — supports claude-, gpt-, gemini-*

Response#

Returns criteria (array of 9 objects with id, name, axis, score, reasoning), activation_score, instruction_score, judge_score (0-100), verdict (Poor / Needs work / Good / Excellent, derived from judge_score: ≥90 Excellent, ≥75 Good, ≥50 Needs work, else Poor), suggestions, and usage.

example response
{
"owner": "owner",
"repo": "repo",
"path": "skills/my-skill/SKILL.md",
"criteria": [
  { "id": "trigger_clarity", "name": "Trigger Clarity", "axis": "activation", "score": 3, "reasoning": "..." }
],
"activation_score": 75.0,
"instruction_score": 80.0,
"judge_score": 77.8,
"verdict": "Good",
"suggestions": ["Add explicit trigger examples", "..."],
"usage": { "model": "claude-haiku-4-5-20251001", "input_tokens": 2900, "output_tokens": 1100 }
}

This endpoint calls an LLM and may take 10-30 seconds to respond. Multi-provider: pass a claude-*, gpt-*, or gemini-* model ID to use different providers.

Errors#

Status	Cause
404	Repository or skill path not found
422	LLM generation failed
429	Rate limit exceeded
503	Service not configured (missing API key)
504	Operation timed out

Example#

bash

curl -X POST "https://api.skill-lab.dev/v1/repos/owner/repo/judge?path=my-skill"

javascript

const res = await fetch("https://api.skill-lab.dev/v1/repos/owner/repo/judge?path=my-skill", { method: "POST" });
const data = await res.json();
// data.judge_score

python

import httpx

res = httpx.post("https://api.skill-lab.dev/v1/repos/owner/repo/judge?path=my-skill")
data = res.json()
# data['judge_score']

OptimizeAutomatically improve the skill TriggersGenerate trigger test YAML sklab evaluateRun the judge locally alongside static checks

PreviousTriggers NextHealth