{"name":"eval-case-generate","description":"Eval case generator for an agent or LLM task: describe the task and get back a draft llm eval dataset of {input, expected, rubric} cases spanning typical, edge, and adversarial scenarios. Pass example cases to steer style, and set n for how many to generate (default 8, max 20). Every case is labeled a candidate needing human review, not validated ground truth: this drafts a first pass at agent test cases, it doesn't certify them. Use it as an agent regression tests starter, an eval dataset generator for a new task, or a quick way to seed a rubric-graded benchmark before a human tightens it.","price":"$0.02","method":"POST","input_schema":{"properties":{"task":{"type":"string","description":"The task or agent behavior to generate eval cases for, e.g. 'a customer-support agent that answers refund policy questions'. Required, max 4,000 chars."},"examples":{"type":["array","object","string"],"description":"Optional example case(s) to steer style/format. JSON-serializable, capped at 20,000 chars when serialized."},"n":{"type":"number","description":"How many eval cases to generate. Integer, 1-20. Default 8."}},"required":["task"]},"input_example":{"task":"Evaluate a trivia-answering agent for factual accuracy","examples":[{"input":"What is 2+2?","expected":"4","rubric":"Response must state the answer is 4."}],"n":5},"output_example":{"task":"Evaluate a trivia-answering agent for factual accuracy","requested_n":5,"returned_n":5,"cases":[{"input":"What is the capital of France?","expected":"Paris","rubric":"Response must name Paris."},{"input":"What year did WW2 end?","expected":"1945","rubric":"Response must state 1945."}],"review_required":true,"note":"These are candidate eval cases generated by an LLM and need human review before use as ground truth.","model":"hermes-3-llama-3.1-405b","source":"morpheus"},"x402_payment_required":true,"x402_facilitator":"https://api.cdp.coinbase.com/platform/v2/x402"}