Summary
The evaluator feature is being removed from the Ark core repository (see mckinsey/agents-at-scale-ark PR for the removal). The evaluator service needs to be migrated into the marketplace as a pluggable module.
What needs migrating
- Evaluator service (
services/ark-evaluator/) — Python service that runs evaluations against query results
- CRDs:
Evaluation and Evaluator custom resource definitions
- Controllers: evaluation and evaluator reconciliation controllers
- Webhooks: admission webhooks for evaluation/evaluator resources
- API endpoints:
/evaluations and /evaluators REST API routes
- CLI commands:
ark evaluation command group
- Dashboard pages: evaluation listing, detail, editor pages and components
- Integration tests: 8 chainsaw test suites
- Documentation: developer guide, reference docs, samples
- Samples: evaluation demos and examples
Acceptance criteria
Summary
The evaluator feature is being removed from the Ark core repository (see mckinsey/agents-at-scale-ark PR for the removal). The evaluator service needs to be migrated into the marketplace as a pluggable module.
What needs migrating
services/ark-evaluator/) — Python service that runs evaluations against query resultsEvaluationandEvaluatorcustom resource definitions/evaluationsand/evaluatorsREST API routesark evaluationcommand groupAcceptance criteria