evals

Go Artificial Intelligence (GAI) helps you work with foundational models, large language models, and other AI models.

Branch SQLite like git — copy-on-write forks, checkpoint/rollback/promote, kill-9-durable capture. Stock SQLite files on local disk or S3. Built for AI agents and eval harnesses.

🛠 Manage Claude Code sessions with a TUI featuring real-time monitoring, conversation history, and automatic session detection.

AI release engineering - CI gate for prompts, skills, MCP, and models

The OAIEvals Collector: A robust, Go-based metric collector for EVALS data. Supports Kafka, Elastic, Loki, InfluxDB, TimescaleDB integrations, and containerized deployment with Docker. Streamlines OAI-Evals data management efficiently with a low barrier of entry!

Get started with LLMs, FTS and vector search, RAG, and more, in Go!

A Go library and CLI for evaluating Model Context Protocol (MCP) servers using Claude.

A tiny testable, self hosted, provider neutral agent runtime for teams that want control, conformance, and no framework lock-ins.

Regression testing for Claude Code and Codex agents.

Public append-only log for AI evaluation pre-registration. Seal the metric, threshold and dataset digest before you run; bind results afterwards. The log assigns the attempt number, not the claimant.

Generative Artifact Protocol (GAP) — an open standard for token-efficient artifact updates and streaming. Go apply engine + eval CLI.

A Go CLI for benchmarking LLMs against YAML-defined prompt suites.

Deterministic evals for agent skills

Executable documentation for Kno: real scenarios, committed expectations, nightly drift detection.

Regression-test your AI agent configuration - know if that CLAUDE.md edit made your agent better or worse

A small LLM eval harness in Go: typed rubrics with blocking vs advisory criteria, LLM-as-judge grading, N-run repetition with a failure taxonomy, and a deterministic fake client for tests

CI-gated eval harness for any AI system. Golden datasets in, pass or fail out, with per-case tracing. One Go binary, no SDK.

Iterative LLM evaluation pipeline in Go example

Detect your server's IP geolocation and measure direct network reachability with real-time diagnostics.

Go agent harness: process-jail tool-use, hermetic evals, multi-tenant daemon with pluggable auth.