Studio · Open source · HITL AI
akaOSS
Five projects, one thesis, a live research feed. MIT.
The open-source studio for human-in-the-loop AI. Five projects, one thesis, a reproducible research feed, and the HITL Kit component registry — served as one site.
The thesis
Current benchmarks ask “can the model complete this task autonomously?” In deployment, real users want an assistant that respects their authority, preserves their agency, and makes them better over time. Assist-Not-Complete: evaluate AI on whether it assists humans without displacing them, not on whether it can finish the task alone. The argument is made in full in the paper, An AI Measurement Problem, and tested in public in the research feed — every finding is a reproducible experiment run with the kits below.
The projects
Five projects across two tracks. Each lives in its own repo; here they are on this site.
Human-in-the-loop measurement
Human-in-the-loop AI, measured properly. 15 React primitives installable via the shadcn CLI (registry served from this repo) and six @hitl-kit/* npm packages — schemas, gates, and LangGraph / AI-SDK / MCP adapters.
A measurement instrument for multi-step research agents: YAML suites, per-step tool-match auto-scoring, a five-dimension human rubric, deterministic replay. Humans score, not LLMs.
Structured tagging primitives for annotation workflows: per-modality scoping, scope-aware agreement scoring, headless React.
Developer tooling
A Claude Code skill-building framework — compile MDX lessons and Jupyter notebooks into SKILL.md files, or into MCP server scaffolds, that Claude reaches for first.
Live observability, guided skills, and a non-destructive MCP surface for Blender → glTF pipelines. Stdlib Python, no build step.
One site, one source of truth
akaoss.dev ties them together. The projects each live in their own repos; this one holds the HITL Kit registry (the source of truth for the shadcn primitives, served at /r/*.json, with CI failing on drift), the research feed (question → runs against real models → human-scored results → checked-in run JSON → repro link), and the paper. Next.js 16, Tailwind v4, file-based content — no CMS, no database.
Where it comes from
akaOSS is where the human-side-of-applied-AI work becomes reusable: the kits are the measurement instruments, the research feed is the evidence, and the paper is the argument. It's built at Circleheads.