BuiltObservability ClickHouse events, llm_calls, graph_queries, and retrieval_events tables, all populated, plus a new owner-only reset-metrics route that clears a tenant's test/demo activity from these tables (with a typed-confirmation gate) without touching the knowledge base itself.
BuiltLineage monitoring doc_id, extraction_pass, model_version, and prompt_version columns, populated at write time.
BuiltEvaluation eval/run_eval.py runs a real question set against /api/chat and grades pass/fail.
BuiltFeedback An /api/feedback route plus a ClickHouse feedback table.
In progressPerformance Only covered indirectly via latency/cost metrics — no dedicated performance layer.
BuiltLatency latency_ms measured and logged on every call.
BuiltCost / FinOps Cost columns plus aggregation queries by provider, client, and day — now also tracked at ingestion time, with llm_cost_usd, embedding_cost_usd, total_cost_usd, and llm_call_count written for every document processed, not just every chat question.
BuiltAI Ops A real /ai-ops page plus GET /api/admin/ai-ops shows exactly which model and prompt version is live right now (reading the actual PROVIDER_MODEL/PROMPT_VERSION constants /api/chat uses), plus a day-by-day history aggregated from the events table, so a quality shift can be traced to a model change, a prompt change, or neither.
BuiltPrompt / version mgmt A PROMPT_VERSION constant, logged per event.
BuiltGroundedness / hallucination eval A real entailment check (score_groundedness) runs on every request in production, with both a groundedness and a derived hallucination score surfaced in Observability.
BuiltRed-teaming / safety testing The Tests tab runs 8 automated adversarial checks on demand against the live agent, including 3 jailbreak styles (direct override, role-play persona, fictional-story framing), prompt-injection resistance, and a cross-tenant-leakage attempt.
BuiltData observability (freshness, drift) The data quality check now flags any document untouched since ingestion for longer than STALE_DOCUMENT_DAYS (180 by default), a real automated freshness signal on top of the existing prompt-level "prefer latest record" instruction.
BuiltModel & embedding versioning EMBEDDING_MODEL_VERSION is actually written into the chunks table at ingest.