Reviewer Guide¶
Campus RAG Assistant is a source-reviewable AI platform for governed campus knowledge. It combines a cited-answer RAG path with a HITL-gated helpdesk escalation loop: when the knowledge base cannot resolve a question, the system can retry retrieval, use controlled web research, search GitHub issues for duplicates, draft a ticket, and file to GitHub only after human confirmation. The system runs behind one FastAPI backend and Vue 3 SPA with AWS / Azure / mock providers, RAGAS evaluation, LangSmith and Prometheus observability, CI/security gates, redaction, and responsible-AI guardrails.
Review it as an engineering artifact: source code, architecture, screenshots, evaluation results, observability, CI/CD, security posture, and release hygiene. It is not presented as a hosted public product.
What this shows¶
| Capability | What it shows | Evidence |
|---|---|---|
| Cited RAG path | LangGraph retrieval stages, KB-first answers, multi-query retrieval, rerank hooks, source contracts, and opt-in web research | DESIGN.md · EVALUATION.md · ADR-001 · ADR-003 |
| Helpdesk escalation loop | Bounded multi-turn escalation with KB retry, web research, GitHub duplicate search, GitHub ticket drafting/filing, clarifying turns, redaction, HITL confirmation, and four explicit outcomes; ADR-006 tracks the LangGraph supervisor migration | Helpdesk overview · ADR-005 · ADR-006 |
| AI platform architecture | One FastAPI + Vue product surface over AWS / Azure / mock providers, tenant configuration, feature flags, migrations, and CI-safe local mode | ARCHITECTURE.md · ADR-001 |
| Evaluation, observability, and responsible AI | RAGAS baseline, LangSmith traces, Prometheus metrics, k6 load profiles, gitleaks, protected branches, redaction, and human approval before side effects | eval_baseline_v2.md · operations-manual/index.md · ADR-004 |
Suggested review paths¶
| Reviewer | Start here |
|---|---|
| Hiring manager | Case Study |
| Staff / principal engineer | Architecture, Design Notes, ADRs |
| AI engineer | Evaluation, LangGraph KB path, Provider registry ADR |
| Platform / DevOps reviewer | Operations Manual, CI/CD, Security |
| Product reviewer | Screenshots, Case Study, Product Roadmap |
| Agent / orchestration reviewer | Helpdesk overview, Conversation Flow, Engineering Spec, Agentic Rebuild plan |
What this repository implements¶
- Vue 3 product UI with sessions, streaming chat, cited sources, feedback, OAuth handoff, Ask/Agent mode.
- FastAPI backend with JWT cookies, SSE endpoints, Alembic migrations, request IDs, Prometheus metrics.
- Provider registry for AWS, Azure, and mock execution modes.
- LangGraph RAG pipeline (
condense → multi_query → retrieve → rerank → generate → format) plus chain path for true token streaming. - Tenant-hydrated prompt and topic configuration in Postgres.
- RAGAS evaluation harness with a documented v2 retrieval baseline.
- LangSmith trace capture for KB and web-research paths.
- Opt-in web research with disclaimer UI and
source_kind=webmetadata. - Helpdesk escalation: ASK-mode one-shot endpoints plus multi-turn AGENT-mode endpoints with HITL ticket filing.
- CI/CD, gitleaks, dependency review, no tool attribution guard, k6 load testing, release docs, and production hardening backlog.
What this is not¶
- Not an official UC Berkeley or UC product.
- Not a public hosted SaaS product.
- Not a production deployment claim.
- Not a generic chatbot demo.
- Not a claim that the current evaluation set is production-sufficient.
- Not a claim that the current helpdesk supervisor is LLM-driven; that migration is tracked in ADR-006.
See Notice and License for attribution and licensing details.