Campus RAG Assistant

Campus RAG Assistant is a source-reviewable AI platform for governed campus knowledge. It combines a cited-answer RAG path with a HITL-gated helpdesk escalation loop: when the knowledge base cannot resolve a question, the system can retry retrieval, use controlled web research, search GitHub issues for duplicates, draft a ticket, and file to GitHub only after human confirmation. The system runs behind one FastAPI backend and Vue 3 SPA with AWS / Azure / mock providers, RAGAS evaluation, LangSmith and Prometheus observability, CI/security gates, redaction, and responsible-AI guardrails.

Review model

Review it as an engineering artifact: source code, architecture, screenshots, evaluation results, observability, CI/CD, security posture, and release hygiene. It is not presented as a hosted public product.

View source on GitHub ->

Start here

Goal	Start here
90-second overview	Reviewer Guide
Ownership and product judgment	Case Study
System design	Architecture + Design Notes
RAG quality	Evaluation + Baseline
Agentic orchestration	Helpdesk overview + ADR-005 + ADR-006 + ADR-007
Operations and security	Operations Manual

What this shows

Capability	What it shows	Evidence
Cited RAG path	LangGraph retrieval stages, KB-first answers, multi-query retrieval, rerank hooks, source contracts, and opt-in web research	DESIGN.md · EVALUATION.md · ADR-001 · ADR-003
Helpdesk escalation loop	Bounded multi-turn escalation with KB retry, web research, GitHub duplicate search, GitHub ticket drafting/filing, clarifying turns, redaction, HITL confirmation, and four explicit outcomes; ADR-006 tracks the LangGraph supervisor migration	Helpdesk overview · ADR-005 · ADR-006
AI platform architecture	One FastAPI + Vue product surface over AWS / Azure / mock providers, tenant configuration, feature flags, migrations, and CI-safe local mode	ARCHITECTURE.md · ADR-001
Evaluation, observability, and responsible AI	RAGAS baseline, LangSmith traces, Prometheus metrics, k6 load profiles, gitleaks, protected branches, redaction, and human approval before side effects	eval_baseline_v2.md · operations-manual/index.md · ADR-004

Architecture

flowchart LR
  VueSPA["Vue 3 SPA"] --> FastAPI["FastAPI"]
  FastAPI --> LangGraph["LangGraph RAG"]
  FastAPI --> HelpdeskAgent["Helpdesk Agent"]
  HelpdeskAgent --> AgentTools["KB retry / web / GitHub"]
  HelpdeskAgent --> GHIssues["GitHub Issues HITL"]
  LangGraph --> Providers["Provider Registry"]
  Providers --> BedrockKB["AWS Bedrock KB"]
  Providers --> AzureAI["Azure AI Search"]
  FastAPI --> Postgres[("Postgres")]
  FastAPI --> Prometheus["Prometheus"]
  LangGraph --> LangSmith["LangSmith"]
  HelpdeskAgent --> LangSmith
  LangGraph --> RAGAS["RAGAS eval"]

Design detail: Architecture and Design Notes.

Screenshots

Knowledge-base answer

Source transparency

Opt-in web research

LangSmith trace

Helpdesk agent (v3)

More assets: screenshots catalog.

Quality baseline

The project includes a RAGAS golden-set harness and a documented v2 retrieval baseline. Phase 5 retrieval tuning improved AWS context recall to 0.800, passing the retrieval coverage gate. Context precision remains the main improvement target.

This is an engineering baseline, not a marketing claim. Strict RAGAS gates are release controls, not blockers for local demo or ordinary PR CI.

Read more: Evaluation approach and baseline scores.

Stack

Layer	Technologies
Backend	FastAPI, SQLAlchemy, Alembic, JWT auth, rate limiting, Prometheus metrics
Frontend	Vue 3, TypeScript, Pinia, Tailwind, Vitest, Playwright
RAG orchestration	LangGraph (RAG_ENGINE=langgraph) or LangChain ConversationalRetrievalChain (RAG_ENGINE=chain)
Retrieval	Bedrock KB / OpenSearch Serverless, Azure AI Search, multi-query + RRF, optional rerank
LLM	AWS Bedrock, Azure OpenAI, or mock provider
Web search	Mock or Tavily behind research_mode=web
Eval	RAGAS golden dataset, tox -e eval, LangSmith traces
CI/CD	GitHub Actions, tox, gitleaks, dependency review, no tool attribution, optional EB deploy

Feature availability

Configuration	What works
No cloud keys (RAG_FORCE_MOCK=true)	Register/login, chat UX, streaming path, source panel, feedback, local tests
AWS Bedrock KB	Managed KB retrieval, Bedrock generation, LangGraph retrieval stages, LangSmith trace capture
Azure OpenAI + AI Search	Azure provider path with vector/keyword/hybrid retrieval and cited answers
Web research enabled	Per-message web mode with disclaimer UI and WEB-labeled sources (mock or Tavily)
OAuth configured	GitHub OAuth handoff to Vue; Google-ready provider config
Eval keys available	RAGAS golden-set runs, release quality gates, LangSmith trace inspection

Getting started

python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# set RAG_FORCE_MOCK=true, LLM_PROVIDER=mock, RETRIEVER_PROVIDER=mock
docker compose --env-file /dev/null up -d db
alembic upgrade head
PIP_SYNC=0 ./scripts/run-backend-venv.sh          # http://127.0.0.1:8000
./scripts/run-frontend-vue.sh          # http://127.0.0.1:5173

Register a user and start a chat. Responses use the mock provider unless you configure live AWS/Azure providers. The backend runner starts and health-checks the Compose db service by default; set SKIP_DOCKER_DB=1 only when using an existing Homebrew/Postgres service. If another Postgres already owns port 5432, stop it before using the Compose database.

Origin and Scope

This repository builds from the public ets-berkeley-edu/chabot codebase and substantially extends it as an independent portfolio and educational project. The work here focuses on the AI platform surface: Vue product UI, provider abstraction, LangGraph orchestration, RAGAS evaluation, LangSmith observability, CI/CD, load testing, and operational documentation. It is not an official UC Berkeley or UC product.

See Notice for attribution details.

License

Software in this repository is licensed under the Regents of the University of California terms. See Notice for attribution details. Commercial use requires an agreement with UC OTL.