Meeting Intelligence Chatbot
A web chat assistant that syncs meeting transcripts from a shared Google Drive folder, answers questions about them with hybrid-search RAG, and drafts delivery documents (Executive Summary, HLRD, User Stories, Timeline Matrix) through multi-stage LLM pipelines. It is a live project, deployed through a GitHub Actions workflow.
From a meeting to a grounded answer
- 1. Trigger
Transcript lands in Drive
Meeting-recorder JSON, PDF, DOCX, XLSX, PPTX
- 2. Input
Clean & chunk
Filler words removed, speaker-aware chunks
- 3. Data store
Embed & index
MiniLM vectors → FAISS, plus BM25
- 4. Trigger
User asks in chat
Follow-ups keep context through slots
- 5. AI
Hybrid search
Meaning + keywords, fused with RRF
- 6. Output
Grounded answer or document
Groq models, output validated
Overview
The Meeting Intelligence Chatbot is a web chat assistant for teams whose meeting transcripts land in a shared Google Drive folder. People ask it about meetings, decisions, action items and project status, and it answers from the transcripts themselves rather than from the model's general knowledge. It also drafts four delivery documents for a chosen project and time period: an Executive Summary, a High-Level Requirements Document (HLRD), User Stories with acceptance criteria, and a Timeline Matrix of phases, owners, dependencies and risks. Five built-in role personas (Product Owner, Delivery Head, Business Analyst, Scrum Master and General User) guide people who aren't sure which document they need: the bot asks for the project, then the document, then the time period. Any answer can be copied or exported as a branded PDF.
Problem
Meetings already produce plenty of text. Google Meet summaries and meeting notes, and Fireflies recordings, turn every call into a transcript with little effort. The problem is what happens next: the transcripts pile up and sit there. A project's history ends up spread across 90 to 120 days of meetings, and nobody rereads them all. So when a project needs an executive summary or a requirements document, someone has to dig through every meeting about it, pull out the decisions, needs and actions by hand, and write the document from scratch, in a different shape each time. The same business need often comes up in several meetings and changes a little each time, so no single meeting tells the whole story. I wanted the meetings for a project to become the source for its documents directly, in the same structure every time, and for anyone to be able to ask about a project in plain language and get an answer drawn from all of its meetings.
Context
Transcripts arrive as JSON exports from a meeting-recorder tool, each with a title, the transcript lines, an overview, notes and action items. The same folder can also hold PDF, DOCX, XLSX, PPTX, CSV, TXT and Markdown files, and all of them are ingested. Three constraints shaped the build. All LLM calls go through the Groq API, and its rate limits drove many design choices. Embeddings run on CPU, so the embedding model is slow to load. And there is no database: the vector index, its metadata and the Drive sync state are files on disk, and sessions and caches live in memory. The staging server ran Python 3.10 while local development used 3.12.
Solution
I built a FastAPI backend with a separate React chat app around one idea: a prompt library that templates each document completely, so every Executive Summary, HLRD, User Story set and Timeline Matrix comes out in the same structure. Four engineering decisions carry most of the weight. Hybrid retrieval. Every question is searched two ways at once: by meaning, with all-MiniLM-L6-v2 embeddings in a FAISS index, and by exact words, with a custom BM25 index written in pure Python so no extra library is needed. The two ranked lists are merged with Reciprocal Rank Fusion (k=60), weighted 0.7 to vectors and 0.3 to BM25. A config flag keeps vector-only search available as an alternative. Model tiers on Groq. Every model call goes through Groq's LLM APIs, split across three tiers. Small, frequent decisions (intent, slot extraction, query rewriting) use the fast llama-3.1-8b-instant model. Answers use qwen/qwen3-32b at temperature 0.2, which the code notes is there to prevent hallucination. Documents use llama-3.3-70b-versatile, chosen in the code for better structured JSON output. A provider layer falls back to a local Ollama model when Groq fails, though only the Q&A and unified-extraction paths use it so far. Documents in three modes. The fast mode makes one LLM call that extracts requirements, users, decisions, actions, risks and timeline into a single JSON, then formats it into whichever document was asked for. The two-stage mode runs an extraction prompt and then a synthesis prompt per document. The chained mode runs Executive Summary, then HLRD, then User Stories, then Timeline Matrix, passing a compressed hand-off of about 800 tokens from each to the next. Fast mode became the default because, in the code's words, it is faster and avoids rate limits. The prompts live in JSON files with their stage, temperature and token settings. Multi-turn context through slots. Each conversation keeps a small set of typed slots (project, time range, speaker and others). Four prompts run per turn: extract the intent and slots, merge them with the previous turn's slots, rebuild a standalone search query, then answer. Each turn is labelled as new, a follow-up, a refinement, a topic switch or a clarification, and that label decides which slots survive, so a short follow-up like "only last week" still makes sense. Around the models sit validators: spell-correction with protected domain terms and query reformulation on the way in, and JSON repair and empty-answer checks on the way out.
Hybrid retrieval, explained
THE CONCEPT
Hybrid search with Reciprocal Rank Fusion
THINK OF IT LIKE
Two interviewers shortlisting candidates
Hybrid search with Reciprocal Rank Fusion is like Two interviewers shortlisting candidates
- Vector search ranks by meaningis like One interviewer judges overall fit
- BM25 ranks by exact wordsis like The other checks the CV for required skills
- RRF scores each result by its rank in both listsis like The panel combines positions, not raw marks
- Passages both searches like rise to the topis like Candidates both interviewers rank highly get shortlisted
Three ways to draft a document
flowchart TD
R([Document requested]) --> T[Gather project transcript chunks]
T --> M{Pipeline mode?}
M -- fast --> U[One unified extraction call]
U --> F[Format into target document]
M -- two-stage --> S1[Stage 1: extract facts]
S1 --> S2[Stage 2: synthesise document]
S2 --> J[Parse & repair JSON]
J --> F
M -- chained --> ES[Executive Summary]
ES -- hand-off --> HL[HLRD]
HL -- hand-off --> US[User Stories]
US -- hand-off --> TM[Timeline Matrix]
TM --> F
F --> O([Markdown document])Slot state, explained
THE CONCEPT
Slot state across conversation turns
THINK OF IT LIKE
A waiter's order pad
Slot state across conversation turns is like A waiter's order pad
- Slots: project, time range, speakeris like Fields on the order pad: dish, size, table
- FOLLOW_UP keeps every slotis like "Tell me more about that dish"
- REFINEMENT replaces one slotis like "Make that a small one"
- SWITCH clears the slotsis like A new order on a fresh page
- Standalone query rebuilt from slotsis like The waiter reads the full order to the kitchen
My Role
I developed the full application myself, end to end. That covered the ingestion pipeline (the Google Drive incremental sync, the file parsers, transcript cleaning and speaker-aware chunking); the hybrid retrieval layer (embeddings, the FAISS index, the custom BM25 index and rank fusion); the conversation engine (the chat router, LLM intent detection, the four-prompt slot-state flow, the personas and the clarification, feedback and retry sessions); the prompt library and the document pipelines in their fast, two-stage and chained modes; the LLM provider layer with its fallback and retries; the input and output validators; the React chat app with option chips and PDF export; and the deployment with GitHub Actions, PM2 and Nginx.
Architecture
Frontend: a React 18, TypeScript and Vite single-page app styled with Tailwind. It renders the bot's Markdown and tables, turns the bot's numbered lists into clickable option chips, and exports answers to PDF in the browser with html2canvas and jsPDF. Its API wrapper waits up to 300 seconds per request and retries twice on network errors. Backend: FastAPI behind an Nginx TLS reverse proxy, run by PM2. The live app defines about 50 endpoints, including chat, document generation, Drive sync and persona routes. A performance middleware stamps each request with an ID and a timer. Search layer: the FAISS vector index and the BM25 keyword index, fused with RRF and narrowed by metadata filters such as project and date. Ingestion: a Google Drive service syncs the shared folder with a service account, a file processor parses each format, and a transcript processor cleans filler words and chunks meetings by speaker. LLM layer: a provider factory with Groq as primary and Ollama as fallback, exponential-backoff retries and task-based model choice. Data layer: files only. vector_store.faiss, metadata.pkl and gdrive_sync_state.json sit on a persistent volume; conversation sessions and caches are in memory, with sessions expiring after 60 minutes. There are no agents in the current build: no tool calling and no agent loop. Every step is fixed in code.
How the pieces connect
flowchart TD
subgraph Browser
UI[React chat app]
end
subgraph Server
NGX[Nginx TLS proxy] --> API[FastAPI /chat and endpoints]
API --> CONV[Conversation layer]
API --> DOC[Document pipelines]
API --> ING[Ingestion]
CONV --> SRCH[Hybrid search]
CONV --> LLM[LLM layer]
DOC --> LLM
ING --> SRCH
end
UI --> NGX
SRCH --> IDX[(FAISS + BM25 index)]
ING --> SYNC[(Drive sync state)]
ING --> GD[Google Drive folder]
LLM --> GROQ[Groq API]
LLM -. fallback .-> OLL[Ollama local]The retrieval pipeline and its settings
- 1. Trigger
Sync recent files
Only new or updated files
90-day window - 2. Input
Chunk
Meetings vs other documents
800/150 and 600/100 words - 3. AI
Embed
all-MiniLM-L6-v2 on CPU
384-d, batches of 32 - 4. Data store
Hybrid search
FAISS + BM25, fused with RRF
0.7 / 0.3, k=60 - 5. Output
Top passages to the model
Minimum similarity 0.25
top-15
Workflow
A typical question, such as "What did the team decide about X last week?", takes this path. The browser sends the message and the session ID to POST /chat. If the message is a greeting, the bot clears the session, runs a silent incremental Drive sync and replies with the project list and persona options. Otherwise the input validator either reformulates the query or asks for clarification. Next the router checks for an active session (feedback, clarification, thank-you, retry or an unfinished intent) and lets it handle the message. If none is active, an LLM intent analysis decides between generating a document, asking the user questions, or answering directly. For a direct answer, the conversation orchestrator rebuilds a standalone query from the merged slots. Hybrid search, filtered by those slots, returns the most relevant chunks. The answer model writes from those chunks only, the reasoning tags are stripped, the output is validated, and the response comes back with citations and per-step timings. The frontend reveals it with a typewriter effect and shows the copy and PDF buttons.
How /chat decides what to do
flowchart TD
A([Message + session ID]) --> G{Greeting?}
G -- yes --> S[Silent Drive sync + project list]
G -- no --> V{Input valid?}
V -- unclear --> C[Ask for clarification]
V -- yes --> AS{Active session?}
AS -- yes --> H[Session handles reply]
AS -- no --> I{LLM intent}
I -- document --> D[Document pipeline]
I -- ask questions --> Q[Ask for missing details]
I -- direct answer --> O[Rebuild query from slots]
O --> HS[Hybrid search]
HS --> L[Answer from chunks]
L --> OV[Validate output]
OV --> Z([Answer + citations])AI Models
Challenges
Groq rate limits. This was the constraint that shaped the most code. Retries were raised to 5 with exponential backoff, a 15-second pause was added between pipeline stages, the chained HLRD step falls back to single-stage generation to avoid rate limits, and the one-call fast mode became the default. Reasoning tags leaking into answers. Qwen returns its thinking inside <think> tags, and they appeared in user-facing text. A regex now strips them in four places. Malformed JSON from the model. Document parsing runs in layers: parse directly, repair, extract the JSON block, then repair again. The output validator also fixes common JSON mistakes. Context overflow and noisy citations. The number of chunks is capped, token limits are applied before each call, and the per-chunk cap was raised to 4000 characters to avoid truncation. Context is formatted without source numbers so the model stops citing them. A slow first request. The embedding model loads in a background thread at startup, and the app waits for it before going live, so the first user doesn't pay for it. Long requests timing out. The frontend timeout was raised to 5 minutes, and Nginx's read timeout set to 300s to match. Misrouted intents. "Timeline matrix" was being routed to User Stories, and project names were being read as meeting types. The slot prompt now carries explicit rules and a list of known project names. Deployment details. f-strings with nested quotes broke on the Python 3.10 staging server, and Nginx and FastAPI were both adding CORS headers; only FastAPI handles CORS now.
Retrying through Groq rate limits
- 1. AI
Call Groq
- 2. Decision
Rate limited or failed?
- 3. Action
Wait, doubling the delay
retry_delay × 2^attempt
Back to step 1 — until success or 5 retries
Results
The assistant is live, deployed through a GitHub Actions workflow to a PM2 and Nginx server. It answers questions across meetings with hybrid retrieval, keeps context across turns through slots, generates all four delivery documents in both the fast and two-stage modes, guides users through five personas, and exports answers to PDF. The backend has 44 pytest test functions across 4 files: configuration (7), exceptions (11), Pydantic schemas (20) and API integration (6). The CI workflow deploys but does not run them. The code records timings for every step of every request (intent analysis, slot orchestration, vector search and the total), but no measured latency, accuracy, cost or usage figures were saved, so none are reported here.
Learnings
Rate limits, not model quality, drove the architecture. The fast mode as default, the pauses between stages and the single-stage fallbacks all came from Groq's limits. An abstraction only pays off where it's applied. The Ollama fallback protects Q&A, but the document services call Groq directly and have no fallback. Keeping prompts in JSON files made them versionable, but the live executive-summary prompts drifted into inline copies while three JSON prompt files went unused. Docs drifted too: chunk sizes and retry counts in the diagrams no longer match the config. A migration started is not a migration finished. A modular router app exists next to the roughly 5,100-line main.py, but most of its routes still return 501.
Future Scope
The current assistant is a hand-coded workflow that calls LLMs, not an agentic system: the code decides every step. The next phase I've designed (none of it built yet) moves the conversation and document layers onto LangGraph as a single orchestrator. One typed conversation state per session, saved by a checkpointer on Postgres or Redis, would replace the five in-memory session stores, survive restarts and stop sessions being shared between users. The roughly 2,100-line if/else router would become one structured LLM routing decision plus explicit graph edges, and clarifying and persona questions would use LangGraph's built-in pause-and-resume. Q&A would become a tool-using agent over the existing hybrid search, able to search again or combine meetings, with a cap on tool calls to protect latency and rate limits. The document chain would gain a reviewer after each stage that checks the schema and cross-references (every requirement covered by a story, every story scheduled) and regenerates at most twice. Ingestion, retrieval, formatting and validation would stay as plain, tested code that the graph calls. CrewAI was ruled out because the personas are guided question flows, not autonomous agents; LlamaIndex because the hybrid search already works; and stacking several frameworks because it would split state management. The rollout is planned in phases behind a feature flag, starting with tracing and an evaluation set of about 50 real questions on the current app, so the new graph can be compared with the old router before it is removed. The frontend's API contract stays the same throughout.
Proposed: one graph instead of the router
flowchart TD
START([New message]) --> VIN[Validate input]
VIN --> RT{Route: intent + slots}
RT -- greeting --> GR[Greet]
RT -- qa --> QA[Q&A agent with tools]
RT -- document --> SL{Slots complete?}
SL -- no --> COL[Pause and ask user]
COL --> SL
SL -- yes --> DOC[Document subgraph]
RT -- persona --> PER[Persona questions]
RT -- unclear --> CL[Ask one question]
CL --> RT
GR --> OUT[Validate output]
QA --> OUT
DOC --> OUT
PER --> OUT
OUT --> END([Response])Proposed: a reviewer after each document stage
- 1. AI
Generate stage
e.g. User Stories from the HLRD
- 2. Action
Deterministic checks
Schema, counts, ID references
- 3. Decision
Review passes?
Every requirement covered?
Back to step 1 — on failure, at most twice
Proposed migration roadmap
- 1. Action
Baseline
Tracing + evaluation set on today's app
- 2. Data store
One saved state
Checkpointer replaces 5 session stores
- 3. Action
Router → graph
Route node + conditional edges
- 4. AI
Q&A agent
Hybrid search as tools
- 5. AI
Document subgraph
Reviewer loop, cache-aware nodes
- 6. Output
Cleanup
Remove legacy router, sync as a job
Confidentiality note: Built as employer work. The company, its product name, client project names, domains, servers and the Drive folder are withheld; the system is called the Meeting Intelligence Chatbot throughout. Prompts are described rather than reproduced, except the meeting Q&A prompt, which has its own entry in this series.