Experiment

Fast, Two-Stage or Chained: Generating Documents From Meetings

Find a way to generate four delivery documents (Executive Summary, HLRD, User Stories and Timeline Matrix) from a project's meetings that works within the LLM provider's rate limits.

Problem

Asking a model to write a full requirements document in one go from raw meeting text is a lot to ask. Splitting the work into more calls gives the model smaller jobs, but every extra call counts against the Groq rate limits the system runs on.

Setup

Three implementations of the same four documents sit side by side in the codebase, chosen by a PIPELINE_MODE setting and a use_two_stage flag. They share the same document formatters and run on Groq, with llama-3.3-70b-versatile for documents. The input for each run is a project's meeting chunks: up to 30, joined into one transcript.

Prompt / Workflow

Fast mode makes one unified extraction call that returns everything the four documents need as one JSON, then formats it in code. Two-stage mode makes two calls per document: an extraction prompt that lists every relevant fact, then a synthesis prompt that writes the document from them. Chained mode runs the documents in sequence (Executive Summary, HLRD, User Stories, Timeline Matrix), passing a compressed hand-off of about 800 tokens from each stage to the next.

DIAGRAM

The three modes compared

flowchart TD
  T([Project transcript]) --> M{Mode}
  M -- fast --> U[1 call: unified extraction]
  U --> F[Format in code]
  M -- two-stage --> S1[Call 1: extract facts]
  S1 --> S2[Call 2: write document]
  M -- chained --> E[Executive Summary]
  E --> H[HLRD]
  H --> U2[User Stories]
  U2 --> TM[Timeline Matrix]
  F --> D([Document])
  S2 --> D
  TM --> D

The same transcript can take three paths: one extraction call then formatting, two calls per document, or a chain where each document reads a compressed hand-off from the one before.

Input

A project's meeting transcripts, filtered by project and time range.

Output

The four delivery documents, in fixed templated structures.

What worked

Fast mode worked well enough to become the default. Its one call stays clear of rate limits, and because formatting happens in code, every document keeps the same structure. The chain's compressed hand-offs kept later stages within context: each stage reads a short summary of the previous document, not its full text.

What failed

The multi-call modes hit Groq's rate limits. Keeping them running took retries raised to 5, a 15-second pause between stages, a 15-second initial backoff, and a fallback in the chained HLRD step to single-stage generation. The original executive summary, a single pass driven by a selected template, was replaced by the two-stage version and kept only as a deprecated backup. No side-by-side quality comparison of the three modes was recorded, so this experiment has no quality scores to report.

PROCESS FLOW

How fast mode became the default

1. Action
Three modes built
Fast, two-stage, chained
2. Decision
Multi-call modes rate-limited
Groq API rate limits
3. Action
Retries and pauses added
5 retries, 15s between stages
4. Action
Single-stage fallback
In the chained HLRD step
5. Output
Fast mode is the default
Two-stage kept for detail

As the code records it: the multi-call modes ran into rate limits, the fixes added delays and fallbacks, and the one-call mode became the default.

LOOP

Retrying a rate-limited call

1. 1. AI
  Call the model
3. 2. Decision
  Rate limited?
5. 3. Action
  Wait, doubling the delay
Back to step 1 — until success or 5 retries

A rate-limited call waits and tries again with the delay doubling each time, until it succeeds or reaches the retry limit.

Learning

Rate limits, not model quality, decided the architecture. In the code's own words, simple mode is faster and avoids rate limits. The richer two-stage path is worth its extra calls only where depth matters most.

Decision

Fast mode is the default. Two-stage generation is kept for the detailed executive summary endpoint, and the chained pipeline stays available behind PIPELINE_MODE=chained.

Next Step

The planned redesign adds a reviewer after each stage of the chain, and an evaluation set of about 50 real questions and expected documents. That set is what will finally let the modes be compared on quality, not just on speed and rate limits.