How I Built a Multi-Agent (11 agents) System to Keep Requirements, Docs, and Sprints in Sync

I built an AI-powered system that detects when your sprint board disagrees with your requirements docs — and fixes it automatically.

After months of building, I’m excited to share Project Sentinel: a multi-agent AI system that tackles one of the most painful (and mostly invisible) problems in software delivery — requirement drift.

The Problem

Requirements drift silently. A decision gets made in a Monday standup. It never makes it into the SRS. The sprint board keeps stale backlog items nobody notices. By the time QA catches the mismatch, it’s already cost days of rework.

Sound familiar?

I built Project Sentinel to make requirement drift visible, traceable, and fixable — with AI doing the heavy lifting and humans staying in control.

What It Does

Project Sentinel is a human-in-the-loop, AI-driven requirement governance system that continuously compares three sources of truth:

• Meeting decisions — informal decisions captured in meeting summaries
• Project documents — your SRS and an auto-generated BRD
• Azure DevOps sprint board — your actual in-flight tasks

When it detects a conflict, it doesn’t just flag it. It generates precise, actionable change suggestions with severity ratings, routes them to a Business Analyst for review, and applies approved changes across DevOps, your documents, and your audit trail — all in one coordinated workflow.

Why I Built It This Way

The temptation in AI projects is to build one giant prompt and call it done. I went the opposite direction: a specialized multi-agent architecture where each agent has a single, well-defined responsibility.

• Accuracy — smaller, focused context windows mean fewer hallucinations per agent
• Debuggability — when something breaks, you know exactly which agent to look at
• Composability — each agent can be tested, replaced, or upgraded independently

The 11 Agents

• OrchestratorAgent — coordinates the full drift-analysis pipeline
• MeetingAgent — extracts structured decisions from meeting text
• DocumentAgent — hybrid-searches indexed SRS/BRD chunks
• DevOpsAgent — reads active sprint items from Azure DevOps
• ComparisonAgent — finds conflicts across all three sources
• SuggestionAgent — generates precise, severity-rated change proposals
• ActionAgent — executes approved changes (DevOps + Cosmos + SQL + Teams)
• BRDGeneratorAgent — auto-generates BRD markdown from the SRS
• DevOpsSyncAgent — produces Add / Modify / Remove sync suggestions
• SRSUpdaterAgent — merges approved changes back into the SRS
• ChangeRequestAgent — triages informal changes into Path A or Path B

The AI Stack — and Why

• Azure OpenAI (GPT-4o) — the reasoning engine behind every agent decision. Chosen for its balance of reasoning depth and speed on structured tasks like conflict detection and change generation.
• Azure OpenAI (text-embedding-ada-002) — embeds document chunks into vector space so the system finds the right requirement section even when the exact keywords don’t match.
• Azure AI Search (hybrid) — combines vector similarity with BM25 keyword search. Vectors catch meaning, keywords catch exact terminology — together they surface the most relevant chunks for any meeting decision.
• LangChain 0.3 — the orchestration layer for structuring LLM chains, managing prompts, and wiring agents together with far less boilerplate.

How It Flows

React 19 frontend → FastAPI backend → the multi-agent layer (Orchestrator → Meeting → Document → DevOps → Comparison → Suggestion → Action → BRD → Sync → SRS → ChangeRequest) → Azure services: OpenAI (GPT-4o + embeddings), AI Search, Cosmos DB, Azure SQL, Blob Storage, Azure DevOps, Teams, and Key Vault.

Key Design Decisions

• Human-in-the-loop, always. Every suggestion needs explicit BA approval before anything is applied. AI proposes, humans decide. This was non-negotiable.
• Path A vs Path B triage. When a change request comes in: Path A touches requirements → update SRS → re-index → regenerate BRD → re-sync DevOps (full cascade). Path B is task-only → straight to DevOps without touching the SRS.
• Async-first backend. FastAPI + AsyncIO throughout, with asyncio.to_thread wrappers around the synchronous Azure SDK calls, so the API stays responsive even during long pipeline runs.
• Project-scoped multi-tenancy. Every resource is namespaced by project_id (UUID), so multiple projects, teams, and boards run independently on one deployment.

The Stack

• Frontend — React 19, TypeScript, Vite, Tailwind CSS, TanStack Query
• Backend — FastAPI, Python 3.11+, Pydantic, SQLAlchemy
• AI / LLM — Azure OpenAI (GPT-4o + ada-002), LangChain 0.3
• Search — Azure AI Search (hybrid vector + keyword)
• Storage — Azure Cosmos DB, Azure SQL, Azure Blob Storage
• Integrations — Azure DevOps API, Microsoft Teams
• Infrastructure — Azure App Service, Azure Static Web Apps, Azure Key Vault

Project Architecture

Project Source Code

https://github.com/onsaurav/project-sentinel

What I Learned

The hardest part of AI systems isn’t the AI — it’s the workflow design. Deciding what a human must always approve, what the system can infer, and how to cascade changes cleanly across multiple data stores took far more thought than the prompt engineering.

And multi-agent systems live and die by their contracts. The cleaner the interface between agents, the more reliable the whole pipeline becomes.

If requirement drift is a real pain on your team — or you’re building AI-powered developer tooling — I’d love to compare notes. Reach out via the contact form or connect with me on LinkedIn.