Skip to main content

Architecture Review Exercise

Compare three architectures for the same product requirement and justify your choice.

Scenario​

Product: Customer support system that:

  • Answers questions about orders, products, and policies
  • Can process refunds (requires approval)
  • Can escalate to human agents
  • Handles 10,000 conversations/day
  • Needs to integrate with existing CRM and order systems

Architecture Options​

Option A: Single Agent with Tools​

Option B: Manager + Specialists​

Option C: Decentralized Handoff​

Comparison Template​

1. Architectural Fit​

CriterionOption AOption BOption C
ComplexityLowMediumHigh
LatencyLowestMediumVaries
Context isolationNoneMediumHigh
Failure isolationPoorGoodBest
Tool managementSimpleModerateComplex

2. Cost Analysis​

Cost FactorOption AOption BOption C
LLM calls/conversation1-32-52-4
Avg tokens/conversation200030002500
Estimated daily costCalculateCalculateCalculate
Infrastructure costLowMediumHigh

3. Failure Analysis​

Failure ModeOption AOption BOption C
Agent loops foreverRisk: HighRisk: LowRisk: Medium
Wrong tool selectionRisk: HighRisk: LowRisk: Medium
Context overflowRisk: HighRisk: LowRisk: Medium
Tool timeoutAffects allIsolatedIsolated

4. Evaluation Requirements​

MetricOption AOption BOption C
Eval complexityLowMediumHigh
Test coverage neededBasicComprehensivePer-agent
Monitoring complexityLowMediumHigh

5. Human-in-the-Loop Requirements​

ActionImplementation
Refund approvalHow to implement?
EscalationHow to implement?
Interrupt and resumeHow to implement?

Your Analysis​

Chosen Architecture​

Option: [A / B / C]

Justification​

1. Why this architecture over others?




2. What control boundaries exist?




3. What failure modes do you expect?




4. How would you evaluate before production?




5. Where is human approval or rollback required?




Implementation Sketch​

Sketch key implementation components for your chosen architecture:

Agent Definition​

# Your implementation here
class CustomerSupportAgent:
    def __init__(self):
        # Define agents, tools, and routing
        pass
    
    async def process(self, message: str, thread_id: str) -> str:
        # Implementation
        pass

Approval Flow​

# Your implementation here
async def handle_refund(request: RefundRequest) -> str:
    # Implementation
    pass

Checkpoint Strategy​

# Your implementation here
def checkpoint_strategy() -> dict:
    # What to checkpoint and when
    pass

Evaluation Criteria​

Your analysis will be evaluated on:

  1. Justification quality — Clear reasoning for architecture choice
  2. Tradeoff awareness — Honest acknowledgment of costs and risks
  3. Failure thinking — Realistic failure mode analysis
  4. Human oversight — Appropriate approval and rollback points
  5. Production readiness — Practical evaluation and monitoring plan

Next Steps​

After completing this exercise:

  1. Review your analysis against real production requirements
  2. Prototype the chosen architecture
  3. Run evaluations before committing
  4. Build monitoring and observability