Agent-to-Agent Communication: Protocols, Queues, and Coordination Patterns

Why Agent-to-Agent Communication Matters in Production

When you move from single-agent pilots to multi-agent systems in production, communication becomes your bottleneck. A single AI agent solving one problem is straightforward. But orchestrate five agents handling claims processing, compliance checks, data validation, and customer notification simultaneously—and you need protocols, queues, and coordination patterns that don't collapse under load or lose messages.

Most organisations treat multi-agent systems like they're scaling a monolith: add more agents, hope they talk to each other, ship it. That fails. Hard. We've seen production deployments where agents deadlock waiting for responses, where messages vanish into message queues never to return, where one agent's latency cascades into 10-minute workflows becoming 2-hour nightmares.

The difference between a 90-day production deployment and an 18-month failure is understanding how agents actually communicate, what protocols enforce reliability, and which coordination patterns fit your workload. This article walks through the concrete patterns we use at Brightlume to keep multi-agent systems running reliably at scale.

The Fundamentals: What Agent-to-Agent Communication Actually Is

Agent-to-agent communication is the structured exchange of information and task delegation between autonomous AI agents. It's not agents chatting casually. It's one agent saying to another: "I need you to validate this claim, return a structured result, and I need it in under 2 seconds or I'm timing out."

At its core, agent-to-agent communication solves three problems:

Task delegation: One agent doesn't do everything. A claims processing agent delegates validation to a compliance agent, which delegates fraud detection to a risk agent. Each agent owns its domain.

State coordination: Agents need to know what's already been done. The compliance agent needs to know what the claims agent found. Without coordination, you get duplicate work, contradictory decisions, and audit nightmares.

Failure handling: When an agent times out, crashes, or returns an error, other agents need to know. A cascade of failures silently compounding is worse than a visible failure you can roll back.

Unlike human conversation, agent communication must be deterministic, auditable, and fast. When you're processing 1,000 claims per hour across six agents, you can't afford ambiguity or latency.

Message Queues: The Nervous System of Multi-Agent Systems

Message queues are how agents don't block each other. Instead of Agent A calling Agent B and waiting for a response, Agent A drops a message in a queue saying "I need validation on claim XYZ," and continues. Agent B picks up the message when it's ready, processes it, and drops a response back in a different queue.

This asynchronous pattern is non-negotiable for production systems because it decouples agent availability from agent responsiveness. If Agent B is slow or temporarily down, Agent A doesn't hang. The message sits in the queue until Agent B recovers.

Queue selection matters for your deployment speed. For 90-day production timelines, you want something battle-tested and operationally simple. AWS SQS, RabbitMQ, or Apache Kafka are the standards. Choose based on throughput requirements and your infrastructure.

SQS is simple for low-to-medium throughput (under 10,000 messages per second). It's managed, requires minimal operational overhead, and integrates cleanly with AWS Lambda-based agents. The tradeoff: you can't easily replay messages or guarantee strict ordering across partitions.

Kafka is your choice if you need high throughput, strict ordering, or the ability to replay messages. It's more operationally complex—you're managing brokers, partitions, replication—but it gives you the observability and replay capability that matters when something goes wrong in production and you need to understand exactly what happened.

RabbitMQ sits in the middle. It's flexible, supports multiple messaging patterns (queues, topics, routing), and runs on-premise or cloud. It's heavier than SQS but lighter than Kafka.

For multi-agent systems specifically, you typically need multiple queue patterns:

• Request queues: Agent A pushes a task request. Agent B consumes and processes.
• Response queues: Agent B pushes results back. Agent A consumes and continues.
• Broadcast topics: One agent announces a state change (claim approved, risk flagged) that multiple agents need to react to.
• Dead letter queues: Messages that fail repeatedly go here so you can debug without blocking the main flow.

The queue architecture itself becomes part of your observability. You monitor queue depth (how many messages are waiting), processing latency (how long messages sit before being consumed), and error rates. If queue depth climbs steadily, you have a bottleneck—either an agent is too slow or you're generating messages faster than agents can consume.

Agent-to-Agent Protocols: The Grammar of Multi-Agent Communication

A protocol is the agreed-upon format and rules for how agents exchange information. Without a protocol, each agent team invents their own format, and integration becomes a nightmare. Protocols standardise the conversation.

The landscape of agent protocols is evolving fast. The Rise of Agent Protocols explores MCP, A2A and ACP, the three dominant approaches emerging in 2024-2025. Understanding the differences matters because they have different tradeoffs for production deployments.

Model Context Protocol (MCP)

MCP is Anthropic's protocol for structured tool use. It's designed so Claude Opus 4 and other models can reliably call external tools and integrations. When you use MCP, you're defining a schema: "Here are the tools available. Here's the input format. Here's what the output looks like."

MCP works well when you have a clear hierarchical structure: one orchestrator agent (usually Claude Opus 4) calling multiple tool agents. The orchestrator understands the full workflow and decides who to call and when.

Production reality with MCP: You get strong type safety and clear interfaces. The model understands the tools it can call because the schema is explicit. But MCP assumes the orchestrator is smart enough to sequence calls correctly. If your workflow is complex—"validate in parallel with three agents, then aggregate results, then decide next steps"—you're pushing logic into the prompt, which becomes fragile.

MCP is excellent for 90-day deployments where you have a clear orchestrator and well-defined tools. It's what we use when a single Claude Opus 4 instance is orchestrating multiple specialist agents or integrations.

Agent-to-Agent Protocol (A2A)

Agent-to-agent protocols - AWS Prescriptive Guidance defines A2A as peer-to-peer agent communication. Unlike MCP (which is hierarchical—one orchestrator calling tools), A2A treats agents as peers. Any agent can initiate communication with any other agent.

Guide to AI Agent Protocols: MCP, A2A, ACP & More provides a comprehensive comparison: A2A enables agents to negotiate, delegate tasks, and collaborate without a central orchestrator. This is powerful for complex workflows where no single agent has full context.

Agents in Dialogue Part 3: Google's A2A Protocol details the specific mechanics: A2A defines message types (requests, confirmations, delegations, status updates), negotiation patterns (agents can refuse tasks or propose alternatives), and collaboration semantics.

Production reality with A2A: You get true decentralisation. Agents can discover each other, negotiate capabilities, and coordinate without a master orchestrator. But this flexibility comes with complexity. You need:

• Service discovery: How does Agent A find Agent B? You typically use a registry (Consul, Eureka) or DNS.
• Negotiation logic: Agents need to handle "I can't do that" responses and retry or escalate.
• Timeout and retry semantics: If Agent A asks Agent B for something and doesn't hear back, what happens? After how long? How many retries?
• Audit trails: With peer-to-peer communication, tracking who talked to whom and what was decided is harder.

A2A shines when you have genuinely autonomous agents that need to collaborate without a central coordinator. In healthcare, for example, a diagnostic agent might need to ask a lab agent for results, then ask a treatment agent for options. Neither is subordinate to the other.

Agent Collaboration Protocol (ACP)

How AI Communication Protocols (MCP, ACP, A2A, ANP) Enable... positions ACP as a middle ground: structured enough for reliability, flexible enough for complex workflows. ACP emphasises interoperability across different agent platforms and models.

ACP typically includes:

• Standard message envelope: Every message has metadata (sender, receiver, message ID, timestamp, priority).
• Intent declaration: Agents declare what they're trying to accomplish, not just the mechanics of the request.
• Capability matching: Agents advertise what they can do. The system matches requests to capable agents.
• Result contracts: Agents declare what they'll return and what conditions they require.

Production reality with ACP: You get interoperability. An agent built with Claude Opus 4 can talk to an agent built with GPT-5 or Gemini 2.0 because they both speak ACP. You also get better observability—the intent is explicit, so you can trace workflows at a semantic level, not just message level.

The cost is more upfront specification. You're defining capabilities, intents, and contracts before agents start talking. This is worth it for enterprise systems where you have 10+ agents and need to onboard new agents without rewriting everything.

Choosing a Protocol for Your Workflow

Here's how we decide at Brightlume:

Use MCP if: You have one orchestrator (Claude Opus 4 typically) and multiple specialist agents or tools. The orchestrator owns the workflow logic. You want strong type safety and fast iteration. Timeline: 2-4 weeks to production.

Use A2A if: You have truly autonomous agents that need to collaborate as peers. No single orchestrator makes sense. You're building agentic health workflows or complex operational systems where agents need genuine autonomy. You're willing to invest in service discovery and negotiation logic. Timeline: 6-12 weeks to production.

Use ACP if: You have 10+ agents, multiple teams building agents independently, and you need strict interoperability. You're building a platform, not a one-off system. You need strong audit trails and semantic observability. Timeline: 8-16 weeks to production.

For most 90-day deployments we do at Brightlume, we start with MCP because it's fastest to production. As systems grow and more agents enter the picture, we migrate to ACP for interoperability. A2A is for specific domains (healthcare, autonomous operations) where peer autonomy is genuinely required.

Coordination Patterns: How Agents Actually Synchronise Work

Once you've chosen a protocol and queues, you need coordination patterns—the recipes for how agents actually work together on complex tasks.

Sequential Coordination

Agent A finishes, then Agent B starts, then Agent C starts. Simple, predictable, auditable.

Example: Claims processing. Agent A validates the claim format. Agent B checks policy coverage. Agent C performs fraud detection. Each step depends on the previous one succeeding.

Implementation: Use a state machine. Each agent transitions the claim from one state to the next. If any agent fails, the claim stays in that state and gets flagged for manual review.

Queue pattern: Single request queue. Agent A consumes, processes, pushes to Agent B's queue. Agent B consumes, processes, pushes to Agent C's queue.

Latency: Sum of all agent latencies. If each agent takes 500ms, total time is 1.5s. This is acceptable for batch processing (claims processed overnight) but not for real-time workflows.

Parallel Coordination

Multiple agents work on the same task simultaneously, then results are aggregated.

Example: Claim validation. Agent A checks policy rules. Agent B checks fraud patterns. Agent C validates customer data. All three run in parallel. Results are merged.

Implementation: Use a fan-out/fan-in pattern. The orchestrator sends the same request to multiple agents, waits for all responses (with a timeout), then aggregates.

Queue pattern: Multiple request queues (one per agent). One response queue that all agents write to. The orchestrator reads from the response queue and correlates responses by request ID.

Latency: Max of all agent latencies, not the sum. If three agents each take 500ms, total time is ~500ms (plus network overhead). Much faster for time-sensitive workflows.

Tradeoff: More complex. You need to handle partial failures (one agent times out, others succeed). You need aggregation logic that makes sense when inputs conflict.

Hierarchical Coordination

Agents are organised in layers. Layer 1 agents do foundational work. Layer 2 agents depend on Layer 1 results. Layer 3 makes final decisions.

Example: Insurance underwriting. Layer 1 agents gather data (medical records, financial history, claims history). Layer 2 agents analyse risk (medical risk agent, financial risk agent). Layer 3 agent makes the underwriting decision.

Implementation: Explicit state management. Each layer completes before the next layer starts. Use a workflow engine (Apache Airflow, Temporal) to manage transitions.

Queue pattern: Queue per layer. Layer 1 agents write to a layer aggregation queue. Aggregator validates all Layer 1 results are complete, then pushes to Layer 2 queue.

Latency: Sequential within layers, parallel across agents in the same layer. Acceptable for high-stakes decisions (underwriting) where accuracy matters more than speed.

Gossip/Broadcast Coordination

One agent announces something ("claim approved", "risk flagged"), and all interested agents react.

Example: Patient workflow in healthcare. A diagnostic agent publishes "patient has diabetes diagnosis." The treatment agent subscribes and reacts (orders insulin). The monitoring agent subscribes and reacts (sets up glucose monitoring alerts). The billing agent subscribes and reacts (updates insurance codes).

Implementation: Publish-subscribe pattern. Agents publish events to a topic. Other agents subscribe to topics they care about.

Queue pattern: One topic per event type. Agents subscribe to topics. When an agent publishes to a topic, all subscribers get notified.

Latency: Fast for the publisher (fire and forget). Subscribers react asynchronously.

Tradeoff: Loose coupling (good) but harder to enforce ordering. If the treatment agent and billing agent both react to "diagnosis", which happens first? You need idempotency—each agent must handle being called multiple times for the same event.

Designing for Failure: Timeouts, Retries, and Fallbacks

In production, agents fail. Networks fail. Queues fill up. You need explicit patterns for handling failure.

Timeouts

Every inter-agent call must have a timeout. If Agent A calls Agent B and doesn't hear back in 5 seconds, Agent A assumes Agent B failed and takes action (retry, escalate, fail the workflow).

Production reality: Set timeouts based on your SLA, not optimistically. If your workflow SLA is 10 seconds end-to-end, and you have three agents, each agent gets a 3-second timeout (with 1 second of overhead). If an agent regularly takes 4 seconds, you either optimise the agent or increase your SLA.

Timeout values are specific to your domain. Claims processing can tolerate 30-second timeouts. Customer service chatbots need sub-second latency. Healthcare diagnostics might need 60-second timeouts for complex analysis.

Retries

When Agent B times out, Agent A retries. But how many times? Immediately or with backoff? These decisions matter.

Exponential backoff: First retry after 100ms, second after 200ms, third after 400ms. This prevents overwhelming a struggling agent with immediate retries.

Circuit breaker pattern: If Agent B fails 5 times in a row, Agent A stops calling it for 30 seconds. This gives Agent B time to recover without being hammered by retries.

Production reality: Implement circuit breakers. We've seen production incidents where Agent A kept retrying Agent B thousands of times per second, creating a denial-of-service cascade. Circuit breakers prevent this.

Fallbacks

When Agent B is unavailable and retries are exhausted, what happens? You need a fallback.

Example 1: If the fraud detection agent is down, claims still get processed but flagged for manual review. The workflow doesn't fail; it degrades gracefully.

Example 2: If the treatment recommendation agent is down in a healthcare workflow, the system returns a default recommendation (standard care) and logs that the agent was unavailable. The patient still gets care; it's just not optimised.

Fallbacks are domain-specific. In financial services, fallback often means "escalate to human." In healthcare, fallback often means "default to conservative treatment." In hospitality, fallback might mean "offer the standard package."

Observability: Seeing What Agents Are Actually Doing

Without observability, multi-agent systems are black boxes. You ship to production, something breaks, and you have no idea why. Observability means you can trace a single request through all agents, see where latency lives, and debug failures.

Structured Logging

Every agent must log structured events: when it receives a request, what it does, when it sends a response, what went wrong.

Minimum fields:

• Request ID (unique identifier for this workflow execution)
• Agent name
• Timestamp
• Action (received request, started processing, sent response, error)
• Duration
• Result (success/failure)

Example:

{
  "request_id": "claim-12345",
  "agent": "fraud_detector",
  "timestamp": "2025-03-15T14:23:45Z",
  "action": "completed_analysis",
  "duration_ms": 234,
  "result": "risk_score_0.87",
  "status": "success"
}

With structured logging, you can query all events for a single claim and see exactly what happened.

Distributed Tracing

Trace a single request through all agents. See where it goes, how long it spends in each agent, where it fails.

Tools like Datadog, New Relic, or open-source Jaeger/Zipkin give you this. When Agent A calls Agent B, the request carries a trace ID. Agent B logs with the same trace ID. You can then reconstruct the entire call graph.

Production reality: Distributed tracing is non-negotiable for multi-agent systems. When a workflow takes 45 seconds instead of 5 seconds, you need to know if it's because one agent is slow or because of queue wait times or network latency. Tracing shows you exactly.

Metrics and Alerts

Track key metrics per agent:

• Throughput: Messages processed per second
• Latency: p50, p95, p99 response times
• Error rate: Percentage of requests that fail
• Queue depth: How many messages are waiting

Set alerts when metrics cross thresholds. If error rate exceeds 5%, alert. If p99 latency exceeds SLA, alert.

Enterprise Governance: Auditing Multi-Agent Systems

In regulated industries (financial services, healthcare), you need to audit what agents did. "The system approved this claim" isn't enough. You need "Agent A validated the policy, Agent B checked fraud patterns, Agent C verified customer identity, then the orchestrator approved."

This is where our experience with AI Agent Security: Preventing Prompt Injection and Data Leaks becomes critical. Multi-agent systems amplify security concerns because data flows between agents.

Audit Trails

Every agent interaction must be logged immutably. Who called whom, with what inputs, what outputs, when.

Implementation: Write all inter-agent messages to an append-only log (AWS CloudTrail, database with immutable tables). Include:

• Timestamp
• Calling agent
• Called agent
• Request payload
• Response payload
• Agent version/model
• User/context that initiated the workflow

For 90-day production deployments, this is built in from day one. It's not an afterthought.

Version Control and Rollback

When you update an agent (new model, new logic), you need to know exactly what changed and be able to roll back if something breaks.

Our guide on AI Model Governance: Version Control, Auditing, and Rollback Strategies covers this in depth. For multi-agent systems specifically:

• Agent registry: Central record of all agents, their versions, their capabilities, their current status.
• Canary deployments: Deploy new agent version to 5% of traffic first. Monitor for issues. If good, roll out to 100%. If bad, roll back immediately.
• Feature flags: Turn agents on/off without redeploying. If an agent is misbehaving, disable it and fall back to the previous version.

Compliance and Data Flow

Understand where data flows. If Agent A (in region X) calls Agent B (in region Y) with customer PII, you need to know that for compliance. Some regulations require data residency—data must stay in one region.

Map your agent topology and data flows. Know which agents handle sensitive data. Implement controls (encryption, access logs, audit trails) accordingly.

Real-World Example: Multi-Agent Claims Processing

Let's walk through a concrete example: insurance claims processing with four agents.

Agents:

1. Intake Agent: Receives claim, extracts data, validates format
2. Policy Agent: Checks policy coverage, limits, exclusions
3. Fraud Agent: Analyzes claim for fraud patterns
4. Decision Agent: Aggregates results, makes approval/denial decision

Workflow:

1. Claim arrives (email, API, form). Intake Agent receives it.
2. Intake Agent validates format (all required fields present, data types correct). Takes 200ms.
3. Intake Agent pushes to Policy Agent queue: "Check coverage for policy ABC123, claim amount $5,000."
4. Intake Agent also pushes to Fraud Agent queue: "Analyze claim XYZ for fraud patterns."
5. Policy Agent checks coverage rules, returns: "Policy covers this claim type, limit $10,000, deductible $500."
6. Fraud Agent analyzes patterns, returns: "Risk score 0.23 (low risk)."
7. Both agents push results to Decision Agent queue (via aggregator).
8. Decision Agent receives both results, applies business logic: "Low fraud risk + coverage confirmed = APPROVE for $4,500 (claim - deductible)."
9. Decision Agent pushes approval to notification queue.
10. Notification Agent sends email to customer.

Latency breakdown:

• Intake: 200ms
• Policy & Fraud (parallel): max(300ms, 400ms) = 400ms
• Decision: 100ms
• Notification: 200ms
• Total: ~900ms

Failure scenarios:

Fraud Agent times out: Decision Agent waits 5 seconds, then fails over. Fallback: use default fraud score (0.5 = medium risk). Claim gets approved but flagged for manual review. Customer gets claim decision in 2 seconds instead of 5 seconds.

Policy Agent returns error: Same pattern. Fallback to manual review.

Decision Agent crashes: Claim sits in aggregation queue. Monitoring alerts. Ops team restarts Decision Agent. Claim is reprocessed. No data loss because all intermediate results are in queues.

Observability:

Claim XYZ has request ID claim-20250315-001. Each agent logs with this ID:

Intake: claim-20250315-001 received, validated, 200ms
Policy: claim-20250315-001 checked coverage, 300ms
Fraud: claim-20250315-001 analyzed, risk 0.23, 400ms
Decision: claim-20250315-001 approved, 100ms
Notification: claim-20250315-001 sent, 200ms

Ops team can query: "Show me all events for claim-20250315-001." They see the entire journey.

Governance:

Every decision is auditable. Regulator asks: "Why was claim XYZ approved?" Answer: "Policy Agent confirmed coverage, Fraud Agent returned risk score 0.23, Decision Agent applied rule 'risk < 0.5 + coverage confirmed = approve'." Full audit trail available.

Building Multi-Agent Systems for Production: The Brightlume Approach

We ship production-ready multi-agent systems in 90 days because we understand these patterns deeply. We don't speculate about protocols or queues. We've built dozens of systems, hit all the failure modes, and learned what works.

Our process:

1. Week 1-2: Define agent topology. How many agents? What does each do? What's the coordination pattern?
2. Week 2-3: Choose protocol (MCP, A2A, or ACP based on complexity). Choose queue technology. Build the message infrastructure.
3. Week 3-6: Build agents. Each agent is a service with clear inputs/outputs. We use Claude Opus 4 for complex reasoning, smaller models for specific tasks.
4. Week 6-8: Integration and testing. Agents talk to each other. Test failure scenarios (timeouts, crashes, partial failures).
5. Week 8-10: Observability and governance. Structured logging, distributed tracing, audit trails.
6. Week 10-12: Production deployment. Canary rollout, monitoring, on-call support.

This timeline assumes your requirements are clear and your data is ready. Most delays come from unclear requirements or data quality issues, not from the AI engineering itself.

For teams moving pilots to production, the key is understanding that multi-agent systems aren't just "more agents." They're a different architecture with different failure modes, observability requirements, and governance needs. Get the protocols and queues right, and you ship fast. Get them wrong, and you'll be debugging for months.

If you're building multi-agent systems and want to move to production quickly, we're here to help. We've seen what works and what doesn't. Our capabilities include designing and building production-ready multi-agent systems. Check out our case studies to see how we've done this for other organisations.

For more on orchestrating multiple agents in production, read our guide on AI Agent Orchestration: Managing Multiple Agents in Production. And if you're still evaluating whether agents are right for your use case, we break down the differences in Agentic AI vs Copilots: What's the Difference and Which Do You Need?

Key Takeaways

Message queues decouple agents. Asynchronous communication via queues prevents agents from blocking each other and enables graceful degradation when agents are slow or unavailable.

Protocol choice drives architecture. MCP for orchestrated systems, A2A for peer collaboration, ACP for interoperable platforms. Each has different production timelines and complexity tradeoffs.

Coordination patterns are recipes. Sequential for dependent tasks, parallel for independent work, hierarchical for layered analysis, gossip for event-driven reactions. Choose based on your workflow, not based on what's trendy.

Failure handling is mandatory, not optional. Timeouts, retries, circuit breakers, and fallbacks must be built in from day one. Production systems fail. Design for it.

Observability shows you what's actually happening. Structured logging, distributed tracing, and metrics let you debug production issues in minutes instead of days.

Governance and auditing are non-negotiable for regulated industries. Every agent interaction must be logged and auditable. This isn't optional; it's the cost of doing business.

Multi-agent systems are powerful. They let you break complex problems into smaller pieces that agents can solve independently and coordinate their solutions. But power without discipline leads to chaos. The patterns in this article—queues, protocols, coordination, failure handling, observability—are the discipline that turns multi-agent systems from interesting prototypes into reliable production systems that actually work.