What is the Single Flow Issue?

In Magic Transit (and Magic WAN), all packets belonging to a single network flow (defined by a 5-tuple: source IP, destination IP, source port, destination port, protocol) are processed by a single CPU core on a single metal.

Current Limitations

Metric Safe Limit Absolute Limit With FastPath
PPS (packets/sec) ~15,000 pps ~35,000 pps ~200,000 pps
Throughput ~250-350 Mbps ~850 Mbps ~1+ Gbps

Why This Happens

  1. Flow Hashing: All packets for a single flow land at the same colo, get hashed to the same metal
  2. UNIMOG: Traffic is directed to a specific metal for consistency
  3. Single Core: The flow is pinned to one CPU core for processing
  4. Processing Overhead: Each packet traverses XDP → Linux Stack → Flow Tracking → Magic Firewall → Routing → GRE encapsulation

Key Definitions

5-tuple

The five components that uniquely identify a network flow:

  1. Source IP Address
  2. Destination IP Address
  3. Source Port
  4. Destination Port
  5. Protocol (TCP/UDP/ICMP/etc.)

Why it matters: All packets with the same 5-tuple values are considered part of the same flow and are processed by a single CPU core on a single metal in Magic Transit. This is the root cause of the single flow limitation.

UNIMOG

Cloudflare's layer 4 edge load balancer

UNIMOG directs traffic to a specific metal (server) for consistency. It ensures that all packets belonging to a single flow (5-tuple) land at the same colo and get hashed to the same metal.

Trade-off: This consistency is essential for maintaining flow state (e.g., TCP connections, IPSec tunnels), but it creates the single flow performance limitation since all traffic for one flow must be processed by a single CPU core.

Opportunity Qualification for AEs & SEs

Critical questions to ask during the sales cycle to identify single flow risk early

1. Traffic Type

  • Do you have VPN traffic (IPSec, GRE, VXLAN)?
  • Will you be doing large file transfers?
  • Any real-time streaming or high-throughput applications?

2. Throughput Requirements

  • What is your expected bandwidth per tunnel?
  • What is your expected bandwidth per flow/session?
  • Do you have any single flows exceeding 250 Mbps?

3. VPN Architecture

  • How many VPN tunnels do you plan to use?
  • Are these active-active or active-standby?
  • What equipment terminates the VPNs? (Palo Alto, Cisco, etc.)

4. Use Cases

  • Site-to-site VPN between offices?
  • Remote access VPN?
  • Cloud-to-cloud connectivity?
  • Backup/disaster recovery replication?

5. Compliance/Security

  • Do you need geo-blocking or country-based filtering?
  • How complex are your firewall rule requirements?

Pre-Sales Preparation Checklist

Before the Discovery Call

  • Review customer's industry (financial services, government often have strict VPN requirements)
  • Check if customer is currently using IPSec VPNs with high throughput
  • Prepare to explain the single flow concept in business terms
  • Have FastPath information ready (if applicable)

During the Technical Deep Dive

  • Document expected throughput per tunnel AND per flow
  • Ask about VPN concentrator models and capabilities
  • Discuss traffic patterns (many small flows vs. few large flows)
  • Explain the limitation BEFORE they encounter it

After the Call

  • Document single flow risk in opportunity notes
  • Flag to Implementation team if high-risk
  • Schedule follow-up to review architecture options
  • Create Feature Request if FastPath needed

Real-World Examples

Pre-Sales Red Flags

Watch for these customer responses during discovery:

At-Risk Scenarios

Customer use cases that are likely to encounter single flow limitations:

Mitigation Strategies

Risk Assessment Tool

Answer a few questions to assess single flow risk:

Quick Reference Card


LIMITS:
• Standard: ~15,000 pps (~250-350 Mbps)
• With FastPath: ~200,000 pps (~1+ Gbps)
COMMON AFFECTED USE CASES:
• IPSec VPN tunnels (single tunnel, high throughput)
• Large file transfers (single TCP connection)
• Video streaming (single flow)
DETECTION:
• SampleRD > 15k pps → Risk
• iconduit-grey drops → Confirmed issue
• veth_xmit drops → Confirmed issue
MITIGATIONS:
1. Multiple tunnels (distribute traffic)
2. ESP FastPath (engineering feature for IPSec)
3. MFW optimization (reduce rule overhead)
PRE-SALES RED FLAGS:
• "Single tunnel for main office"
• "Need 1 Gbps per VPN"
• "Palo Alto Prisma with 2-4 tunnels"
• Large geo-blocking requirements
ESCALATION:
• CUSTESC for active issues
• MPERF for FastPath evaluation
• Product for strategic accounts

Escalation Paths

When to Escalate

Scenario Escalate To Ticket Type
Customer experiencing single flow drops MPLAT + CSUP CUSTESC
FastPath evaluation needed MPLAT MPERF
Feature request for higher throughput Product (MAGIC) Feature Request
Customer considering churn CSE + Product P1 Escalation

Information to Include in Escalation

  1. Customer Details: Account ID, NPID, contract value
  2. Traffic Pattern: Source/dest IPs, ports, protocol
  3. Observed Metrics: PPS, throughput, drop rates
  4. Business Impact: What is the customer unable to do?
  5. Timeline: When did this start? Any deadlines?

Customer Communication Templates

Setting Expectations (Pre-Sales)

"Magic Transit processes traffic on a per-flow basis, with each flow handled by a single CPU core. For most use cases with many concurrent connections, this provides excellent performance. However, if you have specific use cases involving very high throughput through a single VPN tunnel or connection, we should discuss architecture options to ensure optimal performance."

Explaining the Limitation

"We've identified that your traffic pattern includes single flows exceeding our recommended throughput for individual CPU cores. This is similar to a highway where each lane (CPU core) can handle a certain amount of traffic. When one lane gets too much traffic, we see congestion. We have several options to address this..."

Discussing FastPath

"For your IPSec VPN use case, we have an optimization called FastPath that can significantly increase throughput per tunnel. This uses eBPF technology to bypass the normal Linux networking stack for established flows. I'd like to connect you with our engineering team to evaluate if this is a good fit for your deployment."