Defense in depth for outbound voice: how we stop stolen API keys from making calls

Credentials leak. A GitHub repo accidentally goes public. A log line with a trunk password gets shipped to a shared Elastic cluster that half of engineering has read access to. A laptop with a developer's whole SSH keyring and voice-trunk .envfile goes missing at an airport. A contractor's access was supposed to be revoked when their engagement ended and nobody did it.

On most voice platforms, a leaked outbound-trunk credential means someone else dials PSTN on your tab until you notice. The attacker has your username and password — the trunk accepts both and places the call. You find out when the carrier bill lands.

On Voylo, the credential alone isn't enough. We require it plusthe incoming SIP connection to come from an IP address you've pre-authorised on that specific trunk. Credential leak + wrong source IP = rejected before the AI vendor is ever consulted, before any carrier minute is ever incurred, and tagged in your CDR with the attacker's IP so you know exactly which key leaked.

How outbound trunks normally work

Every voice-AI vendor needs an outbound SIP trunk — the thing they dial through to make calls on your behalf. The vendor sits inside their cloud (ElevenLabs, Vapi, Retell, OpenAI Realtime, your in-house stack); they're given a username, password, and SIP URI; they place an INVITE to that URI with a Digestauth header; the trunk 200 OK's; the call connects.

The authentication model is just the username + password. If someone else obtains both, they can register from any IP and dial anywhere. The trunk has no way to distinguish "Vapi sending a legitimate INVITE from 34.140.101.213" from "an attacker in a different country sending an INVITE with the same credentials."

What Voylo does differently

Each outbound trunk has a per-trunk IP allowlist. It's a list of CIDR blocks and individual IPs. The trunk accepts calls only when BOTH conditions are met:

1. The digest credential is correct, AND
2. The incoming SIP connection originates from an IP covered by the allowlist.

Both checks run before the first SIP response is sent, before an AI session is spun up, before any carrier contact. A failed check authenticates as the trunk (so the AGI knows who owns the trunk) but then rejects at the funds / auth gate — producing a visible CDR with the attempted source IP in the rejection reason.

Setting it up

Open your outbound trunk → Allowed source IPs. The field accepts comma- or newline-separated entries:

• Individual IPs: 34.140.101.213
• CIDR ranges: 34.140.0.0/16 (recommended for vendors who rotate egress IPs inside a documented block)
• 0.0.0.0/0— the explicit "allow any IP" marker. Default on new trunks. Use this while you're prototyping; replace it with real IPs the moment you move to production.

For the major voice-AI vendors you probably want the published CIDR range of their outbound SBC. Each vendor documents it:

• ElevenLabs publishes a handful of GCP egress IPs; mentions elsewhere in their docs that they may use other IPs within the broader 34.140.0.0/16 + 35.190.0.0/16 blocks. Use the CIDRs to be safe.
• Vapi, Retell, OpenAI Realtime publish specific SBC IPs — easier, more secure. Pin them exactly.
• Your own backend(if you're dialing through Voylo directly without a middleware vendor) — your server's public IP(s). Static-IP infra is a plus here.

What a rejected attempt looks like

Open Calls after an attacker has probed your trunk. You see rows marked outcome=failed with a clear rejection:

rejected: source IP 203.0.113.45 is not in the
allowed source IPs of trunk "production-vapi"

Every outbound call row — successful or rejected — also carries the source IP as a visible column. In normal operation that column shows your AI vendor's IP, call after call. When something looks off (new IP, geography anomaly, unexpected cadence), it's obvious at a glance.

What this does and doesn't protect against

• Protects against:credential leaks where the attacker doesn't have network access to your authorised IPs. This is 95% of real-world leak scenarios — leaked through logs, screenshots, chat tools, public repos, disgruntled contractors.
• Does NOT protect against:an attacker who compromises the AI vendor itself (or the server at an allowed IP). If the attacker's traffic can originate from your allowed CIDR, the IP check passes. That's a different problem — AI vendor security is on them.
• Does NOT protect against: an attacker inside your own infrastructure who steals the credential AND originates from your allowlisted IP. Again, different problem; your internal network security is on you.

What this IS, is defense in depth. It converts a single credential leak from "$20k problem" into "audit log entry." That alone is worth doing.

Three mistakes we see

1. Leaving 0.0.0.0/0 on in production.It's the default. It's also the first thing to tighten the minute you have a real AI vendor configured. Lock to the vendor's published CIDR immediately.
2. Forgetting to update when the vendor publishes a new IP range.ElevenLabs, Vapi, and Retell all rotate IPs occasionally — subscribe to their changelog and update the trunk. The rejected-call CDR will tell you the day they rotate without warning you: you'll see legitimate calls start failing with the IP-mismatch reason.
3. Setting it per-vendor-session, not per-trunk. If your dev team has one trunk and your prod team has another, each gets its own allowlist. The prod trunk should allow only the prod SBC IPs; the dev trunk may allow dev + staging IPs. Don't share trunks across environments just to save a click — the whole point is per-trunk blast radius.

Pairing with other guardrails

The IP allowlist is one of three per-trunk / per-number outbound protections that compose:

• Allowed caller-IDs on the trunk— restricts which From-numbers the AI can present. Prevents a leaked credential from spoofing a different customer's number on outbound.
• Country-code block list on the number — stops dialing destinations you didn't authorise (toll-fraud's classic entry point). Independent of who is dialing — travels with the number.
• Funds gate on the account— negative wallet auto-pauses outbound. Even if everything else bypasses, the attacker's bill can't compound forever.

A credential-leak attacker needs to beat all four — the right credentials, from the right IP, with an allowed From-number, dialing a non-blocked destination, with a wallet that hasn't tripped. That's not impossible, but it's not a one-leaked-password problem anymore.

The one-line summary

Open your outbound trunk. Replace0.0.0.0/0with your AI vendor's actual published CIDR. Save. Thirty seconds of configuration that converts the most common credential-leak attack from a bill into an audit log entry.