Why AI Inferencing Demands a New Network Geography

While the tech world obsesses over GPU clusters and training models, a fundamental infrastructure gap threatens to limit AI's real-world impact. Hunter Newby, a telecom veteran who built carrier hotels and internet exchange points across North America, argues that the shift from AI training to AI inferencing will expose a critical weakness: many cities and entire states lack the neutral interconnection infrastructure needed to support low-latency AI applications.

AI and low-latency inferencing

AI inferencing operates on a completely different model than traditional internet traffic. Where content delivery networks cached static content, AI inference requires real-time, deterministic routing with guaranteed latency thresholds. Newby describes "latency zones:" geographic areas defined by millisecond thresholds where traffic must be delivered with precision. Sub-one-millisecond for 50 optical miles, sub-two for 100 miles, and so on.

This isn't the old eyeballs-versus-content debate. As Newby puts it, "AI wants to get to the endpoint—"whether that's a robot in a factory, a connected car, or enterprise collaboration tools. Applications like robotics and agentic AI systems require instructions delivered in milliseconds, not seconds. This creates an opportunity for carriers to escape the commoditized transit market by offering premium routing services, much like how FedEx transformed package delivery by guaranteeing overnight service.

Internet exchange point deserts

Using data from PeeringDB, Newby mapped a startling reality: many U.S. states have zero internet exchange points. Cities like Wichita, Kansas—home to major aerospace companies including Airbus, Boeing, Textron, and Dassault—must backhaul all internet traffic hundreds of miles to Kansas City or Denver. Engineers at these companies literally transfer large files on thumb drives because their internet connections have file size limits incompatible with scanning airplane parts.

Newby identified 125 cities meeting specific criteria:

• Minimum 200,000 population

• State land-grant university

• Regional airport

• K-12 systems paying more than $2 per megabit

• No local internet exchange

• More than 100 miles to the nearest exchange.

These cities represent both an infrastructure gap and an economic development barrier.

Building neutral internet exchange points

Newby's Connected Nation Internet Exchange Points (CNIXP) broke ground in Wichita, Kansas in May 2024, creating the state's first neutral internet exchange point. Unlike power-hungry data centers that have become NIMBY targets, these facilities focus on network interconnection density—what Newby calls "the cotter pin that holds the wheel on."

The project has already attracted QAI Moon, a neo-cloud provider focused on low-latency inference, which will deploy AI pods directly connected to the exchange fabric operated by DE-CIX. Local enterprises on Innovation Campus at Wichita State University can finally exchange data locally without backhaul costs and latency penalties.

The fresh bread economy

Newby compares data to bread—both are worth the most when it's fresh. Some buyers pay premium prices for immediate access to data, whether keystrokes from mobile devices or real-time sensor readings. Creating the physical infrastructure to deliver that "fresh bread" requires moving beyond the old model of building data centers wherever land and power are cheap, toward building network interconnection points where geography and enterprise presence demand them.