REP writing

I put together an informal collection of thoughts (this is
not an article or an essay, though I've included parts and
pieces in various discussions and presentations) on regional
peering in my weblog this week. 

At the risk of attracting arrows and flames or looking
totally stupid, I humbly submit this for discussion of this
group. I'd be interested to know where others agree and
where you disagree.

----
http://pete.kruckenberg.com/blog/2003/02/17.html#a68

I have been actively involved with regional and local 
peering in Utah since the mid-90's. I often am asked (or ask 
myself, or am told), why is regional peering important? Many 
people who run large networks argue that regional peering is 
an interesting hobby for "insignificant" networks to engage 
in, but all real networks know that only a few peering 
locations are justifiable.

I beg to differ.

The Internet as we know it today was built around some 
fundamental assumptions or conditions:

- most traffic would aggregate to relatively few centers, 
based around population centers and computer centers, where 
a relatively few applications would generate traffic created 
by a relatively few major Internet businesses serving the 
general populous 
- most traffic ultimately ends up using a relatively few 
networks and interconnect points to get from one user to 
another 
- best-effort is all we can do right now. Since the Internet 
cannot ubitiqously support high-bandwidth, low-latency, 
low-jitter applications, the majority, if not all, 
applications have to operate with the 
least-common-denominator low-bandwidth low-reliability 
expectations. Applications that require more just don't get 
deployed, because they don't work. 
- the network build-out fizzled before the most important 
part happened: broadband to the home. This is improving now, with 
significant but still limited bandwidth increases through 
DSL and cable Internet. But most of the users still access 
the Internet through part-time low-speed dial-up 
connections, that are all but useless for anything other 
than email, chat and limited Web browsing. Cable and DSL are 
usually so restricted that they offer little more than 
always-connected faster versions of dial-up applications.

Often people confuse what is with should be. I am often told 
that the network is the way it is because it should be that 
way, that it's the best way for it to be. I believe that the 
network is the way it is because that was the easiest, 
cheapest and fastest way to provide services that people 
would buy at the time. That was the first generation, going 
from 0 to 10mph. Now people have gotten a taste, and they 
want to see what 30mph might be like.

What have we learned in the last decade, in the first 
generation of the pervasive network?

- everyone wants to generate traffic. This weblog is an 
example. Peer-to-peer, distributed computing, voice-over-IP, 
Internet radio, H.323 and SIP video conferencing, instant 
messenging are all examples. These have all happened in a 
network that was not designed for distributed traffic 
generation and fought it as much as it supported it. Imagine 
what would happen if the network supported what people want 
to do. 
- the network didn't replace human interaction, at best it 
supplemented it. Few people use audio or video over the 
network in their daily interactions, because the network 
doesn't support it yet. Email and IM are nice conveniences, 
but their exclusive use works only in certain situations, so 
they are usually supplemented with face-to-face and 
old-fashioned phone calls. The next network will make human 
interaction more effective, and may often be able to 
effectively replace face-to-face communications. 
- the network is to reinforce human interaction, not 
eliminate it. The people we most communicate with live closest to us. 
The most frequent (and often most important) human 
interaction usually happens in our neighborhood and cities, 
with our neighbors, clergy, family, friends, co-workers, 
bosses, legislators, teachers and others. If the network 
could make human interaction more convenient and efficient 
without making it less effective, we would achieve more. We 
have a glimpse of this from the first generation network, 
and we want more.

When "the other foot falls" and users can get high-speed 
networks to their homes that enables many high-quality 
services to be used simultaneously, the network will look 
very different. Today's applications will use a small 
fraction of the overall network traffic. The majority of 
bandwidth will happen between users and other users and 
service providers in their communities. National-level 
traffic will be significant, but relatively small compared 
to local traffic. At the center of it all will be a 
local/regional traffic exchange, that will interconnect 
high-speed user networks with each other and with service 
providers.

A regional traffic exchange is critical to these new kinds 
of applications:

- lower latency. Many next-generation applications are 
constrained by the speed of light through fiber. Shorter 
distances enabled more of the applications people want to 
use. 
- more reliability. Local exchanges eliminate points of 
failure, congestion and fluctuation between end users. They 
also provide better alternative paths to national- and 
regional-level paths. Many next-generation applications are 
much more sensitive to congestion and fluctations introduced 
when traffic has to travel inter-state to reach other 
networks. 
- lower cost, higher bandwidth. Ultimately it doesn't make 
sense to haul a bunch of traffic out of the local area just 
to bring it right back. Bandwidth has to be cheap and 
plentiful to support the next-generation uses of the 
network. At a national level, the costs are always higher 
because the distances are longer. Keeping local traffic 
within the local area will ensure that costs stay low while 
plenty of high-speed bandwidth is available

Did I say that we still don't know what happens when users 
get cheap, pervasive high-speed bandwidth? It can only make 
regional peering a more critical part of the network 
infrastructure.

Is regional peering only important in the future? Though my 
own experience with high-speed networks shows it'll become 
much more important as first-mile connectivity improves, 
regional peering fills some important roles now.

- application performance (particularly TCP-based 
applications) is directly related to bandwidth, latency and 
packet loss. The same TCP connection that is constrained to 
~ 6Mbps through a national network can run at 4-8x that 
speed when a regional exchange provides a shorter, 
less-congested path. Regional exchanges are also very 
popular with gamers, telecommuters and other latency- and 
jitter-sensitive application users (such as video).
- lower costs: even though a regional exchange may off-load 
only a small fraction of Internet traffic, the costs to 
connect to the exchange are low enough to justify the 
investment
- product differentiation: Internet is Internet is Internet. 
Sales people need something to tell customers to make them 
different than other companies who sell the same commodity. 
Telling potential customers that your network ensures better 
performance to regional and local networks is a powerful 
message, and closes deals.
- community investment. People live where they live, and are 
invested in their communities. Texas pride is very real. 
People do care about where their network traffic goes, 
especially when events in far-off places affect their 
abilities to reach their local friends and neighbors. 
Keeping local traffic local is becoming a more powerful 
message as people rely more on the network.

Did I say that I beg to differ about the importance of 
regional and local exchanges? I've been wrong before, but 
I'd have to say I've had lots of time to see what's 
happening and think about it, and I feel more right than 
ever. More people are starting to say the same things, too. 
I don't know that regional exchanges will replace national 
exchanges, but I think they will eventually fill a role just 
as noticable and critical.