Once upon a time, some explorers stumbled across a new continent, and named it Internet. When they returned to their home country and explained their discovery, a group of pioneers became very excited. They convinced a few schools and companies to help them set up shop on the new continent, and founded a small city. Each organization in the city would get a few buildings and there were some narrow streets connecting them; people would use those roads to visit the other organizations. The flow of people was about even--each organization got about as many visitors as it had members who visited somewhere else.
Meanwhile, settlers were moving into Internet. They didn't want to live in the dense, busy city, though, so they settled in suburbs outside it. As in the big cities, each suburb had a bunch of roads to move around within it, but they also had at least one road to the city. Some had more than one, leading to different areas of it. This is because there were only houses in the suburbs; if the settlers wanted to go to a store, they had to go to the city, where the businesses all were.
The city grew and grew until it simply ran out of room; then the problems started. Real estate prices in the city shot up; suddenly it was pretty expensive to put a business there.
A few clever companies found a solution, though: they would rent a small office in the city and then put stores in a few of the suburbs. This was much cheaper than renting a big storefront in the city, but it made for a longer drive for the suburb residents: they had to drive from their suburb to the city and then drive from there to the suburb the store was in. That meant that a lot of people were driving twice as much as they used to. These stores were so popular (because the merchandise was cheaper) that soon most of the traffic going through the city was really people driving from one suburb to another. And the population of Internet was growing rapidly; the roads could barely handle one load, let alone two!
Now the population of Internet had two choices. They could build more roads from the suburbs to the city; this would allow them to handle the extra traffic, but wouldn't make their drives any shorter. On the other hand, they could instead build roads between the suburbs, bypassing the city entirely. Suburb-to-suburb traffic could use these roads instead, freeing up the roads to the city for people who really did need to go there.
If it wasn't clear, the city is the Internet backbone, the suburbs are the consumer Internet providers, and the solution the businesses came up with is P2P.
Right now, the majority of Internet traffic is P2P--and it's all running through the backbone. The backbone/consumer ISP model was made for a time when individuals mostly only communicated with organizations, so it made sense for each ISP to simply link to the backbone, where the organizations were. But P2P is turning that on its head; a lot of Internet traffic is now going to other consumers. Hence, the ISPs would probably be wise to link to each other so they can take some of the P2P load off the backbone.
The interesting wrinkle is Google. Google has leased a ton of fiber and is putting data centers all over the place; Cringely is convinced that they're building a network for all that P2P traffic to flow though. Basically, he thinks that Google will offer to let ISPs peer with the Google network; once enough of them do, that network will become sort of a second backbone. And as a happy coincidence, Google will be closer to Internet users than any other company; they can use that closeness to deliver high-bandwidth products, give faster service, or lease one or both of these capabilities to others. He thinks that the only way Google can fail is if the ISPs build their P2P backbone themselves instead of using Google's.