Research Illustrates Damage ISP Shenanigans Can Do, It's Not Just Netflix Users Getting Shafted
We've covered the battles between ISPs and various large-scale content providers multiple times before. From deliberately throttling Netflix users to older spats that prevented Time Warner customers from watching cable channels they'd legally paid for, these kinds of disagreements are common in America these days. A new report from M-Lab, however, illustrates the degree to which these battles can impact all of an ISPs customers, including those who don't use video on demand services like Netflix.
Details on how M-Lab configured its tests are available in this PDF, but the company ran its benchmarks and monitoring by setting up multiple access points within a single location and testing network performance when connected to various ISPs. In order to understand how the tests work, you need to understand the definition between a Transit ISP and an Access ISP. An Access ISP is an ISP like Cablevision, Time Warner, Verizon, or Comcast -- these are the companies we pay for service directly.
A Transit ISP is a company like Cogent or Level 3. These companies are not paid directly for access to a service, but they serve a vital role in connecting Point A to Point B. In many cases, the ISP wars we've discussed at Hot Hardware have been precipitated by battles between Access and Transit ISPs. In such instances, companies like Verizon have bleated about the need for appropriate compensation, intimating that they incur enormous expense thanks to irregular network traffic flows and that Cogent or Level 3 should be forced to pay an appropriate free for the privilege of connecting to their customers.
At the same time, however, anti net neutrality arguments have been made that the government has no role to play in controlling the behavior of ISPs, that ISPs can be trusted to prioritize their traffic fairly, and that these companies that are so desperate to merge and establish a single-player presence in almost every single one of the Top 20 US markets will not abuse that privilege. It therefore seems prudent to examine whether or not said companies have conducted themselves appropriately.
So -- how well did the major ISPs do when it came to providing bandwidth to customers who connected to their services across a third-party Transit ISP like Cogent?
Click to enlarge
From May 2013 - April 2014, they did terribly, and while Time Warner Cable was never quite as bad as the rest of the bunch, it was more than bad enough for complaints to flood in. Compare that performance to the smaller cable provider Cablevision's performance in NYC over the same period.
Click to enlarge
The data speaks for itself. This is not an incidental or minor issue; for almost an entire year Verizon, Comcast, and Time Warner Cable degraded customer performance to the point that the FCC would no longer have classified these connections as "Broadband" connections at all. They do this, according to M-Lab, by simply refusing to fix the problem.
To understand how this works, imagine renting the bottom floor of a two-floor house. Your landlord (and the owner) refuses to fix the leaky roof. Every time it rains, water pours into your downstairs apartment -- but because of where the leaks are located, they cause him no great problem while regularly soaking your bedroom.
When you ask him to please get the roof fixed or at least put a bucket down on the floor, he demands you pay him additional money for the bucket, the time and energy he'll spend dumping out the water when it rains, and a monthly bucket rental fee. Offer to provide your own bucket, and he'll refuse.
That's what ISPs have done for months to the users of Cogent, Level 3, and other Transit ISPs, with usable bandwidth often dropping below 0.5Mbps during peak traffic hours. Median round trip packet times for connecting across Cogent's network from January 2013 to September 2014 was up to 120ms on Verizon, 60-70ms on Time Warner Cable, and above 50ms for Comcast.
These patterns of degradation weren't unique to New York City or to Cogent's network, and they impacted anyone who sends traffic across a transit ISP. If you're thinking "How do I know if I send my traffic across a third party ISP or not?" you've hit on precisely the problem.
It's worth digging into the entire 37 page report from M-Lab, which illustrates, in spectacular fashion, how these kinds of traffic problems plague the entire US network and are nothing more than a deliberate cash grab by greedy companies that are legally allowed to gouge the end customer for insane profit margins. It's not just Netflix customers -- if you contract with Cisco for Netmeeting, or run a business VPN, you can run into precisely these kinds of problems if the "Transit ISPs" and "Access ISPs" are having a spat in your area.
This graph from Medium plots the problem nicely.
This is why the TWC - Comcast Merger Is Such A Bad Idea
One of the problems with net neutrality is that most common readings of the term wouldn't actually prevent this kind of arrangement. Net neutrality, as its commonly discussed and proposed, is a consumer-facing agreement related to what the ISP can do to your data over the last mile. It says nothing about peering arrangements with other network providers, and those agreements are the real problem in this case.
You don't have to be in favor of comprehensive regulation of the Internet, however, to see that our attempt to take a light hand have been disastrous. The original point of deregulating the Internet was that it was supposed to lead to a proliferation in consumer choice and a wide number of service providers competing to offer customers the highest downloads speeds. Go back and check the records -- that was precisely the point of the entire endeavor. However noble its goal, the idea that simply throwing open the market would lead to a proliferation of choice has clearly died. Instead of many choices, customers in most markets have 1-2. If the Comcast /TWC merger goes through, they'll be done to even fewer.
The current ISP structure in America has allowed a handful of companies to position themselves, leechlike, at the center of a pulsing mass of data, secure in the knowledge that they can charge everyone exorbitant rates , mouth platitudes at Congress, and escape with buckets of cash. Maybe you think government regulation is the answer to this. But the current system -- in which cable and telco companies actively work to champion legislation that prevents cities and states from partnering with other providers to improve their Internet access or operating their own networks, then have the guts to tell Congress that such arrangements constitute "competition" is so fundamentally broken it's hard to grasp.
Details on how M-Lab configured its tests are available in this PDF, but the company ran its benchmarks and monitoring by setting up multiple access points within a single location and testing network performance when connected to various ISPs. In order to understand how the tests work, you need to understand the definition between a Transit ISP and an Access ISP. An Access ISP is an ISP like Cablevision, Time Warner, Verizon, or Comcast -- these are the companies we pay for service directly.
A Transit ISP is a company like Cogent or Level 3. These companies are not paid directly for access to a service, but they serve a vital role in connecting Point A to Point B. In many cases, the ISP wars we've discussed at Hot Hardware have been precipitated by battles between Access and Transit ISPs. In such instances, companies like Verizon have bleated about the need for appropriate compensation, intimating that they incur enormous expense thanks to irregular network traffic flows and that Cogent or Level 3 should be forced to pay an appropriate free for the privilege of connecting to their customers.
At the same time, however, anti net neutrality arguments have been made that the government has no role to play in controlling the behavior of ISPs, that ISPs can be trusted to prioritize their traffic fairly, and that these companies that are so desperate to merge and establish a single-player presence in almost every single one of the Top 20 US markets will not abuse that privilege. It therefore seems prudent to examine whether or not said companies have conducted themselves appropriately.
So -- how well did the major ISPs do when it came to providing bandwidth to customers who connected to their services across a third-party Transit ISP like Cogent?
;)
Click to enlarge
From May 2013 - April 2014, they did terribly, and while Time Warner Cable was never quite as bad as the rest of the bunch, it was more than bad enough for complaints to flood in. Compare that performance to the smaller cable provider Cablevision's performance in NYC over the same period.
;)
Click to enlarge
The data speaks for itself. This is not an incidental or minor issue; for almost an entire year Verizon, Comcast, and Time Warner Cable degraded customer performance to the point that the FCC would no longer have classified these connections as "Broadband" connections at all. They do this, according to M-Lab, by simply refusing to fix the problem.
To understand how this works, imagine renting the bottom floor of a two-floor house. Your landlord (and the owner) refuses to fix the leaky roof. Every time it rains, water pours into your downstairs apartment -- but because of where the leaks are located, they cause him no great problem while regularly soaking your bedroom.
When you ask him to please get the roof fixed or at least put a bucket down on the floor, he demands you pay him additional money for the bucket, the time and energy he'll spend dumping out the water when it rains, and a monthly bucket rental fee. Offer to provide your own bucket, and he'll refuse.
That's what ISPs have done for months to the users of Cogent, Level 3, and other Transit ISPs, with usable bandwidth often dropping below 0.5Mbps during peak traffic hours. Median round trip packet times for connecting across Cogent's network from January 2013 to September 2014 was up to 120ms on Verizon, 60-70ms on Time Warner Cable, and above 50ms for Comcast.
These patterns of degradation weren't unique to New York City or to Cogent's network, and they impacted anyone who sends traffic across a transit ISP. If you're thinking "How do I know if I send my traffic across a third party ISP or not?" you've hit on precisely the problem.
It's worth digging into the entire 37 page report from M-Lab, which illustrates, in spectacular fashion, how these kinds of traffic problems plague the entire US network and are nothing more than a deliberate cash grab by greedy companies that are legally allowed to gouge the end customer for insane profit margins. It's not just Netflix customers -- if you contract with Cisco for Netmeeting, or run a business VPN, you can run into precisely these kinds of problems if the "Transit ISPs" and "Access ISPs" are having a spat in your area.
This graph from Medium plots the problem nicely.
This is why the TWC - Comcast Merger Is Such A Bad Idea
One of the problems with net neutrality is that most common readings of the term wouldn't actually prevent this kind of arrangement. Net neutrality, as its commonly discussed and proposed, is a consumer-facing agreement related to what the ISP can do to your data over the last mile. It says nothing about peering arrangements with other network providers, and those agreements are the real problem in this case.
You don't have to be in favor of comprehensive regulation of the Internet, however, to see that our attempt to take a light hand have been disastrous. The original point of deregulating the Internet was that it was supposed to lead to a proliferation in consumer choice and a wide number of service providers competing to offer customers the highest downloads speeds. Go back and check the records -- that was precisely the point of the entire endeavor. However noble its goal, the idea that simply throwing open the market would lead to a proliferation of choice has clearly died. Instead of many choices, customers in most markets have 1-2. If the Comcast /TWC merger goes through, they'll be done to even fewer.
The current ISP structure in America has allowed a handful of companies to position themselves, leechlike, at the center of a pulsing mass of data, secure in the knowledge that they can charge everyone exorbitant rates , mouth platitudes at Congress, and escape with buckets of cash. Maybe you think government regulation is the answer to this. But the current system -- in which cable and telco companies actively work to champion legislation that prevents cities and states from partnering with other providers to improve their Internet access or operating their own networks, then have the guts to tell Congress that such arrangements constitute "competition" is so fundamentally broken it's hard to grasp.
