Developers say: What should we do as the Web becomes more and more complex?

"Is there a word (besides "privilege") for growing to hate the web as a user, but growing to love it as a developer? Asking for a friend"

— getify (@getify)

I can't think of a word for Kyle, but I totally agree with this feeling and have seen such complaints on large community sites. Developers need to create web pages on modules, app installation prompts, mobile web errors, ads, mobile link jumps, EU cookie prompts, etc., and they actually hate web pages more and more.

In this article, I'll discuss the reasons for this hatred, and the biggest problem facing the web today - "complexity": a term I use to describe anything that is included on a typical web page but does not contribute to what the user is doing - reading an article, purchasing a product, etc. I'll also discuss how this complexity problem is largely caused by the larger money problem facing the web.

To put this all into context, let's look at an example.

Complexity Example

The other day I saw this article on mental illness on my Twitter feed. I opened it in Chrome on iOS and here is what I saw:

This article is a perfect example of how complex the web can be. All I wanted to do was read this article about mental illness like everyone else, but before I could start, I had to filter out a lot of crap I didn't care about - like social buttons, the weather, and a reminder of a usage expiration date, all of which accompanied an article that was only about 2,000 words long. I couldn't even read the beginning of the article on my oversized iPhone 6+.

Loading this article took 200+ HTTP requests, about 2MB of data. Using my WIFI, this article took 3 seconds to load. Web page test shows that using normal mobile network, the page takes 13 seconds to load.

I don't mean to pick on CNN, but it's sad to say this, this article is representative of the general web browsing experience. According to the http archive agency, the average web page exceeded 2M in size in May, reaching 2.08M. It's not hard to find a worse example.

Why is it becoming more and more complicated?

I'm not the first to talk about the web's complexity problem, Peter-Paul Koch recently mentioned it:

“Web versions of articles have an extra layer of complexity added on top of them, which is the direct cause of slow web page loading. The reason for the slowness is not inherent to the web. This is the result of so-called modern web development, and we will eventually overcome this complexity problem and completely "kill" it (referring to the web complexity problem)”

I would like to study the following questions he raised:

“The interesting question is: Why is everything getting more complicated?”

PPK goes on to explore the issue of web complexity caused by the proliferation of online tools, specialized libraries, and frameworks. I agree that unnecessary use of tools on web pages directly affects page size and loading speed, but I think there are more systemic issues that cause web complexity.

Let's take a closer look at this CNN article. This page makes over 200 HTTP requests to 25 different domains.

Yes, you read that correctly. Twenty-five-oh. Several of them are clearly advertising related (e.g. ad.doubleclick.net, pixel.moatads.com), some offer analytics functionality, and many are purposely obfuscated to confuse us.

To me, you could pose the question “what’s all this extra stuff for?” in a different way: since minimizing HTTP requests is one of the well-known web performance best practices, why do so many mobile web sites blatantly violate this rule?

You could argue that part of it is tooling, since using interstitials, social media widgets, and the like generates more HTTP requests than handcrafted content, but I believe the answer has to do with money.

Follow the money

Why does CNN show ads? To make money. Why does CNN introduce tracking services? To learn more about readers, to show more targeted ads, and to make more money. Why does CNN use social media buttons? To get people to share the article, to get more page views, to get more people to see the ads, and to make more money. Why does CNN introduce a weather window? Well, I don't know why, they should have removed this little window.

Again, I don't mean to use CNNs as a "bad example", but rather to use them as a specific example of a model that is somewhat pessimistic about content on the web.

My friend Brian Rinaldi recently wrote that the web’s content model is broken, claiming that we as web users have radically devalued content and authors. He claims that because we refuse to pay for content, people who produce it must resort to increasingly aggressive tactics to monetize their content — that some other purpose makes it possible for that content to be produced.

The payment system has (largely) failed, so we have a ton of ads, tracking scripts, pop-ups, and the like, all working together to provide enough reward for the actual content behind the clutter.

What are you doing?

A lot of people are fighting this problem (it's complicated), but interestingly most of the results are coming from outside the browser.

Flipboard is probably the first big attempt to solve the problem of web clutter. The principle of Flipboard is to get the content from the web page, generate a summary, and then close the link to the original article. This is a pretty good browsing experience when you don’t need to load the entire article, and all the clutter comes from here, so we only need to get a snapshot of the article. As shown in the picture, this is an article "Fanway Park" in Flipboard's iOS app:

But what's even more interesting is that Flipboard doesn't just have content previews, it has now partnered with content providers to allow us to use these articles directly in the Flipboard app - the browser entirely mentioned above. An example is some articles about mental illness from Flipboard's iOS app.

Although unlike the browser version, I still see the same content here, the Flipboard version of the article is obviously more complex. It is not like the browser version, which loads almost instantly. Why can it be displayed so quickly on the web:

The browser version used 200+ HTTP requests from 25 domains, while the Flipboard version used only 4 HTTP requests from 2 domains: cdn.flipboard.com and ad.flipboard.com.

As a company, Flipboard is successful, with over 10 million active readers in 2014 and a valuation approaching $1 billion, making it clear that reading text on mobile devices has surpassed reading text in a browser. Flipboard's success is remarkable, and its business model has been more or less copied by other companies.

Flipboard's Competitors

In May this year, Facebook launched the Instant Articles service. Here’s how they introduced their service:

“More and more people are reading news on their phones, and we want Facebook to have a faster and richer experience for that. People share a lot of articles on Facebook, especially on the Facebook mobile app. So far, these articles take an average of 8 seconds to load, which is much slower than the loading speed of articles of a single content type on Facebook. The reading experience of Instant Articles is 10 times better than reading articles on the regular mobile web.”

Doesn't it sound a lot like Flipboard? If you're a publisher and you're going to let Facebook manage your content, in return, your readers will have a better and faster reading experience, and you may also share in some of the classified advertising revenue.

Let’s see what this looks like in action. BuzzFeed is a participant in the Instant Article program, and they recently published an article called 13 steps to instantly improve your day. As you might expect, the mobile web version of the article is loaded with a bunch of junk, as well as iOS app widgets, social buttons, ads, and more.

Just like the CNN article we saw earlier, BuzzFeed’s website also loaded a bunch of advertising scripts, tracking scripts, and lace stuff. In the end, the entire page made more than 200 HTTP requests and downloaded almost 4MB of data:

Let’s compare it to Facebook’s Instant Articles:

Compared to the previous Flipboard example, this BuzzFeed article is just as busy, despite having similar content. It also loads very quickly, and in my testing, it only made 5 network requests (Facebook also uses several preloading algorithms and loads it before you click, which really makes it feel like it's done "instantly")

There's a lot of debate going on about whether this is a good business model for publishers, and that's certainly the debate going on right now, but it's hard to deny that Flipboard and Instant Articles offer a very elegant reading experience on mobile devices, something that mobile browsers have been eager to address.

Flipboard and Facebook aren’t the only players in this game, the biggest player in tech, Apple, has already announced its entry into the space — with the newly released Apple News using essentially the same business model as Flipboard and Instant Article.

Is this the end of the Web?

No. It's important to remember that no matter how great the user experience Flipboard, Facebook, and Apple provide, they are still private services provided by independent companies. These companies can control how content is consumed and what users can access. Apple can completely control what content is accessed on Apple devices.

Coupled with the fact that these companies require some level of partnership with content providers, these apps only offer content from their ecosystems, which means that the content in these apps only covers a small portion of the entire web content. Since publishing and sharing on the web is very easy, this gives the web a huge advantage over these apps.

That being said, it’s hard to argue that the browser reading experience is better than what Flipboard and Facebook’s Instant Articles offer in their native apps. John Gruber perhaps put it best in a post responding to Facebook’s Instant Articles:

“I’m attracted to this emphasis on speed. Not just because native mobile code is winning in app development, but because with software like Instant Articles, native is making the browser-based web look like a relic, even if that web is just for publishing articles. If I’m right, the overwhelmingly-text I write in Daring Fireball may cause a problem. Daring Fireball’s pages load quickly, but they’re pages I don’t need to connect to very often. I’m worried that this inheritance of the web’s slowness, this ill-considered design trend towards over-producing pages, will start to hurt DF’s traffic.”

Is there any action needed?

Some developments should help with the current state of the Web.

HTTP/2

The recently published HTTP/2 specification provides a way to substantially reduce latency over the network by serving compressed HTTP headers, and loading resources in parallel over a single TCP connection. Once implemented in browsers, HTTP/2 significantly reduces loading times for sites that rely on large numbers of HTTP requests, such as those shown in this article.

The recently published HTTP/2 specification provides a way to substantially reduce latency in web services by compressing HTTP headers and loading resources in parallel using a single TCP connection. Once HTTP/2 is implemented in browsers, it will significantly reduce load times for websites that rely on large numbers of HTTP requests, such as those shown earlier in this article.

Mobile-friendly

Last year, Google announced that they would penalize sites that were not mobile friendly in their search results and would add the words “mobile friendly” next to sites that met the requirements in their search guidelines.

It's a small tweak, but it could give incentives to developers and publishers of site content (whose work is highly correlated with search engine traffic) to keep their objectionable content to a minimum, and early research suggests it appears to have a significant impact on search results.

Proxy Browser

Opera Mini has long been a successful proxy browser, caching resources on its own servers to reduce the amount of data sent to each individual device. Chrome for Android and iOS now include a similar option, and while they are both turned off by default, it is still an option for users who need to increase their network speed.

Ad Blocker

On desktop devices, many ad blockers are the main tool for attacking the Internet, but they have not yet made their way to users' mobile phones, largely because mobile system vendors are very active in preventing them.

Google has a good reason to block ad blockers, as 80%-90% of their revenue comes from online advertising. News says Google removed ad blockers from Google Play as early as 2013.

Historically, Apple has also blocked ad blockers, but that is about to change. Apple recently announced that they will open the ad blocker API in the Safari browser on iOS 9. Whether this is an attack on Google or a friendly gesture to Safari users, it will ultimately reduce the tediousness of iOS users choosing ad blocker apps.

future

Despite all its flaws, I still think it's ripe for innovation in the browser space. Why should the real profit of article publishers be to insert huge ads and create a terrible experience for everyone?

I don't have the answer, and smarter people than me have spent years figuring this out. Still, it seems like this is the best, craziest thing we can do. I still don't think it's going to go anywhere anytime soon in the open web, but we really need to start thinking about how to clean up this mess.