From 31e9f9e9b5c64f450d684d356366b89c945939d4 Mon Sep 17 00:00:00 2001 From: shantsis Date: Mon, 29 Nov 2021 21:05:06 -0500 Subject: [PATCH] Edits for Compression 2021 Chapter --- src/content/en/2021/compression.md | 61 ++++++++++++++++++------------ 1 file changed, 37 insertions(+), 24 deletions(-) diff --git a/src/content/en/2021/compression.md b/src/content/en/2021/compression.md index 51e4c433477..fa9723db4c9 100644 --- a/src/content/en/2021/compression.md +++ b/src/content/en/2021/compression.md @@ -23,12 +23,14 @@ unedited: true ## Introduction -Users' time is valuable, and they shouldn't have to wait long for a web page to load. The HTTP protocol allows the responses to be compressed, which decreases the time needed to transfer the content. Even when taking the compression and decompression time into account, compression often leads to significant improvement in the user experience. It can reduce [page weight](./page-weight), improve [web performance](./performance) and boost search rankings, so it's an important part of [Search Engine Optimization](./seo). +A user's time is valuable, so they shouldn't have to wait a long time for a web page to load. The HTTP protocol allows to compress responses and this decreases the time needed to transfer the content. When taking the compression and decompression time into account, this often leads to significant improvement in the user experience. It can reduce [page weight](./page-weight), improve [web performance](./performance) and boost search rankings. As such it's an important part of [Search Engine Optimization](./seo). -This chapter analyzes the current state of, and provides guidance for, lossless compression applied to HTTP responses. The lossy and lossless compression of [media](../2020/media) formats such as images, audio and video is at least, if not more, important for increasing page loading speed, but this is not in the scope of this chapter, as compression usually forms part of these file formats themselves. +This chapter discusses lossless compression applied on a HTTP response. Lossy and lossless compression used in [media](../2020/media) formats such as images, audio and video are equally if not more important for increasing page speed. However these are not in the scope of this chapter as they usually are part of the file format itself. ## HTTP compression is useful for many types of content +{# TODO - is this title too long? Can it be shortened? #} + HTTP compression is recommended for text-based content, such as [HTML](./markup), [CSS](./css), [JavaScript](./javascript), JSON, or SVG, as well as for `woff`, `ttf` and `ico` files. Media files such as images that are already compressed do not benefit from HTTP compression since, as mentioned previously, their representation already includes internal compression. {{ figure_markup( @@ -41,34 +43,39 @@ HTTP compression is recommended for text-based content, such as [HTML](./markup) ) }} -Compared to the other content types, `text/plain` and `text/html` uses the least amount of compression, with merely 12% and 14% using compression at all. This might be because `text/html` is more often dynamically generated than static content such as JavaScript and CSS. However, it is recommended to compress dynamic content as well, and the next sections show how to configure this. +Compared to the other content types, `text/plain` and `text/html` uses the least amount of compression, with merely 12% and 14% using compression at all. This might be because `text/html` is more often dynamically generated than static content such as JavaScript and CSS. However, it is also recommended to compress dynamic content. The next sections show how this is configured. ## Server settings for HTTP compression For HTTP content encoding, the HTTP standard defines the [Accept-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding) request header, with which a HTTP client can announce to the server what content encodings it can handle. The server's response can then contain a [Content-Encoding](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding) header field that specifies which of the encodings was chosen to transform the data in the response body. -Both Brotli and Gzip are supported by virtually all browsers. On the server side, most popular servers like nginx and Apache can be configured to use Brotli and/or Gzip. We want to give some pointers on how to best configure text compression on your server. There are two types of compression settings: +Both Brotli and Gzip are supported by virtually all browsers. On the server side, most popular servers like nginx and Apache can be configured to use Brotli and/or Gzip. There are two types of compression settings: - for precompressed (static) content: this content is already compressed beforehand, ideally with the highest level possible, and the web server should be set up to find the appropriate compressed files based on the filename extension, for example. - for dynamic content, which is compressed on the fly for each request by the web server (or a plugin) itself, for dynamically generated text content -When compressing text with Brotli or Gzip it is possible to select different compression levels. Higher compression levels will result in smaller compressed files, but take a longer time to compress. During decompression, CPU usage tends not to be higher for more heavily compressed files. Rather, files that are compressed with a higher compression level are slightly faster to decode. +{# TODO: these bullets dont flow well gramatically but i don't understand it enough to fix it #} +When compressing text with Brotli or Gzip it is possible to select different compression levels. Higher compression levels will result in smaller compressed files, but take a longer time to compress. During decompression, CPU usage tends not to be higher for more heavily compressed files. Rather, files that are compressed with a higher compression level are slightly faster to decode. -Many support dynamically and/or precompressed HTTP and many of them support [Brotli](https://en.wikipedia.org/wiki/Brotli). +Many support dynamically and/or precompressed HTTP and many of them support Brotli. Practically all text compression is done by one of two HTTP content encodings: Gzip and Brotli. Both are widely supported by browsers: can I use Brotli / can I use Gzip. On the server side, most [popular servers](https://en.wikipedia.org/wiki/HTTP_compression#Servers_that_support_HTTP_compression) can be configured to use [Brotli](https://en.wikipedia.org/wiki/Brotli) and/or [Gzip](https://en.wikipedia.org/wiki/Gzip). -Depending on the web server software you use, compression needs to be enabled, and the configuration may be separate for precompressed and dynamically compressed content. Here are a few pointers for two of the most popular web servers. For Apache, Brotli can be enabled with mod\_brotli, and Gzip with mod\_deflate. For nginx instructions for enabling Brotli and for enabling Gzip are available as well. +{# TODO - this paragraph feels like a repeat of above #} + +Depending on the web server software used, compression needs to be enabled, and the configuration may be separate for precompressed and dynamically compressed content. For Apache, Brotli can be enabled with mod\_brotli, and Gzip with mod\_deflate. For nginx instructions for enabling Brotli and for enabling Gzip are available as well. + +{# TODO: i've seen at least 2 other chapters mention britoli and gzip compression, most recently javascript. you could link to that chapter to learn more... or they should be linking to you, not sure actually #} ## Trends in HTTP compression -The graph below shows the usage share trend of lossless compression from the HTTPArchive metrics over the last 3 years. The usage of Brotli has doubled since 2019, while the usage of Gzip has slightly decreased, and overall the use of HTTP compression is growing on desktop and on mobile. +The graph below shows the usage share trend of lossless compression from the HTTP Archive metrics over the last 3 years. The usage of Brotli has doubled since 2019, while the usage of Gzip has slightly decreased, and overall the use of HTTP compression is growing on desktop and on mobile. {{ figure_markup( image="compression-format-trend-desktop.png", caption="Compression method trend for desktop.", - description="This breaks down the usage of Brotli, Gzip or No text compression on desktop for 2019, 2020 and 2021. For 2019, it shows 7.4% Brotli, 29.5% Gzip, 63.1% No text compression. For 2020, it shows 9.1% Brotli, 30.8% Gzip, 60.1% No text compression. For 2021, it shows 14.0% Brotli, 28.2% Gzip, 58.8% No text compression.", + description="A stacked bar chart showing the usage of Brotli, Gzip or No text compression on desktop for 2019, 2020 and 2021. For 2019, it shows 7.4% Brotli, 29.5% Gzip, 63.1% No text compression. For 2020, it shows 9.1% Brotli, 30.8% Gzip, 60.1% No text compression. For 2021, it shows 14.0% Brotli, 28.2% Gzip, 58.8% No text compression.", chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vQtfyTM9VEweN_Hli3IuxxqU1CRap4V5Q28baEs7aEBResoPRgk9Dwp1m_vdS9lzNlfO8J4hZN7GPT7/pubchart?oid=1962012452&format=interactive", sheets_gid="703407907", sql_file="compression_format_trend.sql" @@ -78,14 +85,14 @@ The graph below shows the usage share trend of lossless compression from the HTT {{ figure_markup( image="compression-format-trend-mobile.png", caption="Compression method trend for mobile.", - description="This breaks down the usage of Brotli, Gzip or No text compression on mobile for 2019, 2020 and 2021. For 2019, it shows 7.5% Brotli, 30.8% Gzip, 61.6% No text compression. For 2020, it shows 9.1% Brotli, 31.6% Gzip, 59.3% No text compression. For 2021, it shows 14.3% Brotli, 28.6% Gzip, 57.1% No text compression.", + description="A stacked bar chart showing the usage of Brotli, Gzip or No text compression on mobile for 2019, 2020 and 2021. For 2019, it shows 7.5% Brotli, 30.8% Gzip, 61.6% No text compression. For 2020, it shows 9.1% Brotli, 31.6% Gzip, 59.3% No text compression. For 2021, it shows 14.3% Brotli, 28.6% Gzip, 57.1% No text compression.", chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vQtfyTM9VEweN_Hli3IuxxqU1CRap4V5Q28baEs7aEBResoPRgk9Dwp1m_vdS9lzNlfO8J4hZN7GPT7/pubchart?oid=646268399&format=interactive", sheets_gid="703407907", sql_file="compression_format_trend.sql" ) }} -Of the resources that are served compressed, the majority are using either Gzip (66%) or Brotli (33%). The other compression algorithms are used infrequently. This split is virtually the same of desktop and mobile. +Of the resources that are served compressed, the majority are using either Gzip (66%) or Brotli (33%). The other compression algorithms are used infrequently. This split is virtually the same for desktop and mobile. {{ figure_markup( image="compression-algorithms-for-http-responses.png", @@ -99,7 +106,7 @@ Of the resources that are served compressed, the majority are using either Gzip ## First-party vs third-party compression -[Third Parties](./third-parties) have an impact on the user experience of a website. Historically the amount of compression used by first parties compared with third parties was significantly different. Let's see what it looks like in the year 2021: +[Third Parties](./third-parties) have an impact on the user experience of a website. Historically the amount of compression used by first parties compared with third parties was significantly different.
@@ -162,18 +169,18 @@ From these results we can see that, [*compared to 2020*](../2020/compression#fir ## Compression levels -Compression level is a parameter given to the encoder adjusting the amount of effort the encoder is applying to find redundancy in the input, to consequently achieve higher compression density. +Compression level is a parameter given to the encoder to adjust the amount of effort is applied to find redundancy in the input in order to consequently achieve higher compression density. Brotli encoding allows compression levels from 0 to 11, while Gzip uses levels 1 to 9. Higher levels can be achieved for Gzip as well, with a tool such as Zopfli. This is indicated as `opt` in the graph below. A higher compression level results in slower compression, but does not substantially affect the decompression speed, even making it slightly faster. -We used the HTTPArchive `summary_response_bodies` data table to analyze the compression levels currently used on the web. This is estimated by re-compressing the responses with different compression level settings and taking the closest actual size, based on around 14000 compressed responses that used Brotli, and 14000 that used Gzip. +We used the HTTP Archive `summary_response_bodies` data table to analyze the compression levels currently used on the web. This is estimated by re-compressing the responses with different compression level settings and taking the closest actual size, based on around 14000 compressed responses that used Brotli, and 14000 that used Gzip. When plotting the amount of instances of each level, it shows two peaks for the most commonly used Brotli compression levels, one around compression level 5, and another at the maximum compression level. Usage of compression levels below 4 is rare. {{ figure_markup( image="compression-levels-brotli.png", caption="Compression levels for Brotli.", - description="This breaks down the usage of compression levels of responses using Brotli, for compression level 0 to 11. It shows peaks at level 5 and at level 11. Each level is also broken down by content type.", + description="Stacked bar charts showing the usage of compression levels of responses using Brotli, for compression level 0 to 11. It shows peaks at level 5 and at level 11. Each level is also broken down by content type.", chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vQtfyTM9VEweN_Hli3IuxxqU1CRap4V5Q28baEs7aEBResoPRgk9Dwp1m_vdS9lzNlfO8J4hZN7GPT7/pubchart?oid=1630047521&format=interactive", sheets_gid="150560131" ) @@ -184,20 +191,24 @@ Gzip compression is applied largely around compression level 6, extending to lev {{ figure_markup( image="compression-levels-gzip.png", caption="Compression levels for Gzip.", - description="This breaks down the usage of compression levels of responses using Gzip, for compression level 1 to 9, as well as `opt` which corresponds to optimizers such as Zopfli. It shows peaks at level 1 and at level 6, as well as a smaller peak at level 9. Each level is also broken down by content type.", + description="Stacked bar charts showing the usage of compression levels of responses using Gzip, for compression level 1 to 9, as well as `opt` which corresponds to optimizers such as Zopfli. It shows peaks at level 1 and at level 6, as well as a smaller peak at level 9. Each level is also broken down by content type.", chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vQtfyTM9VEweN_Hli3IuxxqU1CRap4V5Q28baEs7aEBResoPRgk9Dwp1m_vdS9lzNlfO8J4hZN7GPT7/pubchart?oid=586666706&format=interactive", sheets_gid="150560131" ) }} -## How to analyze compression on your sites +{# TODO - should data be broken down more to say which content type? #} + +## How to analyze compression on sites -You can check which content of your website is already using HTTP compression in the [Firefox Developer Tools](https://developer.mozilla.org/en-US/docs/Tools) or the Chrome DevTools. In the developer tools, open the Network tab and reload your site. A list of responses such as HTML, CSS, JavaScript, fonts and images should appear. To see which ones are compressed, you can check the content encoding in their response header. You can enable a column to easily see this for all responses at once. To do this, right click on the column titles, and in the menu navigate to Response Headers and enable Content-Encoding. +To check which content of a website is already using HTTP compression, the [Firefox Developer Tools](https://developer.mozilla.org/en-US/docs/Tools) or the Chrome DevTools can be used. In the developer tools, open the Network tab and reload your site. A list of responses such as HTML, CSS, JavaScript, fonts and images should appear. To see which ones are compressed, you can check the content encoding in their response header. You can enable a column to easily see this for all responses at once. To do this, right click on the column titles, and in the menu navigate to Response Headers and enable Content-Encoding. Responses that are Gzip compressed will show "gzip", while those compressed with Brotli will show "br". If the value is blank, no HTTP compression is used. For images this is normal, since these resources are already compressed on their own. If you hover the mouse over the values in the Size column, you can also see "transferred over network" and "resource size" to compare the compressed and actual sizes. This data can also be seen for the entire site: this is indicated as "size" / "transferred size" in Firefox and "transferred" and "resources" in Chrome on the bottom left hand side of the Network tab. +{# TODO is this section above necessary? it has less to do with discussing data #} + {{ figure_markup( image="content-encoding.png", caption='Chrome DevTools checking the content-encoding of responses', @@ -207,9 +218,9 @@ If you hover the mouse over the values in the Size column, you can also see "tra ) }} - A different tool that can analyze compression on your site is Google's Lighthouse tool. It runs a series of audits, including the "Enable text compression" audit. This audit attemps to compress resources to check if they reduced by at least 10% and 1,400 bytes. Depending on the score, it can show a compression recommendation in the results, with a list of the resources that can be compressed to benefit your website. + A different tool that can analyze compression on a site is Google's Lighthouse tool. It runs a series of audits, including the "Enable text compression" audit. This audit attemps to compress resources to check if they reduced by at least 10% and 1,400 bytes. Depending on the score, it can show a compression recommendation in the results, with a list of the resources that can be compressed to benefit a website. -The HTTP Archive [runs Lighthouse audits](./methodology#lighthouse) for every mobile page, and from this data we can see that 72% of websites pass this audit. This is 2% less than [last year's](../2020/compression#identifying-compression-opportunities) 74% which is, despite more usage of text compression overall compared to last year, a slight drop. +The HTTP Archive [runs Lighthouse audits](./methodology#lighthouse) for every mobile page, and from this data we observed that 72% of websites pass this audit. This is 2% less than [last year's](../2020/compression#identifying-compression-opportunities) 74%, which is despite more usage of text compression overall compared to last year, a slight drop. {{ figure_markup( image="text-compression-lighthouse-scores.png", @@ -223,15 +234,17 @@ The HTTP Archive [runs Lighthouse audits](./methodology#lighthouse) for every mo ## How to improve on compression -Before thinking about how to compress content, it is often wise to reduce the content to transmit to start with. One way of achieving this is to use so-called "minimizers", e.g. HTMLMinifier, CSSNano, or UglifyJS. +Before thinking about how to compress content, it is often wise to reduce the content transmited to begin with. One way of achieving this is to use so-called "minimizers", such as HTMLMinifier, CSSNano, or UglifyJS. -After having the minimal form of the content you want to transmit, the next step is to ensure compression is enabled. You can verify it is enabled as highlighted in the previous section, and configure your web server if needed. +After having the minimal form of the conten to transmit, the next step is to ensure compression is enabled. You can verify it is enabled as highlighted in the previous section, and configure your web server if needed. When using only Gzip compression (also known as Deflate or Zlib), you can add support for Brotli. In comparison to Gzip, Brotli compresses to smaller files at the same speed, decompresses at the same speed, and is widely supported: can I use Brotli. +{# TODO - the wide support has been mentioned twice already #} + You can choose a well-tuned compression level. What compression level is right for your application might depend on multiple factors, but keep in mind that a more heavily compressed text file does not need more CPU when decoding, so for precompressed assets there's no drawback from the user's perspective to set the compression levels as high as possible. For dynamic compression, we have to make sure that the user doesn't have to wait longer for a more heavily compressed file, taking both the time it takes to compress as well as the potentially decreased transmission time into account. This difference is borne out when looking at compression level recommendations for both methods. -When using Gzip compression for precompressed resources, consider using [Zopfli](https://en.wikipedia.org/wiki/Zopfli), which generates smaller Gzip compatible files. Zopfli uses an iterative approach to find an very compact parsing, leading to 3--8% denser output, but taking substantially longer to compute, whereas Gzip uses a more straightforward but less effective approach. See this comparison between multiple compressors, and this comparison between Gzip and Zopfli that takes into account different compression levels for Gzip. +When using Gzip compression for precompressed resources, consider using [Zopfli](https://en.wikipedia.org/wiki/Zopfli), which generates smaller Gzip compatible files. Zopfli uses an iterative approach to find an very compact parsing, leading to 3-8% denser output, but taking substantially longer to compute, whereas Gzip uses a more straightforward but less effective approach. See this comparison between multiple compressors, and this comparison between Gzip and Zopfli that takes into account different compression levels for Gzip.
@@ -258,7 +271,7 @@ When using Gzip compression for precompressed resources, consider using [Zopfli]
{{ figure_link(caption="Recommended compression levels to use.") }}
-Improving the default settings on web server software would provide significant improvements to those who are not able to invest time into web performance, especially Gzip quality level 1 seems to be an outlier and would benefit from a default of 6, which compresses 15% better on the HTTPArchive `summary_response_bodies` data. Enabling Brotli by default instead of Gzip for user agents that support it would also provide a significant benefit. +Improving the default settings on web server software would provide significant improvements to those who are not able to invest time into web performance, especially Gzip quality level 1 seems to be an outlier and would benefit from a default of 6, which compresses 15% better on the HTTP Archive `summary_response_bodies` data. Enabling Brotli by default instead of Gzip for user agents that support it would also provide a significant benefit. ## Conclusion