Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compression 2020 more edits #1911

Merged
merged 4 commits into from
Jan 24, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 22 additions & 24 deletions src/content/en/2020/compression.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,11 @@ results: https://docs.google.com/spreadsheets/d/1NKbP4AqMkgCNCsVD3yLhO2d0aqIsgZ7
queries: 19_Compression
featured_quote: Using HTTP compression makes a website load faster and therefore guarantees a better user experience.
featured_stat_1: 23%
featured_stat_label_1: Compressed requests which use Brotli
featured_stat_label_1: Compressed responses which use Brotli
featured_stat_2: 77%
featured_stat_label_2: Compressed requests which use Gzip
featured_stat_label_2: Compressed responses which use Gzip
featured_stat_3: 74%
featured_stat_label_3: Websites that pass the Lighthouse audit with maximum score on text compression
unedited: true
---

## Introduction
Expand All @@ -38,23 +37,23 @@ For most text-based assets, such as [HTML](./markup), [CSS](./css), [JavaScript]

{{ figure_markup(
image="compession-methods-by-content-type.png",
caption="Compression Methods for Different Content Types",
caption="Compression methods for different content types",
description="A stacked bar chart showing the usage rate of different compression algorithms broken down by the content type. The stacked bars divide up the use of Brotli, Gzip, and no compression. `text/html` is the only content type that is compressed less than 50% of the time. `application/json` and `image/svg+xml` are each approximately 64% compressed. `text/css` and `application/javascript` are each approximately 85% compressed. `application/x-javascript` and `text/javascript` are greater than 90% compressed.",
chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vTxUj8-0vKTqPAblIXqekSbiRh1D1lEuA3gVD9w23qwGPtJRE8FbgrURfPAgfFZX2l0t84Wy5ZAGqzR/pubchart?oid=1658254159&format=interactive",
sheets_gid="107138856",
sql_file="19_01.type_of_content_encoding.sql"
)
}}

The figure shows the percent of requests of a certain content type using either Brotli, Gzip or no text compression.
The figure shows the percent of responses of a certain content type using either Brotli, Gzip or no text compression.
It is surprising that while all those content types would profit from compression, the range of percentages varies widely over the different content types: only 44% use compression for `text/html` against 93% for `application/x-javascript`.

For image-based assets text-based compression is less useful and not widely employed. The data shows that the percent of image requests that employ either Brotli, or Gzip is very low, less than 4%. For more info on non text-based assets, check out the [Media](./media) chapter.
For image-based assets text-based compression is less useful and not widely employed. The data shows that the percent of image responses that employ either Brotli, or Gzip is very low, less than 4%. For more info on non text-based assets, check out the [Media](./media) chapter.

{{ figure_markup(
image="http-compression-methods-for-image-types.png",
caption="Compression methods for image types on desktop.",
description="This breaks down what compression methods, if any, are used for all the content types which are images. For all three image types, i.e. jpeg, png and gif, around 96.5% use no compression is used.",
caption="Compression by content type as a percent for desktop.",
chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vTxUj8-0vKTqPAblIXqekSbiRh1D1lEuA3gVD9w23qwGPtJRE8FbgrURfPAgfFZX2l0t84Wy5ZAGqzR/pubchart?oid=1287110333&format=interactive",
sheets_gid="449339162",
sql_file="19_01.type_of_content_encoding.sql"
Expand Down Expand Up @@ -83,12 +82,12 @@ Since precompression is done beforehand, we can spend more time compressing the
</thead>
<tbody>
<tr>
<td>precompressed</td>
<td>Precompressed</td>
<td>11</td>
<td>9 or Zopfli</td>
</tr>
<tr>
<td>dynamically compressed</td>
<td>Dynamically compressed</td>
<td>5</td>
<td>6</td>
</tr>
Expand All @@ -105,7 +104,7 @@ Many [popular servers](https://en.wikipedia.org/wiki/HTTP_compression#Servers_th

## Current state of HTTP compression

Approximately 60% of HTTP responses are delivered with no text-based compression. This may seem like a surprising statistic, but keep in mind that it is based on all HTTP requests in the dataset. Some content, such as images, will not benefit from these compression algorithms and is therefore not often used, as shown in figure 19.2.
Approximately 60% of HTTP responses are delivered with no text-based compression. This may seem like a surprising statistic, but keep in mind that it is based on all HTTP responses in the dataset. Some content, such as images, will not benefit from these compression algorithms and is therefore not often used, as shown in figure 19.2.

<figure>
<table>
Expand All @@ -119,7 +118,7 @@ Approximately 60% of HTTP responses are delivered with no text-based compression
</thead>
<tbody>
<tr>
<td><em>No Text Compression</em></td>
<td><em>No text compression</em></td>
<td class="numeric">60.06%</td>
<td class="numeric">59.31%</td>
<td class="numeric">59.67%</td>
Expand All @@ -131,7 +130,7 @@ Approximately 60% of HTTP responses are delivered with no text-based compression
<td class="numeric">31.21%</td>
</tr>
<tr>
<td>br</td>
<td>Brotli</td>
<td class="numeric">9.10%</td>
<td class="numeric">9.11%</td>
<td class="numeric">9.11%</td>
Expand All @@ -151,21 +150,21 @@ Approximately 60% of HTTP responses are delivered with no text-based compression
Of the resources that are served compressed, the majority are using either Gzip (77%) or Brotli (23%). The other compression algorithms are used infrequently.

{{ figure_markup(
image="compression-algorithms-for-http-requests.png",
caption="Compression algorithm usage rates.",
description="A bar chart showing the usage rates of different compression algorithms for HTTP requests. 77.39% of HTTP requests that use compression employ the Gzip algorithm, 22.59% use Brotli, and 0.03% use some other method.",
image="compression-algorithms-for-http-responses.png",
caption="Compression algorithm for HTTP responses.",
description="A bar chart showing the usage rates of different compression algorithms for HTTP responses. 77.39% of HTTP responses that use compression employ the Gzip algorithm, 22.59% use Brotli, and 0.03% use some other method.",
chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vTxUj8-0vKTqPAblIXqekSbiRh1D1lEuA3gVD9w23qwGPtJRE8FbgrURfPAgfFZX2l0t84Wy5ZAGqzR/pubchart?oid=1523202090&format=interactive",
sheets_gid="1365871671",
sql_file="19_01.type_of_content_encoding.sql"
)
}}

In the graph below, the top 11 content types are displayed with box sizes representing the relative number of requests. The color of each box represents how many of these resources were served compressed, orange indicates a low percentage of compression while blue indicates a high percentage of compression. Most of the media content is shaded orange, which is expected since Gzip and Brotli would have little to no benefit for them. Most of the text content is shaded blue to indicate that they are being compressed. However, the light blue shading for some content types indicate that they are not compressed as consistently as the others.
In the graph below, the top 11 content types are displayed with box sizes representing the relative number of responses. The color of each box represents how many of these resources were served compressed, orange indicates a low percentage of compression while blue indicates a high percentage of compression. Most of the media content is shaded orange, which is expected since Gzip and Brotli would have little to no benefit for them. Most of the text content is shaded blue to indicate that they are being compressed. However, the light blue shading for some content types indicate that they are not compressed as consistently as the others.

{{ figure_markup(
image="compression-algorithms-by-content-type-desktop.png",
caption="Top compressed content types on desktop.",
description="Treemap chart showing image/jpeg (91,926,198 requests - 3.27% compressed), application/javascript (80,360,676 requests - 84.88% compressed), image/png (66,351,767 requests - 3.7% compressed), text/css (54,104,482 requests - 84.0% compressed), text/html (48,670,006 requests - 44.25% compressed), image/gif (39,390,408 requests - 3.42% compressed), text/javascript (35,491,375 requests - 90.74% compressed), application/x-javascript (22,714,896 requests - 93.14% compressed), application/json (13,453,942 requests - 63.02% compressed), text/plain (4,629,644 requests - 32.89% compressed).",
caption="Compression by type on desktop pages.",
description="Treemap chart showing image/jpeg (91,926,198 responses - 3.27% compressed), application/javascript (80,360,676 responses - 84.88% compressed), image/png (66,351,767 responses - 3.7% compressed), text/css (54,104,482 responses - 84.0% compressed), text/html (48,670,006 responses - 44.25% compressed), image/gif (39,390,408 responses - 3.42% compressed), text/javascript (35,491,375 responses - 90.74% compressed), application/x-javascript (22,714,896 responses - 93.14% compressed), application/json (13,453,942 responses - 63.02% compressed), text/plain (4,629,644 responses - 32.89% compressed).",
chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vTxUj8-0vKTqPAblIXqekSbiRh1D1lEuA3gVD9w23qwGPtJRE8FbgrURfPAgfFZX2l0t84Wy5ZAGqzR/pubchart?oid=777357707&format=interactive",
sheets_gid="449339162",
sql_file="19_01.type_of_content_encoding.sql"
Expand All @@ -176,7 +175,7 @@ Figure 19.1 above breaks down the percentage of compression used per content typ

## First-party vs third-party compression

In the [Third Parties](./third-parties) chapter, we learn about third parties and their impact on performance. This is also true for third-party request compression.
In the [Third Parties](./third-parties) chapter, we learn about third parties and their impact on performance. Using third parties can also have an impact on compression.

<figure>
<table>
Expand Down Expand Up @@ -237,7 +236,7 @@ In the [Third Parties](./third-parties) chapter, we learn about third parties an

When we compare compression techniques between first and third parties, we can see that third-party content tends to be compressed more than first-party content. Additionally, the percentage of Brotli compression is higher for third-party content. This is likely due to the number of resources served from the larger third parties that typically support Brotli, such as Google and Facebook.

Compared with [last year's results](../2019/compression#first-party-vs-third-party-compression), we can see that there was a significant increase in the use of compression, notably Brotli for first parties, almost to the point that the use of compression is around 40% for both first and third party, and for desktop and mobile. However within the requests that do use compression, for first parties, the ratio of Brotli compression is only 18%, while the ratio for third parties is 27%.
Compared with [last year's results](../2019/compression#first-party-vs-third-party-compression), we can see that there was a significant increase in the use of compression, notably Brotli for first parties, almost to the point that the use of compression is around 40% for both first and third party, and for desktop and mobile. However within the responses that do use compression, for first parties, the ratio of Brotli compression is only 18%, while the ratio for third parties is 27%.


## How to analyze compression on your sites
Expand All @@ -259,10 +258,9 @@ Another tool to better understand compression on your site is Google's [Lighthou

Because the [HTTP Archive runs Lighthouse audits](./methodology#lighthouse) for each mobile page, we can aggregate the scores across all sites to learn how much opportunity there is to compress more content. Overall, 74% of websites are passing this audit, while almost 13% of websites have scored below a 40. This is a 11.5% improvement when compared to [last year's](../2019/compression#identifying-compression-opportunities) 62.5% of passing scores.

{# TODO(authors): Should this x-axis be labeled "Number of Requests"? Should it be more like "Percent of `enable-text-compression` scores"? #}
{{ figure_markup(
image="text-compression-lighthouse-scores.png",
caption="Lighthouse \"enable text compression\" audit scores.",
caption="Text compression Lighthouse scores.",
description="Stacked bar chart breaking down the scores pages receive for the \"enable text compression\" Lighthouse audit. It shows that 7% of sites score less than 10%, 6% of sites are scoring between 10-39%, 10% of sites scoring between 40-79%, 3% of sites scoring between 80-99%, and 74% of sites have a pass with over 100% of text assets being compressed.",
chart_url="https://docs.google.com/spreadsheets/d/e/2PACX-1vTxUj8-0vKTqPAblIXqekSbiRh1D1lEuA3gVD9w23qwGPtJRE8FbgrURfPAgfFZX2l0t84Wy5ZAGqzR/pubchart?oid=1438276663&format=interactive",
sheets_gid="1284073179",
Expand All @@ -272,6 +270,6 @@ Because the [HTTP Archive runs Lighthouse audits](./methodology#lighthouse) for

## Conclusion

Compared with [last year's Almanac](../2019/compression), there is a clear trend towards using more text compression. The number of requests that don't use any text compression went down a little more than 2%, while at the same time the use of Brotli has increased by almost 2%. The Lighthouse scores have improved significantly.
Compared with [last year's Almanac](../2019/compression), there is a clear trend towards using more text compression. The number of responses that don't use any text compression went down a little more than 2%, while at the same time the use of Brotli has increased by almost 2%. The Lighthouse scores have improved significantly.

Text compression is widely used for the relevant formats, although there is still a significant percentage of HTTP requests that could benefit from additional compression. You can profit from taking a close look at the configuration of your server and set compression methods and levels to your need. A great impact for a more positive user experience could be made by carefully choosing defaults for the most popular HTTP servers.
Text compression is widely used for the relevant formats, although there is still a significant percentage of HTTP responses that could benefit from additional compression. You can profit from taking a close look at the configuration of your server and set compression methods and levels to your need. A great impact for a more positive user experience could be made by carefully choosing defaults for the most popular HTTP servers.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified src/static/images/2020/compression/content-encoding.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion src/templates/en/2020/stories/content_distribution.html
Original file line number Diff line number Diff line change
Expand Up @@ -929,7 +929,7 @@
</div>
</div>
<div data-amp-original-style="position:absolute;pointer-events:none;left:11.65049%;top:5.50162%;width:81.31068%;height:9.87055%;opacity:1" class="amp-wp-741db49">
<div id="el-e3bb34f4-4127-44b6-9bd0-6330f245077b" data-amp-original-style="pointer-events:initial;width:100%;height:100%;display:block;position:absolute;top:0;left:0" class="amp-wp-a1b70b8"><p class="fill amp-wp-8feef5e" data-amp-original-style="white-space:pre-wrap;overflow-wrap:break-word;word-break:break-word;margin:0.7164179104477608% 0;font-family:'Poppins',sans-serif;font-size:0.388350em;line-height:1.2;text-align:initial;padding:0% 0%;color:#000000"><span data-amp-original-style="font-weight: 700; color: #5b6770" class="amp-wp-7e68e50">Compression methods</span><br><span data-amp-original-style="font-weight: 700; color: #5b6770" class="amp-wp-7e68e50">for different contet types</span></p></div>
<div id="el-e3bb34f4-4127-44b6-9bd0-6330f245077b" data-amp-original-style="pointer-events:initial;width:100%;height:100%;display:block;position:absolute;top:0;left:0" class="amp-wp-a1b70b8"><p class="fill amp-wp-8feef5e" data-amp-original-style="white-space:pre-wrap;overflow-wrap:break-word;word-break:break-word;margin:0.7164179104477608% 0;font-family:'Poppins',sans-serif;font-size:0.388350em;line-height:1.2;text-align:initial;padding:0% 0%;color:#000000"><span data-amp-original-style="font-weight: 700; color: #5b6770" class="amp-wp-7e68e50">Compression methods</span><br><span data-amp-original-style="font-weight: 700; color: #5b6770" class="amp-wp-7e68e50">for different content types</span></p></div>
</div>
<div data-amp-original-style="position:absolute;pointer-events:none;left:11.65049%;top:16.66667%;width:53.39806%;height:0.32362%;opacity:1" class="amp-wp-ae7f839">
<div class="mask amp-wp-a1b70b8" id="el-5697f78c-aa62-45a0-a888-e7ff9dcab9f6" data-amp-original-style="pointer-events:initial;width:100%;height:100%;display:block;position:absolute;top:0;left:0">
Expand Down