SEO 2020 edits #1625

rviscomi · 2020-12-04T06:22:55Z

Progress on #908 #1432

Editors' notes:

global replace of smart quotes to straight quotes
code format for things like robots.txt and HTTP headers
Oxford comma
replace "homepage" to "home page"
right alignment of numeric data in tables
replace passive voice with active voice
various TODOs and fixes

rviscomi

Adding placeholder comments for the content team to resolve all of the existing TODOs while I start the deeper edit.

src/content/en/2020/seo.md

rviscomi · 2020-12-04T06:24:25Z

src/content/en/2020/seo.md

@@ -569,7 +568,7 @@ Feature | Mobile | Desktop
 `orientation` | 33.48% | 33.49%
 `max-device-width` | 26.23% | 28.15%

-<figcaption>{{ figure_link(caption="Media query usage.", sheets_gid="1141218471", sql_file="TODO.sql") }}</figcaption>
+<figcaption>{{ figure_link(caption="Media query usage.", sheets_gid="1141218471", sql_file="TODO..sql") }}</figcaption>


@Tiggerito can you suggest the correct SQL file for this figure?

It looks like I stole this from you 😮

I asked the CSS chapter for some help:

#898 (comment)

And you provided the data...

https://docs.google.com/spreadsheets/d/1sMWXWjMujqfAREYxNbG_t1fOJKYCA6ASLwtz4pBQVTw/edit#gid=1374950017

@rviscomi what's the solution here. Is there any formal SQL to reference?

rviscomi · 2020-12-04T06:24:28Z

src/content/en/2020/seo.md

@@ -637,7 +636,7 @@ Good | 15.44% | 8.39%
 Average | 25.49% | 20.19%
 Poor | 59.06% | 71.42%

-<figcaption>{{ figure_link(caption="Good, Average and Poor ratios of Lighthouse v5 versus v6", sheets_gid="692150551", sql_file="TODO.sql") }}</figcaption>
+<figcaption>{{ figure_link(caption="Good, Average and Poor ratios of Lighthouse v5 versus v6", sheets_gid="692150551", sql_file="TODO..sql") }}</figcaption>


@Tiggerito can you suggest the correct SQL file for this figure?

I think that came from the performance chapter @fellowhuman1101 ?

Ah interesting, so if the query itself doesn't live in the SEO directory then this value wouldn't be very useful and we can omit it.

It’s my intention to handle a full path here, or even relative (../performance/query.sql) in much the same way as we do for images. So would still be good to get the SQL from performance directory.

@bazzadp does it currently work with absolute/relative paths or is there an open issue to track that?

It will work with relative but not absolute. Will need to test to be 100% sure but pretty confident. Plan on looking at this, this weekend when I’ll add absolute support too. Will try to finish this out for launch - saw your updates but not had a chance to look at them yet.

I think that came from the performance chapter @fellowhuman1101 ?

Correct, this came from the performance chapter.

This figure seems to have been removed, so no issue anymore.

Co-authored-by: Jammer <[email protected]> Co-authored-by: Aleyda Solis <[email protected]>

rviscomi · 2020-12-05T23:35:23Z

src/content/en/2020/seo.md


-Let us go through this years’ websites Organic Search optimization main findings.
+{# TODO(authors): Is "Organic Search" a proper noun? Or should it be lowercase? #}


@rviscomi should be lowercase! Should I directly update the doc? Just let me know the best way to proceed to fix :) Thanks

@rviscomi saw your comment below about directly fixing and pushing commits so I already replaced this! Thanks

rviscomi · 2020-12-05T23:36:25Z

src/content/en/2020/seo.md


+{# TODO(analysts, authors): Note that mobile and desktop can't be combined into "all devices" since they are overlapping datasets and most websites would be double-counted. When citing stats throughout the chapter, you need to specify which client you're referring to or include a disclaimer in the intro that stats are mobile unless specified otherwise. #}


@aleyda @Tiggerito I haven't scanned the rest of the document but I'm afraid that this might be common throughout. Could you suggest changes or push commits directly to this branch to resolve any of these issues?

@rviscomi I had a scan through the markdown and only saw this case. I can fix this one. What's the process here, edit the latest draft, and remove the TODO lines, then commit?

I'm around for a few hours then out for the day.

rviscomi · 2020-12-06T04:14:31Z

src/content/en/2020/seo.md


-When analyzing the usage of the disallow statement in robots.txt by using Lighthouse-powered data of over 6 million sites, it was found that 97.84% of them were completely crawlable, with only 1.05% using a disallow statement.
+This is notable as [Google documentation](https://developers.google.com/search/docs/advanced/robots/intro) states that site owners should not use `robots.txt` as a means to hide web pages from Google Search, as internal linking with descriptive text could result in the page being indexed without a crawler visiting the page. Instead, site owners should use other methods, like a `noindex` directive via meta robots.
+{# TODO(authors): Tie this notable fact back to the data: is it notable because the disallow numbers are so low? What does that say about site owners following Google's guidance? #}


@rviscomi it's notable because of Google's guidance about not using robots.txt for that purpose and it is already tied to the "source" by linking to the Google documentation where these guidelines are given using the "Google Documentation" text as anchor text.

The URL is: https://developers.google.com/search/docs/advanced/robots/intro where you can see the guidance:

"You should not use robots.txt as a means to hide your web pages from Google Search results."

--- If this is not clear, could you please let me know how to better rephrase it and link to make it clearer?... I thought it was :(

What you've written is totally relevant and appropriate, my suggestion is to relate it more directly with the data. I'd suggest something like "The low usage of Disallow statements seems to suggest that site owners are adhering to Google's guidance." if you agree with that interpretation.

Thanks @rviscomi ! Understood :D I added the context on why adding disallow along more "indexable" pages rather than "noindexed" is what is notable.

src/content/en/2020/seo.md

…rchive.org into seo-2020-edits

rviscomi

Slow progress. More TODOs. Please feel free to work on them while I finish the edits. The best workflow would be to resolve them by suggesting changes in the PR.

rviscomi · 2020-12-06T08:12:43Z

src/content/en/2020/seo.md


-As part of our examination, we took a look at the incidence rates of different types of structured markup. The available formats include [RDFa](https://www.w3.org/TR/rdfa-primer/) and [Schema.org](https://schema.org/) which come in both the microformats and [JSON-LD](https://www.w3.org/TR/json-ld11/) flavors. Google has recently [dropped the support for data-vocabulary](https://developers.google.com/search/blog/2020/01/data-vocabulary), a vocabulary that was primarily used to implement breadcrumbs. 
+{# TODO(authors): Is schema.org itself a "format"? #}


@rviscomi I really can't think of another way to call it - could you or maybe @Tiggerito suggest one?

schema.org is a vocabulary, it defines what can exist in its world. Formats are things like json-ld, microdata and rdfa which are ways to write information in a vocabulary. data-vocabulary is an alternate vocabulary that Google is dropping.

Chiseling in a stone is a format, the hieroglyphics you make are the vocabulary.

This said after a few 🍷s. So don't quote me, unless it works.

There are more formats. Those are the ones I looked for because they are the main ones that Google officially cares about.

In the future I think it would be worth looking into other things like open graph (a vocabulary+rdfa format) and its hybrid/proprietary use by Facebook, Twitter and Pinterest.

@Tiggerito omg yes! they're vocabularies :) Will update accordingly @rviscomi!

Tony, you always bring "structured data light", with wine or without :D

src/content/en/2020/seo.md

rviscomi · 2020-12-06T08:12:58Z

src/content/en/2020/seo.md


+{# TODO(authors): Is this disparity really noteworthy? The difference seems quite small. #}


@rviscomi Although the disparity between mobile vs. desktop is not "big" it's important to show how there are still slightly more desktop pages featuring one, because of mobile first index. Would you suggest to eliminate the mention of the desktop instead?

SGTM, I'll drop this

rviscomi · 2020-12-06T08:13:07Z

src/content/en/2020/seo.md


-Additionally, we found that 38.61% of desktop pages and 39.26% of mobile pages feature JSON-LD or microformat structured data in the raw HTML, while 40.09% of desktop pages and 40.97% of mobile pages feature structured data in the rendered DOM. 
+{# TODO(authors): This section introduces a few stats but doesn't go into your interpretations of the results. What do you hope readers take away from these stats? #}


@rviscomi I added context/interpretation of SD raw vs. rendered stats.

src/content/en/2020/seo.md

rviscomi · 2020-12-06T08:14:45Z

src/content/en/2020/seo.md

+
+With the increasing popularity of mobile devices to browse and search across the web, search engines have been taking mobile friendliness into consideration as a ranking factor for several years.
+
+{# TODO(authors): MFI has been discussed earlier, so the "in fact" doesn't pack as much punch this time. Consider rephrasing. #}


@rviscomi the "last year reference" that had been included was correct. The MFI for new sites was launched in July last year (as can be seen here) but the confusion was caused because I hadn't linked to that "announcement page" but instead to one published this July describing the different stages of the process and the next ones. So, it was indeed last year.
What I've done to avoid confusions is to 1) add directly the year "2019" 2) link tot he announcement page about the last year MFI launch for all new sites.

Sounds great thank you

rviscomi · 2020-12-06T08:14:51Z

src/content/en/2020/seo.md

+{# TODO(authors): MFI has been discussed earlier, so the "in fact" doesn't pack as much punch this time. Consider rephrasing. #}
+In fact, [since 2016](https://developers.google.com/search/blog/2016/11/mobile-first-indexing) Google has been moving to a mobile-first index, meaning that the content that is crawled, indexed, and ranked is the one accessible to mobile users and the [Smartphone Googlebot](https://developers.google.com/search/docs/advanced/crawling/googlebot?hl=en).
+
+{# TODO(authors): Can you clarify the timeline? You say "July last year" but the blog post is dated July 2020. Would "July this year" change how you structure this sentence chronologically? #}


@rviscomi I reworded this too along the previous change.

rviscomi · 2020-12-06T08:15:25Z

src/content/en/2020/seo.md

+    {{ figure_link(
+      caption="Percent of pages that include each media query feature.",
+      sheets_gid="1141218471",
+      sql_file="TODO..sql"


@Tiggerito I think this change obsoleted the last comment thread, reviving it here

My response was:

It looks like I stole this from you 😮

I asked the CSS chapter for some help:

#898 (comment)

And you provided the data...

https://docs.google.com/spreadsheets/d/1sMWXWjMujqfAREYxNbG_t1fOJKYCA6ASLwtz4pBQVTw/edit#gid=1374950017

…arer

@Tiggerito

cc @Tiggerito could you please help me out by adding the sheets_gid and SQL file to this new table? :)

…means

… table

…rchive.org into seo-2020-edits

rviscomi · 2020-12-06T23:13:31Z

src/content/en/2020/seo.md


+{# NOTE(authors): I've made some ruthless edits to this section to remove everything related to synthetic measurement of CWV, including the entire Lighthouse discussion, which is orthogonal to the real-user aspect of CWV. Please push back if you disagree with any of these edits. #}


⚠️ @aleyda FYI this was a significant edit. I wanted to makes sure the CWV discussion was focused on real-user data from CrUX and not get it confused with lab data from Lighthouse.

Thanks for the heads up @rviscomi ! @fellowhuman1101 since these are major editions from the performance section, could you please take a look too in case there's something you would like to change/replace? :) I'll check it out too but want to make sure you're ok with them!

@aleyda Confirming-- no changes requested. I think @rviscomi 's edits are judicious and help disambiguate Crux data's role as a ranking factor from Lighthouse's lab data.

Thanks @fellowhuman1101, that's great - I just took a look too, it's all good @rviscomi :)

rviscomi · 2020-12-06T23:13:45Z

src/content/en/2020/seo.md


+{# TODO(analysts): Please double check the following two sql_files, as these metrics are related to Lighthouse. #}


@antoineeripret @Tiggerito

Good spot @rviscomi . That chart did not come from lighthouse.sql. It's in:

https://docs.google.com/spreadsheets/d/1ram47FshAjzvbQVJbAQPgxZN7PPOPCKIK67VJZCo92c/edit#gid=996380787

I think @fellowhuman1101 did that one?

I now have 20 Github tabs, 10 drive tabs, and 3 sheets tabs open. And 15 tabs in my Visual Studio Code. Hopefully no more needed!

@rviscomi @Tiggerito The data is from the Performance results workbook (Tab: Web Vitals per device) >> https://docs.google.com/spreadsheets/d/164FVuCQ7gPhTWUXJl1av5_hBxjncNi0TK8RnNseNPJQ/edit#gid=1270303192&range=A1

Copied into the SEO chapter to create the charts.

@rviscomi can I reference this by: ../09_Performance/web_vitals_by_device.sql

I've updated the sql reference for this chart and the one after using a relative path

src/content/en/2020/seo.md

rviscomi · 2020-12-06T23:16:56Z

I've currently edited up through the Performance section and I hope to have the rest edited tonight (New York time). Apologies for the delay! Meanwhile there are a few TODOs for the team to look at while I finish. I'll mark it "ready for review" when I'm done, and open it up to comments on my edits beyond the TODOs.

…em every time I also added additional references regarding links to Google documentation and guidelines

…ext one

…rchive.org into seo-2020-edits

Several images have been fixed based on Tony's and Rick's comments.

github-actions · 2020-12-07T12:49:53Z

Images automagically compressed by Calibre's image-actions ✨

Compression reduced images by 44.8%, saving 98.65 KB.

Filename	Before	After	Improvement	Visual comparison
`src/static/images/2020/seo/seo-canonical-implementation-method.png`	29.36 KB	15.94 KB	-45.7%	View diff
`src/static/images/2020/seo/seo-nofollow-ugc-sponsored-attributes.png`	31.42 KB	17.48 KB	-44.4%	View diff
`src/static/images/2020/seo/seo-presence-of-canonical-tag.png`	33.35 KB	18.80 KB	-43.6%	View diff
`src/static/images/2020/seo/seo-presence-of-h-elements.png`	32.67 KB	17.80 KB	-45.5%	View diff
`src/static/images/2020/seo/seo-presence-of-non-empty-h-elements.png`	34.43 KB	19.01 KB	-44.8%	View diff
`src/static/images/2020/seo/seo-robots-directive-use.png`	29.46 KB	16.02 KB	-45.6%	View diff
`src/static/images/2020/seo/seo-title-character-count.png`	29.57 KB	16.56 KB	-44.0%	View diff

492 images did not require optimisation.

Update required: Update image-actions configuration to the latest version before 1/1/21. See README for instructions.

rviscomi · 2020-12-07T22:55:06Z

Thank you everyone! @aleyda if this looks good to you I can merge.

aleyda · 2020-12-08T09:44:43Z

Thank you everyone! @aleyda if this looks good to you I can merge.

It does! Please merge @rviscomi :)

@ipullrank could you please leave a bit of time today to go through the pending reviews/feedback needed from you? I think is the only pending atm. Thanks a lot!

seo init

a790c5f

rviscomi added the editing Content excellence label Dec 4, 2020

rviscomi added this to the 2020 Content Writing milestone Dec 4, 2020

rviscomi commented Dec 4, 2020

View reviewed changes

rviscomi assigned aleyda and Tiggerito Dec 4, 2020

Update src/content/en/2020/seo.md

c52e820

This was referenced Dec 4, 2020

Featured quote and 3 stats added #1628

Closed

2020 SEO chapter #1589

Merged

rviscomi and others added 2 commits December 4, 2020 15:01

Apply suggestions from code review

dedd5c0

Co-authored-by: Jammer <[email protected]> Co-authored-by: Aleyda Solis <[email protected]>

metadata

d25cbaa

rviscomi assigned rviscomi and unassigned aleyda and Tiggerito Dec 4, 2020

rviscomi added 3 commits December 5, 2020 17:57

straight quotes

8b8abdf

Merge branch 'main' into seo-2020-edits

27d2445

up to robots.txt

aa54176

rviscomi commented Dec 5, 2020

View reviewed changes

rviscomi added 2 commits December 5, 2020 21:41

up to meta robots

e19c463

up to headings

c3b4c15

rviscomi commented Dec 6, 2020

View reviewed changes

aleyda and others added 4 commits December 6, 2020 09:01

Changed all instances of "Organic Search" to lowercase

cb1cec8

up to media queries

5708cbd

Merge branch 'seo-2020-edits' of github.com:HTTPArchive/almanac.httpa…

68b4440

…rchive.org into seo-2020-edits

rm todos

f3922ae

rviscomi commented Dec 6, 2020

View reviewed changes

aleyda added 3 commits December 6, 2020 09:28

Replaced instances of "home pages" to "pages" whenever relevant to do

2b2fef1

Rephrased and split the "Canonical Conflict" paragraph to make it cle…

08d33df

…arer

Added context/interpretation of SD raw vs. rendered stats

995691d

aleyda and others added 9 commits December 6, 2020 13:07

Replaced "formats" for "vocabularies" when referring to Schema.org

571182f

Added table graph with structured data formats Usage

9ff483e

cc @Tiggerito could you please help me out by adding the sheets_gid and SQL file to this new table? :)

Added the clarification of empty Headings inclusion in the text

76cd294

Added context to outgoing external links, YoY comparison and what it …

f05fd80

…means

Merge branch 'main' into seo-2020-edits

016ab54

review feedback

e70ba5b

added sheets_gid and sql_file to Usage of each structured data format…

335522a

… table

up to performance

84c87b5

Merge branch 'seo-2020-edits' of github.com:HTTPArchive/almanac.httpa…

9ece92a

…rchive.org into seo-2020-edits

rviscomi commented Dec 6, 2020

View reviewed changes

rm todos

5cb8fa8

aleyda and others added 6 commits December 7, 2020 00:28

Added context on why using disallow along indexable pages is notable

ff70777

Added context on "outgoing" links in the intro to avoid mentioning th…

5026531

…em every time I also added additional references regarding links to Google documentation and guidelines

Took the "Text versus image links" section before to add context to n…

17132ca

…ext one

completed edit

f233ce4

Merge branch 'seo-2020-edits' of github.com:HTTPArchive/almanac.httpa…

750eca6

…rchive.org into seo-2020-edits

robots.txt status

015b1b8

rviscomi marked this pull request as ready for review December 7, 2020 01:40

antoineeripret and others added 2 commits December 7, 2020 13:47

SEO images modified

11abb98

Several images have been fixed based on Tony's and Rick's comments.

Optimised images with calibre/image-actions

da2ba79

Tiggerito and others added 3 commits December 8, 2020 08:29

added sql references

48df45c

feedback

d5e7ad5

rm unedited

6326d22

tunetheweb merged commit c23c72e into main Dec 8, 2020

tunetheweb deleted the seo-2020-edits branch December 8, 2020 10:37


		Let us go through this years’ websites Organic Search optimization main findings.
		{# TODO(authors): Is "Organic Search" a proper noun? Or should it be lowercase? #}


		{# TODO(analysts, authors): Note that mobile and desktop can't be combined into "all devices" since they are overlapping datasets and most websites would be double-counted. When citing stats throughout the chapter, you need to specify which client you're referring to or include a disclaimer in the intro that stats are mobile unless specified otherwise. #}


		As part of our examination, we took a look at the incidence rates of different types of structured markup. The available formats include [RDFa](https://www.w3.org/TR/rdfa-primer/) and [Schema.org](https://schema.org/) which come in both the microformats and [JSON-LD](https://www.w3.org/TR/json-ld11/) flavors. Google has recently [dropped the support for data-vocabulary](https://developers.google.com/search/blog/2020/01/data-vocabulary), a vocabulary that was primarily used to implement breadcrumbs.
		{# TODO(authors): Is schema.org itself a "format"? #}


		{# TODO(authors): Is this disparity really noteworthy? The difference seems quite small. #}


		Additionally, we found that 38.61% of desktop pages and 39.26% of mobile pages feature JSON-LD or microformat structured data in the raw HTML, while 40.09% of desktop pages and 40.97% of mobile pages feature structured data in the rendered DOM.
		{# TODO(authors): This section introduces a few stats but doesn't go into your interpretations of the results. What do you hope readers take away from these stats? #}


		With the increasing popularity of mobile devices to browse and search across the web, search engines have been taking mobile friendliness into consideration as a ranking factor for several years.

		{# TODO(authors): MFI has been discussed earlier, so the "in fact" doesn't pack as much punch this time. Consider rephrasing. #}


		{# NOTE(authors): I've made some ruthless edits to this section to remove everything related to synthetic measurement of CWV, including the entire Lighthouse discussion, which is orthogonal to the real-user aspect of CWV. Please push back if you disagree with any of these edits. #}


		{# TODO(analysts): Please double check the following two sql_files, as these metrics are related to Lighthouse. #}

SEO 2020 edits #1625

SEO 2020 edits #1625

Conversation

rviscomi commented Dec 4, 2020 • edited Loading

rviscomi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rviscomi Dec 6, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rviscomi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rviscomi commented Dec 6, 2020 • edited Loading

github-actions bot commented Dec 7, 2020

rviscomi commented Dec 7, 2020

aleyda commented Dec 8, 2020

rviscomi commented Dec 4, 2020 •

edited

Loading

rviscomi Dec 6, 2020 •

edited

Loading

rviscomi commented Dec 6, 2020 •

edited

Loading