Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General - Add nofollow to some pages or use rotots.txt #62

Closed
DNA-styx opened this issue Jan 1, 2025 · 1 comment
Closed

General - Add nofollow to some pages or use rotots.txt #62

DNA-styx opened this issue Jan 1, 2025 · 1 comment

Comments

@DNA-styx
Copy link
Owner

DNA-styx commented Jan 1, 2025

Google is doing some crazy spidering. Just found 10 pages of this kinda thing pointing to my one page.

https://stats.dnagames.site/hlstats.php?mode=playerinfo&player=5&ppa_sort=obj_count&ppa_sortorder=desc&maps_sort=headshots&maps_sortorder=desc&server_sort=server&server_sortorder=desc&playerkills_page=2&obj_sort=obj_bonus&obj_sortorder=asc&playerkills_sort=name&playerkills_sortorder=desc&roles_sort=deathsTotal&roles_sortorder=desc&weap_sort=smhits&weap_sortorder=asc

Will try using rotbots.txt with:

User-agent: *
    Disallow: /*?*teams_sort=*
    Disallow: /*?*teams_sortorder=*
    Disallow: /*?*server_sort=*
    Disallow: /*?*server_sortorder=*
    Disallow: /*?*obj_sort=*
    Disallow: /*?*obj_sortorder=*
    Disallow: /*?*ppa_sort=*
    Disallow: /*?*ppa_sortorder=*
    Disallow: /*?*playerkills_sort=*
    Disallow: /*?*playerkills_sortorder=*
    Disallow: /*?*roles_sort=*
    Disallow: /*?*roles_sortorder=*
    Disallow: /*?*weap_sort=*
    Disallow: /*?*weap_sortorder=* 

and also these

    Disallow: /ingame.php 
    Disallow: /show_graph.php 
    Disallow: /sig.php
    Disallow: /trend_graph.php
    Disallow: /assets/ 
    Disallow: /hlstatsimg/
    Disallow: /includes/ 
    Disallow: /updater/
@DNA-styx
Copy link
Owner Author

DNA-styx commented Jan 4, 2025

?mode=chathistory Should be blocked as well, lots of empty pages being generated.

Perhaps everything should be blocked apart from the player's pages?

DNA-styx added a commit that referenced this issue Jan 4, 2025
@DNA-styx DNA-styx closed this as completed Jan 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant