Adding configuration elements to block based on user-agents #20
Conversation
templates/Caddyfile.j2
Outdated
| } | ||
|
|
||
| handle @badbots { | ||
| abort |
There was a problem hiding this comment.
Wouldn't it be better to send a HTTP header code 403 Forbidden (https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403) instead of just closing the connection?
There was a problem hiding this comment.
Well, I would say that if they were respecting us, we could be respecting them. But there are not. We want to be sure that they do not come back and do not try again.
There was a problem hiding this comment.
Artefactual sends 403 status code by default in their Nginx Role for AtoM:
https://github.com/artefactual-labs/ansible-nginx/blob/master/defaults/main/main.yml#L70
I still wonder, if sending a 403 is better than no answer to keep them away.
|
Apart from the response to send, I think all points have been solved. |
|
We can easily reply 403 if you prefer. I will just need to double check how to do it. |
…ing TCP connection
|
@tomcbe I have tested how to reply a 403 with some simple HTML content. I have made a new commit to do 403 instead of closing the TCP connection. |
|
@robin-francois I approved your PR now, but as this is a repository managed by simplificator. I can't merge it. @cedricwider @tizpuppi Can one of you review this PR and merge it if agree with the proposed changes? |
…d when defining the variables
|
Closing this PR in favor of #22 |
With the increase of traffic generated by generative AI robots, and since those bots do not respect the robots.txt, user-agent filtering seems at the moment a good approach to prevent services being overloaded.
This PR adds a new variable to configure a block list of user agents. This list is case sensitive.