Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Robin francois/master: Adding configuration elements to block based on user-agents #22

Merged
merged 7 commits into from
Oct 21, 2024

Conversation

cedricwider
Copy link
Contributor

@cedricwider cedricwider commented Oct 21, 2024

This PR was forged by a rebase onto master. It consists only of the changes from #20

From #20

With the increase of traffic generated by generative AI robots, and since those bots do not respect the robots.txt, user-agent filtering seems at the moment a good approach to prevent services being overloaded.

This PR adds a new variable to configure a block list of user agents. This list is case sensitive.

@cedricwider cedricwider changed the title Robin francois/master Robin francois/master: Adding configuration elements to block based on user-agents Oct 21, 2024
@cedricwider cedricwider merged commit 2473656 into master Oct 21, 2024
25 checks passed
@cedricwider cedricwider deleted the robin-francois/master branch October 21, 2024 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant