Releases: VIPnytt/RobotsTxtParser
Releases · VIPnytt/RobotsTxtParser
Version 2.1.0
- PHP 8 support.
- New method AllowClient->
isCovered()
. - Method AllowClient->
hasPath()
is deprecated.
Version 2.0.2
Bug fixes:
- Don't throw errors when parsing an empty robots.txt file #5
Version 2.0.1
Bug fixes
- Rules didn't match when contained both wildcard(s)
*
and an end anchor$
#5 - Rules didn't always match when contained special characters like
+()[]
Other improvements
- Normalize rules starting with an wildcard
*
to always have an prefixed slash/
- Added more test cases
- Minor code and test optimizations
Version 2.0.0
Version 2.0.0 stable is finally here, with lots of new abilities, features and improvements.
The essentials:
- All protocols now supported library-wide
robots.txt
rendering- Import/Export rule sets as arrays
- Cache
robots.txt
files to MySQL (optional) - Delay handling with precision using MySQL (optional)
- New directives:
Comment
NoIndex
Request-rate
Robot-version
Visit-time
- Some new abilities at directive level
- Lots of improvements under the hood
Version 1.x
incompatible!
Version 2.0.0 rc 2
- Bug fix: DelayManager->debug() didn't return all entries
- Bug-fix: Wrong user-agent sometimes passed to the MySQL Delay handler
- No longer force checks the Comment directive for comments
- Refactored (internal) error handler
- Minor code optimizions
- More tests (code coverage at 97,3% 🎉 )
Version 2.0.0 rc 1
- Renamed
$delayClient->getQueue()
tocheckQueue()
Import
class, for importing exported arrays- Updated default byte-limit from 64 to 512 kilobytes
- Performance optimizions
- Fixed an bug causing the user-agent string parser to generate warnings
- Minor refactoring, with performance in mind
- Refactored documentation
Version 2.0.0 beta 2
Version 1.x incompatible.
Changelog since version 2.0-beta.1:
- PHP 7.1 compatible
- Added RenderClient class
- Minor speed improvements
- Updated
vipnytt/useragentparser
to v1.0 - Make sure some cURL options is not overwritten.
- The confusing
isHostListed()
func is removed. - Refactored UriParser
- Refactored Clean-param directive classes
- Refactored Host directive classes
- HostClient
getWithFallback
renamed togetWithUriFallback
- SQL debug methods
- Documentation
Version 2.0.0 beta 1
Version 1.x incompatible.
Changelog since version 2.0-alpha:
- Support for the
NoIndex
directive - cURL now replaces GuzzleHttp
- Cache and UriClient classes now supports all available protocols
- No longer lists empty user-agents.
- Directive aliases (for simple typos / errors).
- Retry after HTTP header support.
- Improved Status-code handler, no longer throws exceptions on strange codes.
- Improved multi-User-agent directive handler.
- Host directive now also supports URL redirect host as fallback
- Tons of code optimizions
- Refactoring
- Documentation
Version 2.0.0 alpha
Alpha release, use at your own risk!
Version 1.x incompatible.
Short version change-log (since version 1.0.1):
- More directives and specifications supported.
- Increased specification coverage (including bug-fixes and new handlers).
- Built-in caching system.
- Built-in delay handling system.
- Restructured for better maintainability and easier usage.
- More documentation.
See README for more information.
Version 1.0.1
- More accurate end anchor
$
rule matching.