You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository has been archived by the owner on Apr 2, 2024. It is now read-only.
The LogstreamerInput uses filepath.Walk() for locating files to compare to the file_match regexes.
This is normally fine in a mostly sane directory structure.
In the case of ridiculously large/complex filesystems with numerous levels containing millions of files, this can be somewhat inefficient.
The pattern matching is useful so not suggesting a replacement for regexing.
This change should still follow the same rules for log_directory and file_match.
I propose adding an additional config option to use globbing specifically for the filepath.Walk portion. https://github.com/mozilla-services/heka/blob/dev/logstreamer/filehandling.go#L280
The LogstreamerInput uses
filepath.Walk()
for locating files to compare to the file_match regexes.This is normally fine in a mostly sane directory structure.
In the case of ridiculously large/complex filesystems with numerous levels containing millions of files, this can be somewhat inefficient.
The pattern matching is useful so not suggesting a replacement for regexing.
This change should still follow the same rules for log_directory and file_match.
I propose adding an additional config option to use globbing specifically for the
filepath.Walk
portion.https://github.com/mozilla-services/heka/blob/dev/logstreamer/filehandling.go#L280
glob_pattern = "/this/*/path*/would/*take/*/*/*/forever/to/*/walk/*/in/a*/reasonable/*/*/*/timeframe/*"
The text was updated successfully, but these errors were encountered: