Repository navigation
Conversation
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Currently, in_tail checks for files at previously unwatched paths every refresh_interval (60 seconds by default). Add opt-in enable_dir_watcher to watch the directories of the configured paths and check for new files when those directories change, without waiting for the next periodic check. Group multiple directory changes into a single refresh of the watched file list. Keep periodic checks as a fallback for directory changes that are not detected, and limit the number of watched directories. Keep the option disabled by default to preserve existing discovery timing. Rotated files that match path can also be discovered earlier, including cases where the existing refresh behavior rereads them. Fixes #2985 Signed-off-by: Takuro Ashie <ashie@clear-code.com> Assisted-by: LLM GPT-6.1 Sol (OpenAI)
Add dir_watcher_limit with a default of 100 and require a positive value. When candidates exceed the limit, rotate a window of the interleaved candidates on periodic refreshes. Directory changes keep the current window, while periodic requests merged into a pending refresh still advance it. Keep the starting directory when candidates change, and detach watchers outside the new window before attaching replacements to avoid exceeding the limit during rotation. Candidate exploration remains unrestricted. Assisted-by: LLM GPT-6.1 Sol (OpenAI) Signed-off-by: Takuro Ashie <ashie@clear-code.com>
Keep unresolved directory prefixes during traversal so a symbolic link followed by .. is interpreted like file lookup on each platform. Canonicalize watcher candidates only after traversal, normalizing Windows relative components before resolving symbolic links. Test both glob and literal paths with files on either side of the link to distinguish Windows and Unix lookup behavior. Assisted-by: LLM GPT-6.1 Sol (OpenAI) Signed-off-by: Takuro Ashie <ashie@clear-code.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Fixes #2985
What this PR does / why we need it:
(This PR depends on #5527 and includes its commits.)
Currently,
in_taildiscovers files at previously unwatched paths throughperiodic checks controlled by
refresh_interval(60 seconds by default).This PR adds opt-in
enable_dir_watcherto check for new files whendirectories derived from the configured paths change, without waiting
for the next periodic check. Multiple change notifications are grouped
into a single refresh.
directories.
time-dependent paths, and symbolic links.
dir_watcher_limit(default: 100, positive integer). When candidatesexceed the limit, rotate the watched subset on periodic refreshes.
This limits active directory watchers, not candidate exploration.
The option remains disabled by default to preserve existing discovery
timing. Rotated files matching
pathmay also be discovered earlier,including cases where existing refresh behavior rereads them.
Detection latency depends on the platform and filesystem: libev uses
inotify where available and falls back to stat polling.
Docs Changes:
The in_tail documentation needs entries for:
enable_dir_watcher(default: false)dir_watcher_limit(default: 100)It should also describe platform-dependent detection latency, periodic
refresh fallback, and earlier discovery of matching rotated files.
Release Note:
in_tail: Add opt-in directory watching for earlier discovery of new files,
with a configurable watcher limit and periodic rotation of watched
directories when the limit is exceeded.
Assisted-by: LLM Qwen3.8-Flash-Next
Assisted-by: LLM GPT-6.1 Sol (OpenAI)