fix(core): use recursive FSEvents on macOS instead of non-recursive kqueue#34523
Merged
FrozenPandaz merged 1 commit intonrwl:masterfrom Feb 25, 2026
Merged
Conversation
✅ Deploy Preview for nx-docs canceled.
|
✅ Deploy Preview for nx-dev canceled.
|
Contributor
| View your CI Pipeline Execution ↗ for commit c086f96
☁️ Nx Cloud last updated this comment at |
comp615 commented Feb 19, 2026
Contributor Author
comp615 left a comment
comp615 left a comment There was a problem hiding this comment.
Note: I feel like there might be a better refactor / solution here (e.g. pull them all up since they are common? Should FileEventKind::Modify(ModifyKind::Data(_)) => continue, even exist now?)...but I didn't have a full understanding and this is a narrow change. Feel free to build upon!
![nx-cloud[bot]](https://avatars.githubusercontent.com/in/80458?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Important
At least one additional CI pipeline execution has run since the conclusion below was written and it may no longer be applicable.
Nx Cloud has identified a possible root cause for your failed CI:
This CI failure appears to be related to the environment or external dependencies rather than your code changes.
No code changes were suggested for this issue.
You can trigger a rerun by pushing an empty commit:
git commit --allow-empty -m "chore: trigger rerun" git push 
🎓 Learn more about Self-Healing CI on nx.dev
…queue PR nrwl#34329 switched all watched paths to non-recursive to reduce inotify watch count on Linux. On macOS, the notify crate uses kqueue for non-recursive watches instead of FSEvents. kqueue silently fails at scale (~5,250+ directories) due to vnode table pressure, causing the daemon to never detect file changes in large monorepos. This fix uses platform-conditional watch modes: - macOS: single recursive watch on the workspace root (uses FSEvents) - Linux/Windows: non-recursive per-directory watches (preserves nrwl#33781 fix) On macOS, FSEvents handles recursive watching natively from a single root path, so directory enumeration and dynamic registration are skipped entirely. This also improves daemon startup time on macOS (from ~10 minutes to <1 second in a 354-project monorepo). Fixes nrwl#34522 Amp-Thread-ID: https://ampcode.com/threads/T-019c7804-5473-73ae-861f-d5dc1572a1ce Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019c7804-5473-73ae-861f-d5dc1572a1ce Co-authored-by: Amp <amp@ampcode.com>
comp615 force-pushed the fix/macos-watcher-event-filter branch from 7cb2bda to c086f96 Compare comp615 changed the title 
FrozenPandaz approved these changes Feb 25, 2026
Collaborator
FrozenPandaz left a comment
FrozenPandaz left a comment There was a problem hiding this comment.
Thank you! It makes sense to stick with kqueue recursive watcher for macos.
I'm going to follow up with some unit tests to prevent this from happening in the future.
Thank you for your contribution! 👍
FrozenPandaz added a commit that referenced this pull request Feb 25, 2026
Adds comprehensive test coverage for PR #34523 (fix for #34522). Rust unit tests (watcher.rs): - test_macos_uses_single_recursive_watch: Verifies macOS uses 1 recursive watch vs 11,022 non-recursive watches, preventing kqueue vnode exhaustion - test_extract_new_directories_filters_ignored: Tests ignore filtering - Helper function determine_watch_strategy() for testing watch setup logic TypeScript integration test (watcher.spec.ts): - should detect file changes in large directory structures: Creates 10,000+ directories and verifies file events are actually delivered. This would fail with 22.5.0 on macOS due to silent kqueue failure. These tests verify the fix prevents kqueue vnode table exhaustion on macOS in large monorepos (5,000+ directories) by using FSEvents instead.
FrozenPandaz added a commit that referenced this pull request Feb 25, 2026
## Current Behavior PR #34523 fixed the macOS file watcher issue (#34522) but did not include comprehensive test coverage. ## Expected Behavior Tests should verify that the fix works correctly and catch any future regressions. ## Related Issue(s) Adds test coverage for #34523 and #34522 --- ## Changes **TypeScript integration test** (`packages/nx/src/native/tests/watcher.spec.ts`): Added **"should detect file changes in large directory structures"** - a comprehensive integration test that: 1. Creates 10,000+ directories simulating a monorepo-scale workspace 2. Starts a real `Watcher` instance 3. Creates and modifies files deep in the directory tree 4. Verifies that file change events are actually delivered This test validates the actual behavior users care about - that file watching works reliably in large repos - rather than testing implementation details. It would catch any regression where events fail to be delivered at scale. ## Testing TypeScript integration test validates the actual bug fix - that file events are delivered reliably in large directory structures with 10,000+ directories.
FrozenPandaz pushed a commit that referenced this pull request Feb 26, 2026
…queue (#34523) ## Current Behavior Since Nx 22.5.0, the daemon's native file watcher silently drops all file change events on macOS in large monorepos (~5,250+ watched directories). `nx watch`, `nx serve`, and any daemon-dependent file watching is broken. The root cause is that #34329 switched all watched paths to `WatchedPath::non_recursive()`. On macOS, the `notify` crate uses **kqueue** for non-recursive watches instead of **FSEvents**. kqueue silently fails at scale due to vnode table pressure (`kern.num_vnodes == kern.maxvnodes`), causing the daemon to never detect file changes. This is a **scale-dependent** bug: it works fine in small workspaces (~30 directories) but breaks silently in large ones. | | **Nx 22.4.5** | **Nx 22.5.0+** | |---|---|---| | **Small repo (~30 dirs)** | Works (FSEvents) | Works (~30 kqueue watches) | | **Large repo (~5,250+ dirs)** | Works (FSEvents) | **Broken** (kqueue silently drops all events) | ## Expected Behavior The macOS file watcher should detect file creates, modifications, and deletions at any scale, matching the behavior of Nx 22.4.x. ## Fix Use platform-conditional watch modes: - **macOS:** Single recursive watch on the workspace root (uses FSEvents natively) - **Linux/Windows:** Non-recursive per-directory watches (preserves the #33781 inotify fix) On macOS, FSEvents handles recursive watching from a single root path, so directory enumeration and dynamic registration are skipped entirely. This also improves daemon startup time on macOS from ~10 minutes to <1 second in a 354-project monorepo. ### What changed in `watcher.rs` 1. **Initial pathset:** On macOS, watch only the root directory recursively via FSEvents instead of enumerating all directories for non-recursive kqueue watches. 2. **Dynamic directory registration (`on_action`):** Wrapped in `#[cfg(not(target_os = "macos"))]` since FSEvents already watches the full tree. Linux/Windows behavior is completely unchanged. ### Why the event filter is fine as-is We verified that with recursive FSEvents watches, macOS emits specific `FileEventKind` variants (`Create(File)`, `Modify(Data(Content))`, `Remove(File)`, `Modify(Name(Any))`) that the current `watch_filterer.rs` already handles correctly. Zero events were rejected by the catch-all. The `Modify(Any)` / `Create(Any)` variants are kqueue artifacts that are not needed with FSEvents. ### Why kqueue fails silently Apple's [File System Events Programming Guide](https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/KernelQueues.html) explicitly recommends FSEvents over kqueue for large hierarchies: *"If you are monitoring a large hierarchy of content, you should use file system events instead."* kqueue requires `open(path, O_EVTONLY)` per watched directory. Under vnode table pressure, the kernel recycles vnodes with kqueue watches attached without notifying the watcher. There is no error, no partial delivery, and no diagnostic signal. ## Tested on - macOS 26.3 (Tahoe), Apple Silicon (arm64), APFS - 354-project pnpm monorepo (~19,865 non-ignored directories) - Verified: file modifications, file creates, and file deletes all detected - Daemon init time: ~10 min (with enumeration) -> <1s (with root-only FSEvents watch) ## Related Issue(s) Fixes #34522 Co-authored-by: Amp <amp@ampcode.com> (cherry picked from commit d5cd6a1)
FrozenPandaz added a commit that referenced this pull request Feb 26, 2026
## Current Behavior PR #34523 fixed the macOS file watcher issue (#34522) but did not include comprehensive test coverage. ## Expected Behavior Tests should verify that the fix works correctly and catch any future regressions. ## Related Issue(s) Adds test coverage for #34523 and #34522 --- ## Changes **TypeScript integration test** (`packages/nx/src/native/tests/watcher.spec.ts`): Added **"should detect file changes in large directory structures"** - a comprehensive integration test that: 1. Creates 10,000+ directories simulating a monorepo-scale workspace 2. Starts a real `Watcher` instance 3. Creates and modifies files deep in the directory tree 4. Verifies that file change events are actually delivered This test validates the actual behavior users care about - that file watching works reliably in large repos - rather than testing implementation details. It would catch any regression where events fail to be delivered at scale. ## Testing TypeScript integration test validates the actual bug fix - that file events are delivered reliably in large directory structures with 10,000+ directories. (cherry picked from commit 5d64f72)
FrozenPandaz pushed a commit that referenced this pull request Feb 26, 2026
…queue (#34523) ## Current Behavior Since Nx 22.5.0, the daemon's native file watcher silently drops all file change events on macOS in large monorepos (~5,250+ watched directories). `nx watch`, `nx serve`, and any daemon-dependent file watching is broken. The root cause is that #34329 switched all watched paths to `WatchedPath::non_recursive()`. On macOS, the `notify` crate uses **kqueue** for non-recursive watches instead of **FSEvents**. kqueue silently fails at scale due to vnode table pressure (`kern.num_vnodes == kern.maxvnodes`), causing the daemon to never detect file changes. This is a **scale-dependent** bug: it works fine in small workspaces (~30 directories) but breaks silently in large ones. | | **Nx 22.4.5** | **Nx 22.5.0+** | |---|---|---| | **Small repo (~30 dirs)** | Works (FSEvents) | Works (~30 kqueue watches) | | **Large repo (~5,250+ dirs)** | Works (FSEvents) | **Broken** (kqueue silently drops all events) | ## Expected Behavior The macOS file watcher should detect file creates, modifications, and deletions at any scale, matching the behavior of Nx 22.4.x. ## Fix Use platform-conditional watch modes: - **macOS:** Single recursive watch on the workspace root (uses FSEvents natively) - **Linux/Windows:** Non-recursive per-directory watches (preserves the #33781 inotify fix) On macOS, FSEvents handles recursive watching from a single root path, so directory enumeration and dynamic registration are skipped entirely. This also improves daemon startup time on macOS from ~10 minutes to <1 second in a 354-project monorepo. ### What changed in `watcher.rs` 1. **Initial pathset:** On macOS, watch only the root directory recursively via FSEvents instead of enumerating all directories for non-recursive kqueue watches. 2. **Dynamic directory registration (`on_action`):** Wrapped in `#[cfg(not(target_os = "macos"))]` since FSEvents already watches the full tree. Linux/Windows behavior is completely unchanged. ### Why the event filter is fine as-is We verified that with recursive FSEvents watches, macOS emits specific `FileEventKind` variants (`Create(File)`, `Modify(Data(Content))`, `Remove(File)`, `Modify(Name(Any))`) that the current `watch_filterer.rs` already handles correctly. Zero events were rejected by the catch-all. The `Modify(Any)` / `Create(Any)` variants are kqueue artifacts that are not needed with FSEvents. ### Why kqueue fails silently Apple's [File System Events Programming Guide](https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/FSEvents_ProgGuide/KernelQueues/KernelQueues.html) explicitly recommends FSEvents over kqueue for large hierarchies: *"If you are monitoring a large hierarchy of content, you should use file system events instead."* kqueue requires `open(path, O_EVTONLY)` per watched directory. Under vnode table pressure, the kernel recycles vnodes with kqueue watches attached without notifying the watcher. There is no error, no partial delivery, and no diagnostic signal. ## Tested on - macOS 26.3 (Tahoe), Apple Silicon (arm64), APFS - 354-project pnpm monorepo (~19,865 non-ignored directories) - Verified: file modifications, file creates, and file deletes all detected - Daemon init time: ~10 min (with enumeration) -> <1s (with root-only FSEvents watch) ## Related Issue(s) Fixes #34522 Co-authored-by: Amp <amp@ampcode.com> (cherry picked from commit d5cd6a1)
FrozenPandaz added a commit that referenced this pull request Feb 26, 2026
## Current Behavior PR #34523 fixed the macOS file watcher issue (#34522) but did not include comprehensive test coverage. ## Expected Behavior Tests should verify that the fix works correctly and catch any future regressions. ## Related Issue(s) Adds test coverage for #34523 and #34522 --- ## Changes **TypeScript integration test** (`packages/nx/src/native/tests/watcher.spec.ts`): Added **"should detect file changes in large directory structures"** - a comprehensive integration test that: 1. Creates 10,000+ directories simulating a monorepo-scale workspace 2. Starts a real `Watcher` instance 3. Creates and modifies files deep in the directory tree 4. Verifies that file change events are actually delivered This test validates the actual behavior users care about - that file watching works reliably in large repos - rather than testing implementation details. It would catch any regression where events fail to be delivered at scale. ## Testing TypeScript integration test validates the actual bug fix - that file events are delivered reliably in large directory structures with 10,000+ directories. (cherry picked from commit 5d64f72)
Contributor
| This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Current Behavior
Since Nx 22.5.0, the daemon's native file watcher silently drops all file change events on macOS in large monorepos (~5,250+ watched directories).
nx watch,nx serve, and any daemon-dependent file watching is broken.The root cause is that #34329 switched all watched paths to
WatchedPath::non_recursive(). On macOS, thenotifycrate uses kqueue for non-recursive watches instead of FSEvents. kqueue silently fails at scale due to vnode table pressure (kern.num_vnodes == kern.maxvnodes), causing the daemon to never detect file changes.This is a scale-dependent bug: it works fine in small workspaces (~30 directories) but breaks silently in large ones.
Expected Behavior
The macOS file watcher should detect file creates, modifications, and deletions at any scale, matching the behavior of Nx 22.4.x.
Fix
Use platform-conditional watch modes:
On macOS, FSEvents handles recursive watching from a single root path, so directory enumeration and dynamic registration are skipped entirely. This also improves daemon startup time on macOS from ~10 minutes to <1 second in a 354-project monorepo.
What changed in
watcher.rson_action): Wrapped in#[cfg(not(target_os = "macos"))]since FSEvents already watches the full tree.Linux/Windows behavior is completely unchanged.
Why the event filter is fine as-is
We verified that with recursive FSEvents watches, macOS emits specific
FileEventKindvariants (Create(File),Modify(Data(Content)),Remove(File),Modify(Name(Any))) that the currentwatch_filterer.rsalready handles correctly. Zero events were rejected by the catch-all. TheModify(Any)/Create(Any)variants are kqueue artifacts that are not needed with FSEvents.Why kqueue fails silently
Apple's File System Events Programming Guide explicitly recommends FSEvents over kqueue for large hierarchies: "If you are monitoring a large hierarchy of content, you should use file system events instead." kqueue requires
open(path, O_EVTONLY)per watched directory. Under vnode table pressure, the kernel recycles vnodes with kqueue watches attached without notifying the watcher. There is no error, no partial delivery, and no diagnostic signal.Tested on
Related Issue(s)
Fixes #34522