Skip to content
This repository was archived by the owner on Sep 1, 2026. It is now read-only.

Rebuilt CLI watcher around new fswatch package - #4026

Merged
John Favret (johnfav03) merged 27 commits into
microsoft:mainfrom
johnfav03:cli-watcher-parcel-integration
Jun 12, 2026
Merged

John Favret (johnfav03) merged 27 commits into
microsoft:mainfrom
johnfav03:cli-watcher-parcel-integration

Conversation

@johnfav03

@johnfav03 John Favret (johnfav03) commented May 21, 2026 •

Copy link
Copy Markdown
Contributor

The CLI watcher (tsgo --watch) was previously implemented with a polling-based approach, using a Run() loop - the key disadvantage of this system is that I would still call Stat() on tracked files even when they didn't change, which yielded an idle CPU usage of ~5% in the vscode repo.

With the addition of the fswatch package (PR #3980), we can rebuild the watcher around OS-level filesystem events, which results in an idle CPU usage of <1% on average in the vscode repo.

Fixes: #3611

Copilot AI review requested due to automatic review settings May 21, 2026 21:23

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request rebuilds tsgo --watch around the new internal/fswatch package (OS filesystem events + debouncing) instead of the previous polling loop, while keeping internal/vfs/vfswatch as a baseline-snapshot change detector for tracked compiler inputs.

Changes:

  • Replaces the CLI watch loop with an event-driven fswatch subscription (recursive + ignore filter) and coalesced DoCycle signaling.
  • Simplifies internal/vfs/vfswatch by removing polling/timing/debounce machinery, leaving only snapshot + compare logic.
  • Introduces a new multi-backend internal/fswatch implementation (fanotify/inotify/kqueue/FSEvents/ReadDirectoryChangesW), plus extensive tests and documentation.

Reviewed changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
internal/vfs/vfswatch/vfswatch.go Removes polling-based watcher loop; keeps snapshot+compare watch-state logic.
internal/vfs/vfswatch/vfswatch_test.go Updates tests for the new NewFileWatcher(fs) constructor.
internal/vfs/vfswatch/vfswatch_race_test.go Updates race/fuzz tests to remove polling-interval related coverage.
internal/fswatch/windows.go Adds Windows backend based on ReadDirectoryChangesW with overlapped I/O.
internal/fswatch/watcher.go Adds core fswatch API surface, watcher lifecycle, per-dir state, and callback filtering.
internal/fswatch/walkdir.go Adds generic walkDir implementation (portable baseline).
internal/fswatch/walkdir_windows.go Adds Windows-optimized walkDir using FindFirstFile/FindNextFile.
internal/fswatch/walkdir_unix.go Adds Unix walkDir using getdents/getdirentries for fewer stats.
internal/fswatch/walkdir_test.go Adds tests validating walkDir behavior (symlinks, perms, entries, callbacks).
internal/fswatch/walkdir_other.go Routes unsupported OSes to the generic walkDir implementation.
internal/fswatch/walkdir_dirent_noreclen.go Adds DragonFlyBSD dirent helpers for reclen/inode extraction.
internal/fswatch/walkdir_dirent_linux.go Adds Linux dirent helpers for reclen/inode extraction.
internal/fswatch/walkdir_dirent_fileno.go Adds FreeBSD/OpenBSD/NetBSD dirent helpers for reclen/inode extraction.
internal/fswatch/walkdir_dirent_darwin.go Adds macOS dirent helpers for reclen/inode extraction.
internal/fswatch/testutil_test.go Adds retry-capable test harness to reduce platform flake impact.
internal/fswatch/README.md Documents fswatch usage, watcher selection, and error semantics.
internal/fswatch/LICENSE Adds upstream MIT license attribution.
internal/fswatch/kqueue.go Adds kqueue backend for BSD/macOS (fd-per-entry model).
internal/fswatch/inotify_linux.go Adds inotify backend for Linux.
internal/fswatch/fanotify_linux.go Adds fanotify backend for Linux with rename probing/fallback.
internal/fswatch/fanotify_linux_test.go Adds Linux fanotify-specific tests (availability, parsing, shutdown, cross-watcher).
internal/fswatch/fsevents_darwin.go Adds macOS FSEvents backend (event classification and lifecycle).
internal/fswatch/fsevents_darwin_ffi.go Adds cgo-free CoreFoundation/CoreServices/libdispatch FFI plumbing and NFC normalization helpers.
internal/fswatch/fsevents_darwin_ffi.s Adds shared assembly trampolines and exported trampoline addresses.
internal/fswatch/fsevents_darwin_ffi_arm64.s Adds arm64 FSEvents Create trampoline + callback shim assembly.
internal/fswatch/fsevents_darwin_ffi_amd64.s Adds amd64 FSEvents Create trampoline + callback shim assembly.
internal/fswatch/fsevents_darwin_ffi_arm64_test.go Adds static disassembly-based test ensuring the arm64 callback only touches safe registers.
internal/fswatch/fsevents_darwin_nfd_test.go Adds macOS Unicode normalization regression tests (NFD/NFC mismatch scenarios).
internal/fswatch/event.go Adds event kinds and per-path coalescing state machine.
internal/fswatch/eventlist_test.go Adds unit tests for event coalescing and atomic drain semantics.
internal/fswatch/debounce.go Adds per-backend debounce mechanism (min/max coalescing windows).
internal/fswatch/CHANGES.md Documents divergence from upstream @parcel/watcher and lists bugfixes/features.
internal/fswatch/cgmanifest.json Adds component governance manifest pointing at upstream source commit/tag.
internal/fswatch/canonicalize_other.go Adds no-op path canonicalization for non-darwin platforms.
internal/fswatch/canonicalize_darwin.go Adds NFC canonicalization for macOS path consistency.
internal/execute/watcher.go Rebuilds CLI watcher to subscribe to fswatch events and signal DoCycle asynchronously.
.golangci.yml Updates linter exclusion pattern to include internal/fswatch.

Comment thread internal/vfs/vfswatch/vfswatch.go Resolved
Comment thread internal/execute/watcher.go Resolved
Comment thread internal/execute/watcher.go Resolved
Comment thread internal/execute/watcher.go Resolved
@DanielRosenwasser

This comment was marked as outdated.

@johnfav03
John Favret (johnfav03) force-pushed the cli-watcher-parcel-integration branch from e87cc21 to 68df16e Compare June 2, 2026 20:48
Comment thread internal/execute/watcher.go Resolved
Comment thread internal/execute/watcher.go Resolved

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comment thread internal/execute/watcher.go Resolved
Comment thread internal/execute/watcher.go Resolved
Comment thread internal/execute/watcher.go Resolved

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main issues here are

  • Watch registrations are only derived from the starting tsconfig file—program files and failed lookups (together, everything in “seen files”) never get watched
  • Those watch registrations are never updated—subscribe() is only called once on startup and sets up all the watches from the initial state. If the tsconfig changes (or the set of seen files, once that’s added into consideration), the watched locations need to be reevaluated.

Tests aren’t catching any of this because they call DoCycle manually—they check what happens after a rebuild gets triggered, but they never check what triggers you ask for. There are various ways you could handle this on the testing front; one would be to define an interface that matches the shape of fswatch.Watcher, allow that to be passed to execute.createWatcher, and use a mock implementation generated by moq in tests. That would let you inspect what watcher.WatchDirectory calls are made by subscribe.

Architecturally, I think I expected vfswatch to be completely deleted. With polling, you needed a complete snapshot of watched files and directories so you could detect whether anything changed between two cycles. But that responsibility has been completely removed—you now know your callback won’t happen unless something has changed that you requested to be notified about. Even if your watch registrations are overly broad, the fswatch library tells you what changed, so you can filter those events against the program, the tsconfig, and the set of seen files to make sure the event is relevant. So I don't think the snapshot of file system state is necessary anymore.

So to recap/restate, watch registrations should be part of the continuous cycle with building, not just a one-time setup, and after that, the decision of whether to build can get much simpler. It should look something like this:

flowchart TD
    A[New build] --> B[Discover current build dependencies]
    B --> C[Register / reconcile watches]
    C --> D[Wait for fswatch event]
    D --> E[Receive event]
    E --> F[Filter event against current build]
    F -->|Relevant change| A
    F -->|Irrelevant change| D
    F -->|Overflow / uncertain| A
Loading

@RyanCavanaugh Ryan Cavanaugh (RyanCavanaugh) added the No linked issue This PR doesn't say what bug it fixes label Jun 8, 2026
@johnfav03 John Favret (johnfav03) removed the No linked issue This PR doesn't say what bug it fixes label Jun 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me... I think Jake Bailey (@jakebailey) was taking a look as well. Nice work!

@jakebailey

Copy link
Copy Markdown
Member

I didn't have time to look or test before I had to leave for the day but I intend to look in the morning

@jakebailey

Copy link
Copy Markdown
Member

I tested this on src/compiler. Things worked normally, but they did not if I rm -rf node_modules and then npm ci.

It detects the delete and I get:

[10:06:48 AM] File change detected. Starting incremental compilation...

error TS2688: Cannot find type definition file for 'node'.
  The file is in the program because:
    Entry point of type library 'node' specified in compilerOptions
  src/compiler/tsconfig.json:4:19 - File is entry point of type library specified here.
    4         "types": ["node"]
                        ~~~~~~


Found 1 error.

[10:06:48 AM] Found 1 error. Watching for file changes.

But then npm ci does not cause any more updates.

Comment thread internal/execute/tsc.go Resolved
Comment thread internal/execute/watcher.go Resolved
Comment thread internal/execute/watcher.go Resolved
Comment on lines +435 to +439
p := tspath.NormalizeSlashes(path)
return strings.HasSuffix(p, "/.git") ||
strings.Contains(p, "/.git/") ||
strings.Contains(p, "/node_modules/.") ||
strings.Contains(p, "/.#")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully this isn't too inefficient, since this is many many passes on the strings done each time

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I can tell this is the most efficient way to handle this without preventing registration of these paths in fswatch directly.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a big deal, just noting it so I remembered

Comment thread internal/execute/watcher.go Resolved

@jakebailey Jake Bailey (jakebailey) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the node_modules remove and install and it works, while the editor funnily did not.

I think that the "should watch" logic is fine, copied from Strada. I think this is worth getting out there now.

@johnfav03
John Favret (johnfav03) added this pull request to the merge queue Jun 12, 2026
@jakebailey
Jake Bailey (jakebailey) removed this pull request from the merge queue due to a manual request Jun 12, 2026
@johnfav03
John Favret (johnfav03) added this pull request to the merge queue Jun 12, 2026
Merged via the queue into microsoft:main with commit bb2240e Jun 12, 2026
21 checks passed
@johnfav03
John Favret (johnfav03) deleted the cli-watcher-parcel-integration branch June 12, 2026 21:41
Christian Vuerings (christianvuerings) added a commit to christianvuerings/typescript-go that referenced this pull request Jun 12, 2026
* origin/main:
  Make LS checkers time out after being idle, segment based on use (take 2) (microsoft#3435)
  Fix external helper crash after module-status updates (microsoft#3994)
  Allow LSP server to use builtin watcher on Windows and macOS (microsoft#4288)
  Rebuilt CLI watcher around new `fswatch` package (microsoft#4026)
  Fix stack overflow crash in base constraint resolution (microsoft#4242)

# Conflicts:
#	internal/project/session.go
Jake Bailey (jakebailey) pushed a commit to jakebailey/TypeScript that referenced this pull request Aug 13, 2026
…t-go#4026)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Elevated CPU usage in --watch mode when idle (compared to tsc)

6 participants