Repository navigation
Rebuilt CLI watcher around new fswatch package - #4026
John Favret (johnfav03) merged 27 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request rebuilds tsgo --watch around the new internal/fswatch package (OS filesystem events + debouncing) instead of the previous polling loop, while keeping internal/vfs/vfswatch as a baseline-snapshot change detector for tracked compiler inputs.
Changes:
- Replaces the CLI watch loop with an event-driven fswatch subscription (recursive + ignore filter) and coalesced DoCycle signaling.
- Simplifies
internal/vfs/vfswatchby removing polling/timing/debounce machinery, leaving only snapshot + compare logic. - Introduces a new multi-backend
internal/fswatchimplementation (fanotify/inotify/kqueue/FSEvents/ReadDirectoryChangesW), plus extensive tests and documentation.
Reviewed changes
Copilot reviewed 38 out of 38 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/vfs/vfswatch/vfswatch.go | Removes polling-based watcher loop; keeps snapshot+compare watch-state logic. |
| internal/vfs/vfswatch/vfswatch_test.go | Updates tests for the new NewFileWatcher(fs) constructor. |
| internal/vfs/vfswatch/vfswatch_race_test.go | Updates race/fuzz tests to remove polling-interval related coverage. |
| internal/fswatch/windows.go | Adds Windows backend based on ReadDirectoryChangesW with overlapped I/O. |
| internal/fswatch/watcher.go | Adds core fswatch API surface, watcher lifecycle, per-dir state, and callback filtering. |
| internal/fswatch/walkdir.go | Adds generic walkDir implementation (portable baseline). |
| internal/fswatch/walkdir_windows.go | Adds Windows-optimized walkDir using FindFirstFile/FindNextFile. |
| internal/fswatch/walkdir_unix.go | Adds Unix walkDir using getdents/getdirentries for fewer stats. |
| internal/fswatch/walkdir_test.go | Adds tests validating walkDir behavior (symlinks, perms, entries, callbacks). |
| internal/fswatch/walkdir_other.go | Routes unsupported OSes to the generic walkDir implementation. |
| internal/fswatch/walkdir_dirent_noreclen.go | Adds DragonFlyBSD dirent helpers for reclen/inode extraction. |
| internal/fswatch/walkdir_dirent_linux.go | Adds Linux dirent helpers for reclen/inode extraction. |
| internal/fswatch/walkdir_dirent_fileno.go | Adds FreeBSD/OpenBSD/NetBSD dirent helpers for reclen/inode extraction. |
| internal/fswatch/walkdir_dirent_darwin.go | Adds macOS dirent helpers for reclen/inode extraction. |
| internal/fswatch/testutil_test.go | Adds retry-capable test harness to reduce platform flake impact. |
| internal/fswatch/README.md | Documents fswatch usage, watcher selection, and error semantics. |
| internal/fswatch/LICENSE | Adds upstream MIT license attribution. |
| internal/fswatch/kqueue.go | Adds kqueue backend for BSD/macOS (fd-per-entry model). |
| internal/fswatch/inotify_linux.go | Adds inotify backend for Linux. |
| internal/fswatch/fanotify_linux.go | Adds fanotify backend for Linux with rename probing/fallback. |
| internal/fswatch/fanotify_linux_test.go | Adds Linux fanotify-specific tests (availability, parsing, shutdown, cross-watcher). |
| internal/fswatch/fsevents_darwin.go | Adds macOS FSEvents backend (event classification and lifecycle). |
| internal/fswatch/fsevents_darwin_ffi.go | Adds cgo-free CoreFoundation/CoreServices/libdispatch FFI plumbing and NFC normalization helpers. |
| internal/fswatch/fsevents_darwin_ffi.s | Adds shared assembly trampolines and exported trampoline addresses. |
| internal/fswatch/fsevents_darwin_ffi_arm64.s | Adds arm64 FSEvents Create trampoline + callback shim assembly. |
| internal/fswatch/fsevents_darwin_ffi_amd64.s | Adds amd64 FSEvents Create trampoline + callback shim assembly. |
| internal/fswatch/fsevents_darwin_ffi_arm64_test.go | Adds static disassembly-based test ensuring the arm64 callback only touches safe registers. |
| internal/fswatch/fsevents_darwin_nfd_test.go | Adds macOS Unicode normalization regression tests (NFD/NFC mismatch scenarios). |
| internal/fswatch/event.go | Adds event kinds and per-path coalescing state machine. |
| internal/fswatch/eventlist_test.go | Adds unit tests for event coalescing and atomic drain semantics. |
| internal/fswatch/debounce.go | Adds per-backend debounce mechanism (min/max coalescing windows). |
| internal/fswatch/CHANGES.md | Documents divergence from upstream @parcel/watcher and lists bugfixes/features. |
| internal/fswatch/cgmanifest.json | Adds component governance manifest pointing at upstream source commit/tag. |
| internal/fswatch/canonicalize_other.go | Adds no-op path canonicalization for non-darwin platforms. |
| internal/fswatch/canonicalize_darwin.go | Adds NFC canonicalization for macOS path consistency. |
| internal/execute/watcher.go | Rebuilds CLI watcher to subscribe to fswatch events and signal DoCycle asynchronously. |
| .golangci.yml | Updates linter exclusion pattern to include internal/fswatch. |
This comment was marked as outdated.
This comment was marked as outdated.
e87cc21 to
68df16e
Compare
Andrew Branch (andrewbranch)
left a comment
There was a problem hiding this comment.
The main issues here are
- Watch registrations are only derived from the starting tsconfig file—program files and failed lookups (together, everything in “seen files”) never get watched
- Those watch registrations are never updated—
subscribe()is only called once on startup and sets up all the watches from the initial state. If the tsconfig changes (or the set of seen files, once that’s added into consideration), the watched locations need to be reevaluated.
Tests aren’t catching any of this because they call DoCycle manually—they check what happens after a rebuild gets triggered, but they never check what triggers you ask for. There are various ways you could handle this on the testing front; one would be to define an interface that matches the shape of fswatch.Watcher, allow that to be passed to execute.createWatcher, and use a mock implementation generated by moq in tests. That would let you inspect what watcher.WatchDirectory calls are made by subscribe.
Architecturally, I think I expected vfswatch to be completely deleted. With polling, you needed a complete snapshot of watched files and directories so you could detect whether anything changed between two cycles. But that responsibility has been completely removed—you now know your callback won’t happen unless something has changed that you requested to be notified about. Even if your watch registrations are overly broad, the fswatch library tells you what changed, so you can filter those events against the program, the tsconfig, and the set of seen files to make sure the event is relevant. So I don't think the snapshot of file system state is necessary anymore.
So to recap/restate, watch registrations should be part of the continuous cycle with building, not just a one-time setup, and after that, the decision of whether to build can get much simpler. It should look something like this:
flowchart TD
A[New build] --> B[Discover current build dependencies]
B --> C[Register / reconcile watches]
C --> D[Wait for fswatch event]
D --> E[Receive event]
E --> F[Filter event against current build]
F -->|Relevant change| A
F -->|Irrelevant change| D
F -->|Overflow / uncertain| A
Andrew Branch (andrewbranch)
left a comment
There was a problem hiding this comment.
Looks good to me... I think Jake Bailey (@jakebailey) was taking a look as well. Nice work!
|
I didn't have time to look or test before I had to leave for the day but I intend to look in the morning |
|
I tested this on It detects the delete and I get: But then |
| p := tspath.NormalizeSlashes(path) | ||
| return strings.HasSuffix(p, "/.git") || | ||
| strings.Contains(p, "/.git/") || | ||
| strings.Contains(p, "/node_modules/.") || | ||
| strings.Contains(p, "/.#") |
There was a problem hiding this comment.
Hopefully this isn't too inefficient, since this is many many passes on the strings done each time
There was a problem hiding this comment.
As far as I can tell this is the most efficient way to handle this without preventing registration of these paths in fswatch directly.
There was a problem hiding this comment.
It's not a big deal, just noting it so I remembered
Jake Bailey (jakebailey)
left a comment
There was a problem hiding this comment.
I tried the node_modules remove and install and it works, while the editor funnily did not.
I think that the "should watch" logic is fine, copied from Strada. I think this is worth getting out there now.
* origin/main: Make LS checkers time out after being idle, segment based on use (take 2) (microsoft#3435) Fix external helper crash after module-status updates (microsoft#3994) Allow LSP server to use builtin watcher on Windows and macOS (microsoft#4288) Rebuilt CLI watcher around new `fswatch` package (microsoft#4026) Fix stack overflow crash in base constraint resolution (microsoft#4242) # Conflicts: # internal/project/session.go
…t-go#4026) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The CLI watcher (
tsgo --watch) was previously implemented with a polling-based approach, using aRun()loop - the key disadvantage of this system is that I would still callStat()on tracked files even when they didn't change, which yielded an idle CPU usage of ~5% in thevscoderepo.With the addition of the
fswatchpackage (PR #3980), we can rebuild the watcher around OS-level filesystem events, which results in an idle CPU usage of <1% on average in thevscoderepo.Fixes: #3611