Skip to content

Keep extracted auto-import packages across cancelled builds - #65412

Open
Andrea Cappuccio (hood) wants to merge 1 commit into
microsoft:mainfrom
hood:perf/autoimport-partial-extraction
Open

Andrea Cappuccio (hood) wants to merge 1 commit into
microsoft:mainfrom
hood:perf/autoimport-partial-extraction

Conversation

@hood

Copy link
Copy Markdown
Contributor

Fixes #65410

While typing, the auto-import warm-up gets cancelled on every keystroke and used to throw away every package it had already extracted, so it kept re-reading and re-parsing node_modules from scratch and never finished if you type faster than one full build. This keeps per-package extraction results on the registry, shared from one clone to the next, so a cancelled build leaves its finished packages for the next one:

  • a package extraction that has started runs to completion even if the build gets cancelled, and concurrent builds wait on the in-flight one instead of starting it again (single-flight per realpath + package name + directory-search flag). New packages still aren't started after cancellation.
  • the kept results get dropped as soon as anything under node_modules or under a kept package's own directory changes (workspace packages), when bucket build preferences or project-reference outputs change, and once a build completes, so memory goes back to what it was once the index is ready.
recharts src/cartesian/Line.tsx, 120 keystrokes at 150 ms, diagnostics + semantic tokens per key main (7 runs) this PR (3 runs)
process CPU over 18 s of typing 42.6–44.8 s 13.5–14.0 s (−68%)
first completion after typing stops 217–257 ms (index never got built) 11–13 ms (index ready)
textDocument/diagnostic median 53–55 ms 47.5–50 ms
completion-heavy typing (completion + diag + semantic tokens per key), CPU ~17.9 s ~17.5–17.9 s (no change)

Measured with this harness: https://gist.github.com/hood/506e87027c83bbec1b44c1f687273349

TYPING_DIR=<recharts> TYPING_FILE=src/cartesian/Line.tsx TYPING_LINE=408 \
  TYPING_ONLY=diagnostic,semanticTokens TYPING_INTERVAL=150ms TYPING_REPS=2 TYPING_POSTCOMPLETE=1 \
  ./lspbench -test.run TestTypingBench

Tests:

  • new TestCancelledBuildKeepsExtractedPackages: a build cancelled mid-extraction doesn't extract that package again on the next build (fails on main), and a node_modules change still re-extracts it.
  • completion lists are identical to main on the same session across 13 prefixes: 21,610 items, 6,335 of them auto-imports.
  • go test ./..., npx hereby lint and npx hereby check:format pass.

AI disclosure: I polished my solution, my breakdown of it and -most importantly- did most of the measurements with help from an AI agent. I've gone 2x through everything and I'll handle any follow-up myself ofc.

The auto-import warm-up is cancelled on every keystroke and used to drop
every package it had already extracted, so while typing it re-read and
re-parsed node_modules over and over without ever finishing. Started
extractions now run to completion and are shared with later builds until
node_modules, a kept package's directory or the build inputs change.
Copilot AI balanced review requested due to automatic review settings November 9, 2026 20:45
@github-project-automation github-project-automation Bot moved this to Not started in PR Backlog Nov 9, 2026
@typescript-automation typescript-automation Bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Nov 9, 2026
@jakebailey Jake Bailey (jakebailey) added the Domain: Performance Performance related label Nov 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Active cache invalidation gaps can reuse stale exports after workspace edits, while cancelled warm-ups can repeatedly restart extraction because compatible cache replacements are not retained.

3 open issues
What changed in this PR

Addresses #65410 by retaining package extraction work across cancelled auto-import builds, reducing repeated parsing while typing in the language server.

Changes:

  • Shares completed and in-flight package extractions across registry clones.
  • Adds cache invalidation for file and build-input changes, plus cleanup after successful builds.
  • Adds regression tests for reuse after cancellation and invalidation after node_modules changes.
File Description
tsc/​internal/​ls/​autoimport/​registry.go Adds shared extraction caching, single-flight coordination, and invalidation.
tsc/​internal/​ls/​autoimport/​registry_test.go Tests cancelled-build reuse and re-extraction after package changes.

Review effort: Balanced

Comment on lines +386 to +388
p.results.Range(func(key extractionKey, _ *pendingExtraction) bool {
affected = caseSensitivity.ContainsPath(key.realpath, tspath.RootedPath(fileName))
return !affected
Comment on lines +386 to +388
p.results.Range(func(key extractionKey, _ *pendingExtraction) bool {
affected = caseSensitivity.ContainsPath(key.realpath, tspath.RootedPath(fileName))
return !affected
Comment on lines +985 to +988
if !partial.usableFor(buildPreferences, projectReferenceOutputs) {
partial = &partialExtractions{}
partial.usableFor(buildPreferences, projectReferenceOutputs)
b.partialExtractions = partial

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Domain: Performance Performance related For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

Status: Not started

Development

Successfully merging this pull request may close these issues.

Auto-import warm-up restarts from scratch on every keystroke while typing

3 participants