Skip to content

Do not emit multiple UTF-8 BOM markers - #2136

Open
arronKler wants to merge 1 commit into
postcss:mainfrom
arronKler:fix/2133-duplicate-bom
Open

Do not emit multiple UTF-8 BOM markers#2136
arronKler wants to merge 1 commit into
postcss:mainfrom
arronKler:fix/2133-duplicate-bom

Conversation

@arronKler

@arronKler arronKler commented Aug 14, 2026

Copy link
Copy Markdown

Why

v8.5.24 started restoring a leading BOM on stringify (#2119). That is correct for a single file, but parse/stringify could still produce more than one U+FEFF:

  • input with several leading BOMs kept extras after the first
  • concatenated CSS (a{}\uFEFFb{}) re-emitted the mid-file marker as part of the next selector
  • a Document of several BOM-tagged roots wrote a BOM before every root

Those extra markers show up in the middle of generated CSS and break the first rule after each one.

Solution

  • Strip every leading U+FEFF / U+FFFE in Input, not only the first
  • Skip stray BOM markers in the tokenizer so they are not part of the AST
  • Emit at most one BOM from Stringifier#root

A single leading BOM is still preserved. parse('\uFEFFa{}').toString() remains \uFEFFa{}.

Fixes #2133

@arronKler
Do not emit multiple UTF-8 BOM markers
d3ecc2c 
8.5.24 restored a leading BOM on stringify. Concatenated CSS and
documents with several BOM-tagged roots then contained U+FEFF in the
middle of the output.

Strip every leading BOM on parse, skip stray markers while tokenizing,
and emit at most one BOM when stringifying.

Fixes postcss#2133
ai
ai reviewed Aug 14, 2026
View reviewed changes
Comment thread lib/tokenize.js

function nextToken(opts) {
if (returned.length) return returned.pop()
skipBom()

@ai ai Aug 14, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need it if we already have this.css.replace(/^[\uFEFF\uFFFE]+/, '')?

This was referenced Aug 14, 2026
Merged
Open
@ai

ai commented Aug 16, 2026

Copy link
Copy Markdown
Member

@romainmenke found an issue with current approach

This removal isn't correct as U+FEFF is only a BOM if found at the start and has another (deprecated) meaning if found elsewhere, as a zero width no-break space.

But we still can think of removing U+FEFF in the context where U+FEFF is illegal anyway

@hannaseithe hannaseithe mentioned this pull request Aug 25, 2026
Merged
@MaciejZet

MaciejZet commented Oct 17, 2026

Copy link
Copy Markdown

I tested d3ecc2c against its base and current main (f1aca7e) locally on macOS.

Chrome 155 confirms the earlier U+FEFF concern: parse('a { grid-area: \uFEFF; }').toString() loses the literal character, leaving an empty value and computed grid-area: auto. Base and main preserve it. The CSS escape \FEFF is preserved by all three.

One remaining Document case: a first root from parse('a{}'), followed by parse('\uFEFFb{}'), still produces a{}\uFEFFb{} with this PR. This gap also exists on the base; current main produces a{}b{}.

The failed Node 14 job resolved postcss-parser-tests@8.10.0, versus 8.9.0 in the lockfile. Switching only those fixtures on Node 14/TypeScript 4.9.5 reproduces the same semicolons.css failure on both base and PR; main passes. That failure is therefore separate from the BOM change.

With locked dependencies, the PR passes all 691 unit tests on Node 10/12/14/16/18/20/22/24/26 and the full pnpm test on Node 26. I haven't reproduced the reporter's webpack/Sass build.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ai ai ai left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Multiple BOM symbols inside generated css in v8.5.24

3 participants

@arronKler @ai @MaciejZet