fix: parse MIME types with WHATWG - #6919

Follow-up to #6414.

I'm trying to find an approach that makes sense here:

• Keep addContentTypeParser as the place for application-specific handling, including exact string overrides for valid values and explicitly registered RegExp parsers for invalid-header recovery, rather than forcing recovery into an onRequest hook (restored by Allow custom parsers to recover content types #6918).
• Use a parsing and validation algorithm that is strict and standards-based while still providing established recovery behavior for common malformed inputs.

I suggest that Fastify parse Content-Type values using the WHATWG MIME type parsing algorithm.

This:

• recovers usable MIME types from malformed parameter syntax as specified by WHATWG
• continues to reject malformed type/subtype values
• preserves parsed parameters for content-type parser matching and reply charset handling
• moves MIME parsing into a dedicated cursor-based parser
• avoids regular-expression backtracking over parameter lists
• improves microbenchmark performance for parameterized and recovery cases
• uses a well-known algorithm maintained in a WHATWG standard

Fastify currently parses MIME parameters using a collection of regular expressions. That differs from the WHATWG recovery algorithm and can produce inconsistent results for malformed parameter syntax.

The observed webhook input appears to begin with Content-Type being set more than once somewhere in the request path. An HTTP implementation or intermediary then combines those field lines with commas before Fastify receives them:

Content-Type: application/json; charset=utf-8
Content-Type: application/json

becomes:

application/json; charset=utf-8,application/json

The sender, intermediary, or server layer responsible for introducing the duplicate is not necessarily known. Fastify receives only the combined value.

Adopting the WHATWG algorithm also resolves other parsing ambiguities, such as duplicate parameters:

text/html; charset=windows-1252; charset=utf-8

The current Fastify parser lets the last parameter win:

{
  type: 'text',
  subtype: 'html',
  parameters: new Map([
    ['charset', 'utf-8']
  ])
}

WHATWG keeps the first parameter value:

{
  type: 'text',
  subtype: 'html',
  parameters: new Map([
    ['charset', 'windows-1252']
  ])
}

This follows the algorithm's directive to set a parameter only when that name does not already exist, preventing a later duplicate from changing the interpretation.

If the second Content-Type field conflicts with the first, the current parser silently discards it:

application/json; charset=utf-8,text/plain

The current Fastify parser matches the valid parameter prefix and parses this as:

{
  type: 'application',
  subtype: 'json',
  parameters: new Map([
    ['charset', 'utf-8']
  ])
}

The trailing ,text/plain is discarded.

WHATWG preserves the complete parameter value:

{
  type: 'application',
  subtype: 'json',
  parameters: new Map([
    ['charset', 'utf-8,text/plain']
  ])
}

The MIME type remains application/json, but the malformed charset value is no longer silently sanitized. This is nonsensical but more representative of reliantly, so in my opinion, preferable.

This does not make comma-separated media types generally valid:

application/json,application/json

The comma is part of the subtype in that case, so the value remains invalid. String parsers cannot register invalid MIME types, but applications can explicitly recover known invalid values using a RegExp content-type parser.

By contrast, comma-containing parameter values remain valid under WHATWG and use the parser registered for their media type. Applications that need custom behavior for a specific valid value can register an exact string parser.

For example:

fastify.addContentTypeParser(
  'application/json; charset=utf-8,text/plain',
  { parseAs: 'string' },
  customParser
)

Fastify normalizes the registered value and the incoming header to the same MIME type representation, so this exact string parser takes precedence over the generic application/json parser.

The parser follows:

• https://mimesniff.spec.whatwg.org/#parse-a-mime-type
• https://fetch.spec.whatwg.org/#collect-an-http-quoted-string

The implementation uses numeric cursor positions and lazy string slices, but preserves the specified parsing behavior.

Parameter names are validated as HTTP tokens before using native String.prototype.toLowerCase(). Since valid HTTP tokens are ASCII, this is equivalent to the specification's ASCII-lowercase operation while avoiding Unicode case-folding surprises.

Performance

Compared with origin/main at a2f59056 using Node.js 24.18.0.

Each result is the paired median of 12 rounds with 500,000 constructions per round.

Benchmark code, run from the repository root:

'use strict'

const Module = require('node:module')
const { execFileSync } = require('node:child_process')
const { performance } = require('node:perf_hooks')

const repo = process.cwd()
const baseRef = process.env.BASE_REF || 'origin/main'

function compile (code, filename) {
  const compiledModule = new Module(filename, module.parent)
  compiledModule.filename = filename
  compiledModule.paths = Module._nodeModulePaths(`${repo}/lib`)
  compiledModule._compile(code, filename)
  return compiledModule.exports
}

const oldSource = execFileSync(
  'git',
  ['show', `${baseRef}:lib/content-type.js`],
  { cwd: repo, encoding: 'utf8' }
)
const implementations = {
  old: compile(oldSource, `${repo}/lib/content-type-old.js`),
  current: require(`${repo}/lib/content-type.js`)
}
const cases = [
  'application/json',
  'Application/JSON',
  'application/json; charset=utf-8',
  'application/json; charset=utf-8; boundary=abc',
  'application/json; name="foo;bar"; charset=utf-8',
  'application/json; charset=utf-8,evil'
]
const iterations = 500_000
const rounds = 12
let checksum = 0

function run (ContentType, input, count) {
  const start = performance.now()
  for (let index = 0; index < count; index++) {
    checksum += new ContentType(input).isValid
  }
  return count / (performance.now() - start) * 1000
}

function median (values) {
  const sorted = values.toSorted((left, right) => left - right)
  return (sorted[5] + sorted[6]) / 2
}

for (const input of cases) {
  const samples = { old: [], current: [] }
  run(implementations.old, input, 300_000)
  run(implementations.current, input, 300_000)

  for (let round = 0; round < rounds; round++) {
    const order = round % 2 === 0 ? ['old', 'current'] : ['current', 'old']
    for (const name of order) {
      samples[name].push(run(implementations[name], input, iterations))
    }
  }

  const oldResult = median(samples.old)
  const currentResult = median(samples.current)
  const change = (currentResult / oldResult - 1) * 100
  console.log(`${input}\n  old ${oldResult.toFixed(0)} ops/s  new ${currentResult.toFixed(0)} ops/s  ${change >= 0 ? '+' : ''}${change.toFixed(1)}%`)
}

console.log(`checksum ${checksum}`)

Plain MIME types are within benchmark noise of the previous implementation. Parameterized and recovery cases are faster.

Validation

• npm run lint
• npm run unit

Result: 2,310 tests passed, 4 skipped, 0 failed.

Checklist

• run npm run test && npm run benchmark --if-present
• tests and/or benchmarks are included
• documentation is changed or added
• commit message and code follows the Developer's Certification of Origin and the Code of conduct