difflib.HtmlDiff produces different output from difflib.ndiff

@loewis

BPO	27709
Nosy	@loewis, @terryjreedy, @j, @tirkarthi

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-08-08.15:40:57.110>
labels = ['type-bug', 'library']
title = 'difflib.HtmlDiff produces different output from difflib.ndiff'
updated_at = <Date 2018-09-22.18:00:18.520>
user = 'https://bugs.python.org/SilentGhost'

bugs.python.org fields:

activity = <Date 2018-09-22.18:00:18.520>
actor = 'xtreak'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2016-08-08.15:40:57.110>
creator = 'SilentGhost'
dependencies = []
files = []
hgrepos = []
issue_num = 27709
keywords = ['patch']
message_count = 2.0
messages = ['272173', '272196']
nosy_count = 6.0
nosy_names = ['loewis', 'terry.reedy', 'eli.bendersky', 'SilentGhost', 'jlwing', 'xtreak']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27709'
versions = ['Python 3.5', 'Python 3.6']

Linked PRs

msg264842 in bpo-26945 reports an odd results of HtmlDiff.make_file, digging into it I've noticed couple of things: different output to one generated directly from difflib.ndiff and underlying issue further up the stack that generates same faulty output with a slightly modified files.

The patch I'm attaching fixes the issue with different outputs, from make_file and ndiff, by not removing EOL characters, I had to re-generate expected test output file.

The degenerate behaviour appears ultimately due to the autojunk heuristic. Given that autojunk defaults to True and there isn't any way to change via most user-facing functions, I don't think there is an easy resolution that can be found here. I'm going to withdraw the earlier patch, since it feels like a half-measure which enable correct behaviour by accident.

As described in the #118150 issue and handled there, yes the autojunk behavior is surprising at times.
But there is also another issue here (reproduced using 3.13, 3.14 and 3.15):

import difflib

A = ["same\n", "diffme\n"]
B = ["same", "diffme\n"]   # line 0 differs only by trailing newline

print(difflib.SequenceMatcher(None, A, B).get_opcodes())  # [('replace', 0, 1, 0, 1), ('equal', 1, 2, 1, 2)]
print(list(difflib.ndiff(A, B)))  # ['- same\n', '?     -\n', '+ same', '  diffme\n']
print(list(difflib.unified_diff(A, B, lineterm='')))  # [..., '-same\n', '+same', ' diffme\n']
print(list(difflib.context_diff(A, B, lineterm='')))  # [..., '! same\n', ..., '! same', '  diffme\n']
print(difflib.HtmlDiff().make_file(A, B))  # no line is marked as changed/added/removed (classes diff_add/diff_chg/diff_sub)

A_stripped = [elem.rstrip() for elem in A]
B_stripped = [elem.rstrip() for elem in B]

print(difflib.SequenceMatcher(None, A_stripped, B_stripped).get_opcodes())  # [('equal', 0, 2, 0, 2)]
print(list(difflib.ndiff(A_stripped, B_stripped)))  # ['  same', '  diffme']
print(list(difflib.unified_diff(A_stripped, B_stripped, lineterm='')))  # []
print(list(difflib.context_diff(A_stripped, B_stripped, lineterm='')))  # []
print(difflib.HtmlDiff().make_file(A_stripped, B_stripped))  # still no line is marked as changed/added/removed

The HtmlDiff is the only one that does end-of-line stripping from the whole difflib. To me, this is a bug because it hides actual differences.
It is caused by the _tab_newline_replace method being called before _mdiff.

Petr Viktorin just told me we should NOT fix the issue (which dates back tot he original implementation in 2004, 22 years ago, cf e064b41). Instead, I will add a warning about it, and an automated test to prevent unexpected regression (if someone would fix the bug).

gh-71896: Add a docs warning about trailing newlines ignored by `diff…

[3.15] gh-71896: Add a docs warning about trailing newlines ignored b…

[3.13] gh-71896: Add a docs warning about trailing newlines ignored b…

[3.14] gh-71896: Add a docs warning about trailing newlines ignored b…

Navigation Menu

Repository navigation

difflib.HtmlDiff produces different output from difflib.ndiff #71896

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

difflib.HtmlDiff produces different output from difflib.ndiff #71896

Description

Linked PRs

Activity

SilentGhost commented on Aug 8, 2016

SilentGhost commented on Aug 8, 2016

Lenormju commented on Jul 18, 2026

Lenormju commented on Jul 18, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions