assert: prevent OOM when generating diff for large objects

[AbdelrahmanHafez]

Fixes #61346

This PR:

• Truncates inspectValue() output in assertion_error.js before diffing. The limit scales with system memory via os.totalmem(): 512KB under 1GB, 1MB under 2GB, 2MB otherwise. Truncation cuts at the last complete line, adds a ... [truncated] marker, and the error message indicates lines were skipped.
• Chunks the myers diff for inputs over 512 lines. Instead of splitting at fixed positions (which produces phantom diffs when the sides have different lengths), chunk boundaries are placed on "anchor" lines that appear in both arrays at roughly the same relative position, verified against an adjacent line to avoid anchoring on lines that just happen to look similar. Boundaries step along the larger remaining side, so every chunk is bounded on both sides even when one side is much larger than the other (e.g. deepStrictEqual(hugeObject, { a: 1 })), which keeps the O((N + M) * D) memory of each chunk's diff bounded.
• Uses a distinct error message when both inspect outputs are truncated to an identical prefix, since claiming the values have the same structure would be wrong there.

The assertion logic is unaffected; only the error output changes.

Trade-offs

If both objects produce very large inspect output and differ only beyond the truncation limit, the diff can't show the difference. The error message says the output was truncated before any difference was found, and error.actual / error.expected are still available for programmatic inspection. The alternative is an OOM crash, which is worse.

When no anchor line is found near a chunk boundary (e.g. completely different content), the split falls back to a proportional position. The diff stays correct, just not minimal around that boundary.

Benchmarks for the full assert paths are in #61347 (comment), the chunking overhead doesn't show up at the API level since util.inspect dominates.

[AbdelrahmanHafez]

@avivkeller
Could you approve the CI run to verify if something's broken? I'd like to fix any potential issues tonight if possible.

[nodejs-github-bot]

CI: https://ci.nodejs.org/job/node-test-pull-request/70780/

[AbdelrahmanHafez]

I've investigated the CI failures:

GitHub Actions failures (coverage-linux, test-linux, test-tarball-linux, x86_64-linux): All terminated with exit code 143 due to runner shutdown signals, not test failures. The tests that completed before shutdown all passed.

##[error]The runner has received a shutdown signal.
##[error]Process completed with exit code 143.

Jenkins CI failures (node-test-commit-linuxone, node-test-commit-plinux, node-test-linux-linked-*): These show "tests failed" but I cannot access the logs to verify if they're related to this PR or infrastructure issues.

Could a maintainer please re-trigger CI or check the Jenkins logs? @avivkeller

[BridgeAR]

Suggestion: split the string into chunks of 512kb and compare each chunk and combine the output. The output will not be perfect, as the combining process would sometimes be weird, while the smaller chunks should actually be much much faster to inspect in full.

[AbdelrahmanHafez]

Good idea, will experiment with that and compare DX, and performance for any objects that are over 512kb. 👍

[AbdelrahmanHafez]

@BridgeAR
I experimented with the chunking approach. Here's what I found:

For large objects (>1MB inspect output), chunking does provide a speedup (1.3-1.5x) when there are many scattered differences. Memory usage per chunk is lower, which helps prevent OOM.

But the problem is that chunks get misaligned. We split actual and expected independently by byte size, but if the values differ in length, the boundaries shift:

ACTUAL CHUNKS                    EXPECTED CHUNKS

Chunk 0:
    prop_01: 'aaa',              prop_01: 'aaa',
    prop_02: 'bbb',              (empty)

Chunk 1:
    prop_03: 'ccc',              prop_02: 'MUCH_LONGER_VALUE',
    prop_04: 'ddd',              (empty)

Chunk 2:
    prop_05: 'eee',              prop_03: 'ccc',
    prop_06: 'fff',              prop_04: 'ddd',

When we diff Chunk 1 (actual) vs Chunk 1 (expected), we're comparing [prop_03, prop_04] against [prop_02]. The diff shows fake inserts/deletes at every boundary.

For the typical use case, both approaches are equally fast (1ms each on my machine) because Myers diff is efficient when there are few actual changes.

I'm leaning towards the 2MB truncation approach. It skipped = true results in a less confusing output than phantom diffs when the object is too large, and the diff output is straightforward without these artifacts.

What do you think?

[BridgeAR]

Thank you for giving it a try!

There is absolutely a bigger likelihood for the chunks to misalign / create diffs that are not needed. That could be reduced by reading the last chunk start and using that for the actual splitting (adding the additional part). That would prevent most of these cases, while absolutely not all (especially, if the diff would be across multiple chunks).

All that said, I am fine with the 2mb truncation, since we do not want to show such a big diff anyway and it is simpler.

One problem that stays with 2mb: the available memory is depending on the available resources at that time on the machine.

[AbdelrahmanHafez]

I'll experiment a little more some time next week if we can calculate diffs in a more robust way that handles the chunks misalignment, will keep you posted here. 👍

[BridgeAR]

The current test is actually so heavy, that it still fails on systems with low memory or a slow CPU. We should as such use a dynamic limit where we take into account system resources. I think on some systems a one MB limit should be max.

We could also skip it there, while we do not really gain much safety on those systems, if we do so.

[AbdelrahmanHafez]

Hi @BridgeAR

I took a different approach to the chunking to work around the misaligned chunks giving phantom diffs. Instead of splitting at fixed positions, I find matching "anchor" lines near each chunk boundary, lines that appear in both arrays at roughly the same relative position. To avoid anchoring on lines that just happen to look similar (like }, appearing everywhere), I skip trivial structural lines and verify that at least one adjacent line also matches (hasAdjacentMatch). This makes phantom diffs very unlikely in practice.

Here's a quick overview of how it works:

• myersDiff is still the public entry point, same signature. For inputs under 512 lines (the common case), it delegates straight to the original algorithm with zero overhead.
• For larger inputs, findAlignedBoundaries walks through the actual array at ~512-line intervals and uses findAnchorNear to search outward for a non-trivial line that exists in both arrays (with proportional mapping to the expected side). Each anchor becomes a chunk boundary.
• If no anchor is found (e.g. completely different content), it falls back to a proportional split. The diff is still correct, just not minimal at the boundary.
• Each chunk is then diffed independently with the original Myers algorithm, and results are concatenated in reverse order (matching the existing output format).

I also addressed the other feedback:

• kMaxInspectOutputLength is now dynamic based on os.totalmem(): 512KB for systems under 1GB RAM, 1MB under 2GB, 2MB otherwise. This should help with the test failing on low-memory CI machines.
• The OOM test skips on systems with less than 1GB total memory via os.totalmem().

I benchmarked the full assert.deepStrictEqual / assert.strictEqual paths (inspect + diff + error construction) on main vs this branch. The diff is a small fraction of the total assert cost since util.inspect dominates, so the chunking overhead doesn't show up at the API level:

Scenario	main (ms)	PR (ms)	Change
deepStrictEqual success, small object (10 fields)	0.002	0.001	-50%
deepStrictEqual success, large object (200 fields)	0.042	0.041	-2%
deepStrictEqual failure, small object (10 fields)	0.034	0.037	+9%
deepStrictEqual failure, medium object (50 fields)	0.069	0.068	-1%
deepStrictEqual failure, large object (200 fields)	0.200	0.195	-3%
deepStrictEqual failure, nested object (depth 4, breadth 3)	0.218	0.200	-8%
deepStrictEqual failure, array (100 items)	0.267	0.261	-2%
deepStrictEqual failure, array (500 items)	1.183	1.146	-3%
strictEqual failure, long strings	0.023	0.023	0%

Apple Silicon, Node v24.12.0, 50 iterations with trimmed mean of middle 60%.

Benchmark script (for reproducing)

'use strict';

const assert = require('assert');

const WARMUP = 10;
const ITERATIONS = 50;

const scenarios = [
  {
    name: 'deepStrictEqual success, small object (10 fields)',
    setup() {
      const obj = createObject(10);
      return [() => assert.deepStrictEqual(obj, { ...obj })];
    },
  },
  {
    name: 'deepStrictEqual success, large object (200 fields)',
    setup() {
      const obj = createObject(200);
      return [() => assert.deepStrictEqual(obj, { ...obj })];
    },
  },
  {
    name: 'deepStrictEqual failure, small object (10 fields, 1 different)',
    setup() {
      const actual = createObject(10);
      const expected = { ...actual, field_5: 'CHANGED' };
      return [() => {
        try { assert.deepStrictEqual(actual, expected); } catch {}
      }];
    },
  },
  {
    name: 'deepStrictEqual failure, medium object (50 fields, 3 different)',
    setup() {
      const actual = createObject(50);
      const expected = {
        ...actual, field_10: 'X', field_25: 'Y', field_40: 'Z',
      };
      return [() => {
        try { assert.deepStrictEqual(actual, expected); } catch {}
      }];
    },
  },
  {
    name: 'deepStrictEqual failure, large object (200 fields, 5 different)',
    setup() {
      const actual = createObject(200);
      const expected = { ...actual };
      for (let i = 0; i < 5; i++) {
        expected['field_' + (i * 40)] = 'CHANGED_' + i;
      }
      return [() => {
        try { assert.deepStrictEqual(actual, expected); } catch {}
      }];
    },
  },
  {
    name: 'deepStrictEqual failure, nested object (depth 4, breadth 3)',
    setup() {
      const actual = createNestedObject(4, 3);
      const expected = createNestedObject(4, 3);
      expected.child_1.child_0.child_2 = { value: 'DIFFERENT' };
      return [() => {
        try { assert.deepStrictEqual(actual, expected); } catch {}
      }];
    },
  },
  {
    name: 'deepStrictEqual failure, array (100 items, 2 different)',
    setup() {
      const actual = createArray(100);
      const expected = createArray(100);
      expected[30].name = 'CHANGED_30';
      expected[70].name = 'CHANGED_70';
      return [() => {
        try { assert.deepStrictEqual(actual, expected); } catch {}
      }];
    },
  },
  {
    name: 'deepStrictEqual failure, array (500 items, 5 different)',
    setup() {
      const actual = createArray(500);
      const expected = createArray(500);
      for (let i = 0; i < 5; i++) expected[i * 100].name = 'CHANGED';
      return [() => {
        try { assert.deepStrictEqual(actual, expected); } catch {}
      }];
    },
  },
  {
    name: 'strictEqual failure, long strings',
    setup() {
      const actual = 'a'.repeat(500) + 'ACTUAL' + 'b'.repeat(500);
      const expected = 'a'.repeat(500) + 'EXPECTED' + 'b'.repeat(500);
      return [() => {
        try { assert.strictEqual(actual, expected); } catch {}
      }];
    },
  },
];

// --- Main ---

const header =
  'Scenario'.padEnd(58) +
  'Avg (ms)'.padStart(10) +
  'Median'.padStart(10);

console.log('Assert Benchmark');
console.log(
  `Warmup: ${WARMUP}, Iterations: ${ITERATIONS}` +
  ' (trimmed mean of middle 60%)\n',
);
console.log(header);
console.log('-'.repeat(header.length));

for (const scenario of scenarios) {
  const [fn] = scenario.setup();
  const r = measure(fn);
  console.log(
    scenario.name.padEnd(58) +
    r.avg.toFixed(3).padStart(10) +
    r.median.toFixed(3).padStart(10),
  );
}

// --- Helpers ---

function measure(fn) {
  for (let i = 0; i < WARMUP; i++) fn();

  const times = [];
  for (let i = 0; i < ITERATIONS; i++) {
    const start = process.hrtime.bigint();
    fn();
    const end = process.hrtime.bigint();
    times.push(Number(end - start) / 1e6);
  }

  times.sort((a, b) => a - b);
  const trimStart = Math.floor(times.length * 0.2);
  const trimEnd = times.length - trimStart;
  const trimmed = times.slice(trimStart, trimEnd);
  const avg = trimmed.reduce((a, b) => a + b, 0) / trimmed.length;
  const median = times[Math.floor(times.length / 2)];
  return { avg, median };
}

function createObject(fieldCount) {
  const obj = {};
  for (let i = 0; i < fieldCount; i++) {
    obj['field_' + i] = 'value_' + i;
  }
  return obj;
}

function createNestedObject(depth, breadth) {
  if (depth === 0) return { value: 'leaf' };
  const obj = {};
  for (let i = 0; i < breadth; i++) {
    obj['child_' + i] = createNestedObject(depth - 1, breadth);
  }
  return obj;
}

function createArray(length) {
  return Array.from({ length }, (_, i) => ({
    id: i, name: 'item_' + i, active: i % 2 === 0,
  }));
}

[AbdelrahmanHafez]

Kind reminder @BridgeAR @avivkeller
I hit this issue a couple more times while working on my projects, would love to see a fix get shipped for it.

[AbdelrahmanHafez]

@BridgeAR @avivkeller I've updated the PR, chunk boundaries now step along the larger remaining side, so asymmetric comparisons (huge object vs tiny one) stay bounded too, and I rebased onto main. I ran the test workflows on my fork and everything passes apart from test-os-checked-function, which fails on nightly main as well. Could you trigger a CI run when you get a chance?