feat: make no-irregular-whitespace language-agnostic

[xbinaryx]

Summary

Make the no-irregular-whitespace rule language-agnostic to prevent crashes on non-JavaScript files (e.g., CSS, JSON, Markdown), while accurately reporting irregular whitespace where applicable.

Related Issues

eslint/eslint#19805

[DMartens]

If we standardize the comments property (for which I am in favor of), we should also update TextSourceCodeBase from @eslint/plugin-kit (and update the source codes of the our languages respectively). For Markdown this would either be setting it to an empty array or extracting the HTML comments.

[mdjermanovic]

@DMartens can you provide more details on what should be updated in TextSourceCodeBase?

[DMartens]

The documentation for creating languages should also be updated to require the comments property if accepted.

[fasttime]

A peculiarity of the invalid characters "\u2028" and "\u2029" is that JavaScript treats them as line separators, whereas CSS, JSON, and Markdown do not. To calculate report locations correctly for different languages, checkForIrregularLineTerminators would need to account for language-specific line-separators (instead of relying on the hardcoded and JavaScript-specific regular expressions LINE_BREAK and IRREGULAR_LINE_TERMINATORS).

A simpler approach could be to calculate start and end locations from character indices rather than line and column separately, and at that point we could merge checkForIrregularWhitespace and checkForIrregularLineTerminators into a single implementation. For example:

function checkForIrregularWhitespaceOrLineTerminators(node) {
	let match;
	while (
		(match = IRREGULAR_CHARACTERS.exec(sourceCode.text)) !== null
	) {
		errors.push({
			node,
			messageId: "noIrregularWhitespace",
			loc: {
				start: sourceCode.getLocFromIndex(match.index),
				end: sourceCode.getLocFromIndex(match.index + match[0].length),
			},
		});
	}
}

The alternative, I think, would involve exposing the line separators for each language as a new property, but using sourceCode.getLocFromIndex() seems simpler, and it should work out of the box.

[mdjermanovic]

I'm in favor of the approach with checkForIrregularWhitespaceOrLineTerminators(node).

[fasttime]

This would work, but it seem a bit fragile because it assumes that JavaScript is the only language with node types named Literal, TemplateElement or JSXText. Maybe a safer solution is checking (somehow) if the language is JavaScript or TypeScript, and registering the visitors only in that case.

[fasttime]

I also think that standardizing a comments property is the most versatile solution.

[fasttime]

If there is consensus on generalizing the comments property as proposed in the RFC, I think it's also safe to deprecate getAllComments() at the same time. ESLint deprecations are documentation-only, so not much will change for existing consumers.

[mdjermanovic]

I'm in favor of generalizing the comments property and deprecating getAllComments() on JS SourceCode.