parse5

This release includes parse5 and parse5-parser-stream.

• Export ERR as ErrorCodes by @milahu in https://github.com/inikulin/parse5/pull/704
• Properly handle CR when peeking by @fb55 in https://github.com/inikulin/parse5/pull/715
• Add CJS build of parser-stream by @fb55 in https://github.com/inikulin/parse5/pull/716

Full Changelog: https://github.com/inikulin/parse5/compare/v7.1.0...v7.1.2

This release is only for the parse5 module.

What's Changed

• feat(parse5): use string types for node names by @43081j in https://github.com/inikulin/parse5/pull/631
• fix(parser): Have </button> close <p> by @fb55 in https://github.com/inikulin/parse5/pull/534
• docs: Fix small typo in code example by @cheeaun in https://github.com/inikulin/parse5/pull/518
• Added coverage reporting by @fb55 in https://github.com/inikulin/parse5/pull/515
• Improved code coverage by @fb55 in https://github.com/inikulin/parse5/pull/517 and https://github.com/inikulin/parse5/pull/547

New Contributors

• @cheeaun made their first contribution in https://github.com/inikulin/parse5/pull/518

Full Changelog: https://github.com/inikulin/parse5/compare/v7.0.0...v7.1.0

Welcome to [email protected]! ✨ This is a huge release with many changes, features and fixes.

From an organisational perspective, the most important change is that parse5 is now maintained by a team, consisting of James (@43081j), Titus (@wooorm) and me (@fb55). We come from three projects that rely on parse5 — namely Cheerio, rehype, and Lit.

We need your support to continue the project! If you care about parse5, please support us financially on OpenCollective.

Headlining features of this release are ES Modules, TypeScript, and performance improvements: 7.0.0 is 45% faster than 6.0.1 with default options, and 167% faster with location information enabled (for the bench/perf benchmark, on an M1 Mac). Version 7.0.0 is a revamp of every part of the library. There are too many changes to list them all here, so here is a high-level overview:

Breaking: ESM

All of parse5’s packages are now ECMAScript Modules. We are providing dual packages for parse5 and parse5-htmlparser2-tree-adapter for now (see https://github.com/inikulin/parse5/pull/418 and https://github.com/inikulin/parse5/pull/496).

To migrate, please read this Gist on how to update. Note that private internals are no longer available; instead, everything that you need should be imported from the main package.

Implemented by @43081j in #351

Breaking: TypeScript

The codebase has been ported to TypeScript. This helped uncover a number of subtle logic bugs, such as dc4e269022ebbae0767d8f790a29d6be1835fe1e, b4b5d4ad6f90b3c9fd03a90e2ed5267929979a11, or a0aff9578bb44511bc169c1d7f9e2f2780f7f8a0. TypeScript also helps us refactor with confidence and a lot of the changes in this release would have been much harder to do without it.

To migrate, please remove @types/parse5* as we now ship our own types.

Implemented by @fb55 in #362

Potentially breaking changes

• parse5 was caught up with the HTML spec, and parsing results might differ in edge-cases (https://github.com/inikulin/parse5/pull/442, https://github.com/inikulin/parse5/pull/451)
• The parse5-serializer-stream package was removed https://github.com/inikulin/parse5/pull/481
  • To migrate, use the serialize function exported by parse5.
• The rewriting stream now splits very long text sections (https://github.com/inikulin/parse5/pull/434) and doesn’t escape text in special tags anymore (#434). If you worked around these issues before, you might have to update your code.
• The htmlparser2 adapter now uses domhandler’s node interface (https://github.com/inikulin/parse5/pull/327 by @TrySound)
  • The format of the tree nodes has changed slightly; eg. some previous properties are now getters and setters, and vice versa.

If you are using deep imports for any parts of the codebase, you will likely encounter some breakages:

• The tokenizer now uses the state machine pattern from htmlparser2 5d7a780 (#362)
• The token queue was replaced with callbacks (https://github.com/inikulin/parse5/pull/404, https://github.com/inikulin/parse5/pull/405, https://github.com/inikulin/parse5/pull/419)
• The OpenElementStack now uses callbacks https://github.com/inikulin/parse5/pull/429
• Mixins were removed (as part of #362)
  • Location tracking now has a substantially lower overhead https://github.com/inikulin/parse5/pull/402
• getNextToken was removed https://github.com/inikulin/parse5/pull/461
• The parser’s _bootstrap method was removed https://github.com/inikulin/parse5/pull/384
• We now drop chunks from the tokenizer right after they are emitted https://github.com/inikulin/parse5/pull/432
• The serializer is no longer a class; instead, different serializer functions call on each other https://github.com/inikulin/parse5/pull/383
• parse5 now uses the entities module for encoding and decoding entities, sharing maintenance & optimisation work with projects such as htmlparser2 (2b92054 (#362), https://github.com/inikulin/parse5/pull/486)
  • entities adopted a variant of parse5’s approach of decoding entities. As a result, decoding performance is equivalent, while memory consumption is slightly lower.

Other changes

• minor add hooks for stack events to tree adapter interface #385
• minor add support for fragments in parse5-parser-stream #487
• minor add serializeOuter (like .outerHTML), scriptingEnabled option #383
• patch fix parsing of << in comments parsed wrongly as <! (#326)
• patch fix position of endTag for mixed-case foreign elements (#353)
• patch fix end position of html, body (#436)
• docs: parse5 has a new documentation website at parse5.js.org #443

New Contributors

Thanks @anko, @TrySound, @samouri, @alan-agius4, and @pmdartus!

Full Changelog: https://github.com/inikulin/parse5/compare/v6.0.1...v7.0.0

• Added (breaking): Tree adapter interface now has updateNodeSourceCodeLocation method which enables usage of custom location info formats (GH #314) (by @DMartens).

• Fixed: Handling of self-closing <hr> tags (by @43081j).
• Fixed: Broken link in TreeAdapter document (GH #317) (by @ursm).
• Fixed: SAXParser example (GH #316) (by @mvasilkov).

• Fixed: Serialization of attributes in non-standard namespaces (by @Zirro).
• Fixed: Quirks and limited-quirks mode detection by doctype (by @squidfunk).

• Fixed: Location info for text events in SAXParser and RewritingStream now contains
  correct endCol and endLine covering all concatenated raw tokens (GH #266).
• Fixed: SAXParser and RewritingStream now flush last buffered chunk when calling .end() with
  no parameters (GH #271).
• Updated (breaking): ParserStream, SAXParser and RewritingStream no longer assume that
  each binary chunk is a valid finished UTF-8 chunk, and instead accept only decoded strings (GH #269).

Starting from this release parse5 functionality will be shipped in separate packages. With parse5 package contatining only basic functionality. Please, refer to the list of packages for more info.

• Updated (breaking): source code location now inserted by tree adapter, so tree adapter developers have control over location info property name. Tree adapters should implement setNodeSourceCodeLocation and getNodeSourceCodeLocation methods. Location info property name added by currently implemented tree adapters has been renamed from __location to sourceCodeLocation (GH #189).

• Updated (breaking): Location info line and col properties have been renamed to startLine and
  startCol
  respectively.

• Updated (breaking): SAXParser now passes token objects to event handlers instead of separate arguments. See SAXParser documentation for more info.
  (GH #247).

• Added: endLine and endCol location info
  properties.

• Added: scriptingEnabled flag to the ParserOptions which controls how <noscript> tags are handled by the parser. (GH #192).

• Added: HTML rewriting stream.
  (GH #222).

• Removed (breaking): parse5 no longer ship TypeScript definitions. Existing TypeScript definitions have been moved to DefinitelyTyped repo. Please, track the PR in the DefinitelyTyped repo for the updates.

This is a major release that delivers few minor (but breaking) changes to workaround recently appeared issues with TypeScript Node.js typings versioning and usage of parse5 in environments that are distinct from Node.js (see https://github.com/inikulin/parse5/issues/235 for the details).

• Updated (breaking): TypeScript were disabled by default. See TypeScript definitions section for the details on how to enable them.
• Updated: API that depends on Node.js specific (namely ParserStream, PlainTextConversionStream, SerializerStream, SAXParser) is now lazily loaded. That enables bundling of the basic functionality for other platforms (e.g. for browsers via webpack).

• Fixed: Loosen the dependency version of @types/node (by @gfx).
• Fixed: Incorrect AST generated if empty string fed to ParserStream (GH #195) (by @stevenvachon).

• Fixed: location.startTag is not available if end tag is missing (GH #181).

• Fixed: MarkupData.Location.col description in TypeScript definition file (GH #170).

• Added: parse5 now ships with TypeScript definitions from which new documentation website is generated (GH #125).
• Added: PlainTextConversionStream (GH #135).
• Updated: Significantly reduced initial memory consumption (GH #52).
• Updated (breaking): Added support for limited quirks mode. document.quirksMode property was replaced with document.mode property which can have
  'no-quirks', 'quirks' and 'limited-quirks' values. Tree adapter setQuirksMode and isQuirksMode methods were replaced with setDocumentMode and getDocumentMode methods (GH #83).
• Updated (breaking): AST collections (e.g. attributes dictionary) don't have prototype anymore (GH #119).
• Updated (breaking): Doctype now always serialized as <!DOCTYPE html> as per spec (GH #137).
• Fixed: Incorrect line for __location.endTag when the start tag contains newlines (GH #166) (by @webdesus).

• Fixed: Fixed incorrect LocationInfo.endOffset for non-implicitly closed elements (refix for GH #109) (by @wooorm).

• Fixed: Incorrect location info for text in SAXParser (GH #153).
• Fixed: Incorrect LocationInfo.endOffset for implicitly closed <p> element (GH #109).
• Fixed: Infinite input data buffering in streaming parsers. Now parsers try to not buffer more than 64K of input data. However, there are still some edge cases left that will lead to significant memory consumption, but they are quite exotic and extremely rare in the wild (GH #102, GH #130)

• Fixed: SAXParser HTML integration point handling for adjustable SVG tags.
• Fixed: SAXParser now adjust SVG tag names for end tags.
• Fixed: Location info line calculation on tokenizer character unconsumption (by @ChadKillingsworth).

• SAXParser (by @RReverser)

  • Fixed: Handling of \n in <pre>, <textarea> and <listing>.
  • Fixed: Tag names and attribute names adjustment in foreign content (GH [#99] (https://github.com/inikulin/parse5/issues/99)).
  • Fixed: Handling of <image>.

• Latest spec changes

  • Updated: <isindex> now don't have special handling (GH #122).
  • Updated: Adoption agency algorithm now preserves lexical order of text nodes (GH #129).
  • Updated: <menuitem> now behaves like <option>.

• Fixed: Element nesting corrections now take namespaces into consideration.

• Fixed: ParserStream accidentally hangs up on scripts (GH #101).

• Fixed: Keep ParserStream sync for the inline scripts (GH #98 follow up).

• Fixed: Synchronously calling resume() leads to crash (GH #98).