What's Changed

• Bump acorn from 5.7.3 to 5.7.4 by @dependabot in https://github.com/sasstools/scss-tokenizer/pull/33
• Fixed ReDoS in "loadAnnotation" function of "previous-map.js" by @sushantmittal in https://github.com/sasstools/scss-tokenizer/pull/49

New Contributors

• @dependabot made their first contribution in https://github.com/sasstools/scss-tokenizer/pull/33
• @sushantmittal made their first contribution in https://github.com/sasstools/scss-tokenizer/pull/49

Full Changelog: https://github.com/sasstools/scss-tokenizer/compare/v0.4.2...v0.4.3

Misc

• Restore prepublishOnly npm script (@xzyfer, ae79d74)

Fixes

• Error handling utf-8 encoded sourcemap urls (#13, @xzyfer)

Fixes

• Fix endInterpolant not being captured (@xzyfer, #20)
• Fix tracking of token position (@xzyfer, #22)

Tests

• Add basic test coverage to interpolant tokenization (@xzyfer, #20)

Dependencies

• Replace babel-preset-es2015 with babel-preset-env (@xzyfer, #23)
• [email protected] (@xzyfer, #23)
• babel-core@^6.26.3 (@xzyfer, #23)
• babel-jest@^23.6.0 (@xzyfer, #23)
• glob@^7.1.3 (@xzyfer, #23)
• jest@^23.6.0 (@xzyfer, #23)
• js-base64@^2.4.9 (@xzyfer, #23)
• source-map@^0.7.3 (@xzyfer, #23)

Misc

• TravisCI now tests all supported Node versions
• Commit package-lock.json

BREAKING CHANGES

• Drop support Node < 6 (@akkumar, #16)
  • required by [email protected]

Tests

• Migrate to jest for tests (@xzyfer, #17)

Dependencies

• [email protected] (@akkumar, #16)
• [email protected] (@akkumar, #16)
• [email protected] (@akkumar, #16)

Remove prepublishOnly npm script

This script is used to transpile the es6 files before publishing but in npm@<5 is runs after the tarball has been created. This results the package being published without the transpiled files 😱

Attempt to fix old npm

Explicitly add index.js to the package.json files array (#12)
This shouldn't be necessary but may resolve an issue with npm@<1.4.0.

See #12

Fix /

Characters we're being swallowed after / which broken the cases
like @import foo/bar;. The second ident bar would be tokenized
as only r.

See #7

Fix commentEnd

The comment tokenizer was failing to exit when comments were missing trailing whitespace before */.

See #6

Improved comment parsing

CSS style comments (/* */) are allow interpolants (#{$foo}),
and disallow everything else.

See #5

Improved Node support

Remove some old dependencies that no long worked with Node >= 4 (b40c729) and test against more Node engines in travis (79dd087).

Add tests

Begin the test suite. It's still rudimentary (a87c091).

Remove undocumented git submodule

The sass-spec suite was being submoduled in for the tests. They have recently created an npm package we're now using that (50c21ea).

Install

npm install scss-tokenizer

Usage

scss.tokenize(css)

API

tokenize

Tokenizes source css and returns an ordered array of tokens with positional
data.

var tokenize = require('scss-tokenizer');
var tokens = tokenize.tokenize(css);

Arguments:

• css (string|#toString): String with input CSS or any object
  with toString() method, like file stream.
• opts (object) optional: options:
  • from: the path to the source CSS file. You should always set from,
    because it is used in map generation and in syntax error messages.