adblocker

• Introduce Config object + remove lists abstractions #111
• Remove bench deps + fix matching by using initiator as sourceUrl #109
• Update benchmarks #108
  • comparison: include request processing for each benchmark
  • Use latest uBlock Origin with WebAssembly enabled
  • Update results from benchmarks and clean-up
  • Add benchmark for regex-based hostname extraction

• Fix stack overflow error on Edge #105
• Update and clean-up travis config #104
• Add rawType attribute to Request class #102

• Clean-up isAnchoredByHostname to be more readable
• Fix memory leak in reverse index (where we would keep a reference to buffer needlessly)
• Make sure Engine does not use more memory than necessary using slice instead of subarray

• [bug] Invalidate serialized engines with ENGINE_VERSION bump #99

• [bug] Restore original buffer size for reverse index creation #98

• [bug] Bump engine version to invalidate existing cache #97

• [perf] Speed-up matching by using raw Uint32Array instead of view #96
  • Average time to process request is now 0.007 ms (instead of 0.028 after implementing lazy loading). This is now even faster than ever (our previous speed was ~0.008 ms per request).

• Bump engine version #95

• [bug] Make sure getId() always returns the same value for a given filter #94
  • Previously the value of ID could change after some internal attribute
    were changed during matching (in particular 'optDomains' and
    'optNotDomains' would be sorted to allow more efficient matching which
    would change the ID).

• [regression] allow unicode characters in filters #93

• Implement lazy loading and compact internal representation #87
  • [BREAKING] serialization module has been removed, instead, each class now
    provides a serialize method as well as a static method deserialize.
  • [BREAKING] FiltersEngine now exposes different methods for update:
    update which expects a diff of filters, updateList and
    updateResources. This API should be a cleared and allows using the
    adblocker without managing filters lists.
  • [BREAKING] ReverseIndex' API dropped the use of a callback to specify
    filters and instead expects a list of filters.
  • [BREAKING] parsing and matching filters can now be done using methods of
    the filters classes directly instead of free functions. For example
    NetworkFilter has a parse and match method (with the same expected
    arguments).
  • ReverseIndex is now implemented using a very compact
    representation (stored in a typed array).
  • toString method of filters should now be more accurate.
  • Addition of numerous unit tests (coverage is now >90%)
• Implement support for :style cosmetic filters #86
  • [BREAKING] getCosmeticsFilters will now return CSS as a single string
    (stylesheet) instead of a list of selectors. This simplifies the usage and
    allows to directly inject this into the page using the method of your
    choice: through content scripts or tabs.injectCSS API.

Quick fix release:

• Fix maximum number of selectors in stylesheet limitation on Chrome #83

This release contains many fixes as well as a few (small) breaking changes.

• Remove tldts dependency and allow to plug any implementation instead #81
  • Adblocker does not include tldts by default anymore
  • APIs now expect either already constructed Request as arguments of both
    hostname and domain when needed (e.g.: getCosmeticsFilters)
  • A makeRequest helper is provided to construct Request objects
  • [BREAKING] engine.match expects a Request as argument
  • [BREAKING] engine.matchAll expects a Request as argument
  • [BREAKING] engine.getCSPDirectives expects a Request as argument
  • [BREAKING] engine.getCosmeticsFilter expects a new domain argument
  • [BREAKING] Request's constructor does not apply default value
    anymore and expects all arguments to be provided and initialized. You
    can now use makeRequest to reproduce the previous behavior of new Request.
• Fix cosmetics injection #79
  • Ignore cosmetic filters with extended syntax
  • Ignore invalid cosmetic filters (using strict validation)
  • Make use of tabs.insertCSS to inject style from background
• Update dependencies #78
• Harden typescript + simplify cosmetic injection #71
• Add support for Ghostery rules #73
  • Add support for bug filter attribute
  • Exceptions can now match filters with bug option
  • Simplify and speed-up serialization for network filters
  • Fix hostname anchor matching by preventing infix match in some corner case

• Update tldts to version 3.1.1

• Fix serialization to include CSP bucket #69
  • [BREAKING] NetworkFilterBucket and ReverseIndex now expect different arguments

• Fix style injection and cosmetic filtering logic #67
  • All cosmetics are now using only one background action (instead of two)
  • No unloading is needed in content-script anymore
  • Simplified and optimized the implementation of CosmeticBucket
  • Internalized the version of serialized engine for auto-invalidation on update
• Fix cosmetic matching (tokenization bug) #65
• Optimize serialization and properly handle unicode in filters #61

• Fix fuzzy matching by allowing tokens of any size #61
• Add support for CSP (Content Security Policy) filters #60
• Add hard-coded circumvention logic (+ IL defuser) #59
  • Simplify 'example' extension
  • Add circumvention module and entry-point in cosmetics injection
  • Clean-up cjs and esm bundles
  • Remove obsolete logic to override user-agent in content-script
  • Simplify travis config (using new pre* hooks)
  • Consolidate 'fetch' module (with metadata about lists)

This release contains massive optimizations as well as a few bug fixes and building improvements:

• Distribute both un-bundled cjs and es6 source #54

• Produce a commonjs build artifact #53

• Update build instructions in README.md #52

• Remove dist folder from source tree #50

• Cosmetics: fix rule matching when hostname is empty #49

• Optimizations #46

  • Requests can now use type as a string or number (e.g.: script or 2).

  // Both are equivalent
  new Request({ type: 2, url, sourceUrl })
  new Request({ type: 'script', url, sourceUrl })
  

  • [BREAKING] format of serialized engine has been changed to store less data
  • [BREAKING] id attribute from filters has been remove, use getId() instead (please note that the id is not stored internally anymore, but generated every time getId() is called).

  // Bad
  filter.id
  // Good
  filter.getId()
  

  • [BREAKING] values returned by getId() will differ from values stored in
    the id attribute for identical filters (the algorithm is now different
    and will do less work).
  • [BREAKING] domains specified in $domains= option are now stored hashed
    instead of as string, and can only be retried in their original form if
    debug flag is used in FiltersEngine
  • [BREAKING] fastTokenizer will now only consider tokens longer than 1
  • [BREAKING] fastTokenizer will now only tokenize up to 2048 characters from URLs
  • [BREAKING] hashes produced by fastHash and fastHashBetween will not
    match what was produced by the same function before this change (the seed
    and hashing algorithm was slightly changed for speed).
  • [BREAKING] un-initialized attributes of filters instances
    (CosmeticFilter and NetworkFilter) will have value undefined
    instead of null or empty string like before. It is recommended
    to use accessors (e.g.: filter.getHostname() instead of
    filter.hostname) to access internal attributes, as they will
    always return consistent types and fall-back to meaningful defaults.

  // Bad
  filter.redirect
  filter.filter
  filter.hostname
  
  // Good
  filter.getRedirect()
  filter.getFilter()
  filter.getHostname()
  

  • [BREAKING] a new Request abstraction supersedes IRequest and
    IRawRequest. This new class offers a more consistent experience to work
    will requests.

  new Request({ url })
  new Request({ url, sourceUrl })
  new Request({ url, sourceUrl, type: 'string' })
  new Request({ url, hostname, domain, type: 'string' })
  

  • [BREAKING] remove support for hosts format (e.g.: 127.0.0.1 domain),
    since servers blocklists can also be exported in hostname anchored format
    (e.g.: ||domain^$third-party). This simplifies the parsing logic.
  • [BREAKING] remove the following unused legacy request types:
    • fromFetch
    • fromDTD
    • fromXLST
    • fromBeacon
    • fromCSP
  • [BREAKING] cpt (Content Policy Type of requests) is now called type,
    to match the terminology of the WebRequestAPI.

  // Bad
  request.cpt
  new Request({ cpt })
  
  // Good
  request.type
  new Request({ type })
  

  • Optimized and simplified implementation of parseJSResource (~4 times faster)
  • Optimized matching of some kinds of filters to prevent any string copy (reduced the number of calls to slice, substr and substring)
  • Optimized buckets ordering by moving matching filters towards the
    beginning of the array. This results in generic filter being tried first.
  • Optimized some classes of filters sharing the same pattern and options,
    with different domains. They are now fused into a single filter. For
    example, the following filters:
    • |https://$script,domain=downloadpirate.com
    • |https://$script,domain=dwindly.io
    • |https://$script,domain=intoupload.net
    • |https://$script,domain=linkshrink.net
    • |https://$script,domain=movpod.in
    • |https://$script,domain=povw1deo.com|povwideo.net|powvideo.net
    • |https://$script,domain=sendit.cloud
    • |https://$script,domain=sfiles.org|suprafiles.me|suprafiles.net|suprafiles.org
    • |https://$script,domain=streamplay.to
    • |https://$script,domain=userscloud.com
    • |https://$script,domain=yourporn.sexy
      will be optimized into: |https://$script,domain=dwindly.io|movpod.in|...|yourporn.sexy
  • tokenize will now allow % as part of tokens for filters
  • CosmeticFilter now support the new +js() syntax to inject scripts
  • NetworkFilter's getTokens() method will now return more tokens in some
    cases. For example, if only one domain is specified in the $domain=
    option, then it can be used as a token (before we would only use the pattern part of each filter to extract tokens).
  • In case a NetworkFilter has no token available (e.g.:
    $image,domain=ads.com), then it can be indexed using the domains
    specified in the $domain= option, if any.
  • Filters of the form *pattern (regex) are now optimized into pattern (plain)
  • Filters of the form |http:// or |https:// or |http*:// are now
    optimized using the newly introduced http and https options. The
    Request instances will now say if they are http or https, and this
    saves string comparisons while matching.
  • Fixed a bug where javascript resources were serialized twice
  • Serialization can now be performed even after engine has been optimized
  • Addition of a serialize method on FiltersEngine class
  • Reverse Index is now created using only one Map instead of two
  • optDomains and optNotDomains are now stored in a compact typed array
    instead of Set and a binary search is used for lookups.
  • Prevent filters from being checked twice for requests by remembering which
    request last checked a given bucket in reverse index (i.e.: magic field)