adblocker

• puppeteer: do not block main frames #401

• fix handling of websocket filters #388
• bump puppeteer to v2 #386
• bump electron to v7 + inject CSS with 'user' origin #385

• fix: fetching of resources.txt for adblocker #373
• feat: add new fromPrebuiltFull(...) helper to initialize engine #373

• make it easier to use HTML filtering outside of WebExtensionBlocker #368
  • export isHTMLFilteringSupported
  • export filterRequestHTML
  • add id attribute on Request (takes requestId value in WebExtension)
• export FilterType from main bundle.

• fix: cosmetics injection in Electron #358
• removeListener regardless of engine config #359
• feat: support subset of HTML filtering rules (^script:has-text(...)) #339
• feat: add support for 'all' option #338
• feat: add support for 'redirect-rule' option #337
• chore: update local assets + generate compression codebooks #335
• chore: clean-ups and small optimizations #334
  • rename engine to blocker in example projects (consistent naming)
  • enable on-the-fly compression in example projects
  • remove unused compact set exports (keep internal only)
  • remove explicit resourcesUrl in fromLists(...) (we always use the one served from CDN)
  • use bare for loops in compact sets and optimization framework
• simplify reverse index by removing ad-hoc tokens handling #333

• feat: allow disabling adblocking in WebExtension context #328
• feat: allow disabling adblocking in Puppeteer page #328
• feat: allow disabling adblocking in Electron session #328
• feat: support inline-script and inline-font options #327

• fix: do not block main document requests #312

• fix (electron): prevent preload script from overwriting existing ones #302

This marks the first "stable release" of @cliqz/adblocker. The project has been stable for a long while and running in production for millions of users. It was about time to graduate it to v1! This does not mark the end of the development and innovation but more of a "checkbox ticked"; people looking at the project can now safely feel like they can use it in production and @cliqz/adblocker will not let them down! In the future, we will continue supporting bleeding edge features, new filters and keep the performance great.

Changelog since v0.14.0:

• small improvements #300
  • minify script injection wrapper to save a few bytes
  • rename 'engine' into 'blocker' in examples for consistency
  • use up-to-date resources.txt from CDN
  • drop 'collapse' type (not supported upstream anymore)
  • expose some extra symbols: detectFilterType and Resources
• chore: clean-ups #294
  • Remove use of eslint completely (all source code is TypeScript so tslint is enough)
  • Remove Dockerfile, run_tests.sh
  • Move bench to TypeScript
  • Remove un-used bench/dataset/ folder
  • Make sure that all sub-packages can be installed and built independently (fix missing deps)
• enable @cliqz/metalint for repository linting #255

• update compression codebooks #289
• clean-up and update local assets + add fanboy-cookiemonster.txt #289
• only register listeners when network/cosmetics filtering is enabled #288
• Improve cosmetics selector tokenization by supporting new cases #287
  • correctly tokenize #selector:not(...) and .selector:not(...)
  • correctly tokenize .selector1.selector2

• fix certificate issue with Pete Lowe adserver

• set Request.tabId to webContentsId in Electron platform

• allow correct size allocation for data views #257

  Implement a mechanism which allows to predict the number of
  bytes needed to serialize any of the data-structures used by the
  adblocker, ahead of time (before serialization). This allows to lift
  the limitation of size completely (beforehand, we had to allocate
  a safe amount of memory to be sure there would be enough space).
  As a benefit, only the required amount of memory is used during
  initialization and updates, and there is no longer an arbitrary and
  hard-coded upper limit.

• create new @cliqz/adblocker-content package with common utils #264

  We currently rely on rollup to create a small bundle for content
  related code imported from @cliqz/adblocker. Multiple times in
  the past the bundler was not aggressive enough and code from
  background was pulled in content bundles. To make sure we do not
  have this issue again, all these content-scripts helpers are moved
  into their own package.

• provide helpers to download and build engines from lists #280

  This change allows to start blocking ads with very little logic in
  Webextension, Electron and Puppeteer platforms! To achieve this,
  blockers abstraction now provide static methods to fetch pre-built
  engines from Cliqz's CDN or build them from scratch using lists of URLs
  to subscriptions. Here is how it looks like:

  Webextension:

  import { WebExtensionBlocker } from '@cliqz/adblocker-webextension';
  
  WebExtensionBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
    blocker.enableBlockingInBrowser();
  });
  

  Electron:

  import { session } from 'electron';
  import fetch from 'cross-fetch'; // or 'node-fetch'
  
  import { ElectronBlocker } from '@cliqz/adblocker-electron';
  
  ElectronBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
    blocker.enableBlockingInSession(session.defaultSession);
  });
  

  Puppeteer:

  import puppeteer from 'puppeteer';
  import fetch from 'cross-fetch'; // or 'node-fetch'
  
  import { PuppeteerBlocker } from '@cliqz/adblocker-puppeteer';
  
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  
  PuppeteerBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
    blocker.enableBlockingInPage(page);
  });
  

• Update assets + re-generate compression codebooks #256
• implement simple event emitter for FiltersEngine and sub-classes #251
• electron: fix bundles #249
• electron: promote mutationObserver option to main config + fix constructor and parse methods #248
• fix source maps support for all packages #219

• speed-up building and unify configurations #212
  • switch typescript config into composite mode to allow faster re-builds and references between projects
  • store all build artifacts in dist instead of splitting them into dist + build
  • only emit es6 modules with TypeScript and produce everything else with rollup (cjs + minimized UMD)
  • only provide one minimized bundle: UMD
  • remove browser key from package.json (not needed as code is cross-platform)
  • add command to watch for changes
• add ElectronBlocker abstraction to perform adblocking in Electron #180
• allow the 'not' pseudoclass in cosmetic filters #184
• switch from tldts to tldts-experimental package #183
  • add bootstrap to root package.json
  • switch from tldts to tldts-experimental (faster, smaller bundles)
  • switch internal commands from npm to yarn
  • remove redundant lint during CI
• change structure of the cliqz/adblocker project into a monorepo #181
  • embrace lerna and yarn workspaces as a way to manage multiple packages
  • adopt conventional commits as a way to structure contributions
  • create few single-purpose packages:
    • @cliqz/adblocker (contains most building blocks)
    • @cliqz/adblocker-circumvention (standalone counter measures for IL)
    • @cliqz/adblocker-webextension (WebExtension wrapper)
    • @cliqz/adblocker-webextension-cosmetics (WebExtension cosmetics support)
    • @cliqz/adblocker-puppeteer (Puppeteer wrapper)
    • @cliqz/adblocker-electron (Electron wrapper)
  • create a few demonstration projects for documentation purposes:
    • @cliqz/adblocker-webextension-example
    • @cliqz/adblocker-puppeteer-example
    • @cliqz/adblocker-electron-example
  • migrate content blockers benchmark into @cliqz/adblocker-benchmarks
  • add licenser.js linter to enforce consistency in licenses and copyright notices
  • add lerna-lint.js linter to enforce consistency between all sub-packages

• In debug mode, make FiltersEngine creation and updates deterministic #176
• Fix bug in ID computation for :style(...) cosmetic filters #176
• Detect invalid cases of domain= options in NetworkFilter #176
• Make generateDiff more robust and cover corner case with ID collision #176
• Add stress-test for FiltersEngine updates. This allows us to validate all past updates of all supported lists #176
• Provide high level puppeteer blocker abstraction #177
  • [BREAKING] rename WebExtensionEngine into WebExtensionBlocker
  • [BREAKING] change format of redirect field in blocking response, it now
    exposes more information about the redirected resource: contentType,
    body and dataUrl (which was the only information originally returned
    by FiltersEngine.match(...)).
  • Rename example into examples and move test webextension into examples/webextension
• Add missing dependencies on @types/puppeteer needed by users of the library #174

• Add helpers to create and manipulate diffs #172
  • getLinesWithFilters(...)
  • generateDiff(...)
  • mergeDiffs(...)
• Add updateFromDiff method on FiltersEngine #172
• Add getFilters method on FiltersEngine #172
• Fix update issue by performing copy in StaticDataView.getBytes #172
• Serialize config.debug as well #172
• Implement support for RegExp network filters #169
• [EXPERIMENTAL] add on-the-fly string compression using short-string
  optimized method. Off by default, it can be enabled by using the
  enableCompression flag in Config. This allows a reduction in size of
  about 20% for FiltersEngine, at the cost of slightly slower updates. #122
• Remove dependency on tslib #167
• Add built-in error detection code in serialized engine #165
  • To prevent un-noticed data corruptions of the serialized adblocker,
    FiltersEngine.serialize now automatically includes a crc32 checksum and
    FiltersEngine.deserialize will automatically check integrity of the given
    serialized engine. Any mismatch will raise an exception like when the
    version of the adblocker does not match between the serialized engine and
    the code using to load it.
• [BREAKING] getCosmeticsFilter API changed to allow finer-grain subsetting
  of cosmetic filters returned: hostname-specific, DOM-specific, generic, etc.
  This allows to inject x70 less custom styles in frames for the same
  blocking, which results in a massive memory decrease as well as less time
  spent in repaint. #163
• [BREAKING] cosmetic unhide filters without hostname constraints are allowed. #163
• [BREAKING] NetworkFilter.isCptAllowed now accept request type as a string. #163
• [BREAKING] drop support for legacy Firefox Bootstrap request types. #163
• Fix matching of hostnames anchors with wildcard. #163
• Add support for $frame option in network filters. #163
• Add support for $document and $doc options in network filters. #163
• Add soft dependency to tldts to simplify API #163
  • left as require/import in normal bundles
  • bundled in minified bundles
• Add tests for Request abstraction #163
• Add static method helpers to create Request instances #163
  • Request.fromRawDetails(...)
  • Request.fromWebRequestDetails(...)
  • Request.fromPuppeteerDetails(...)
  • Request.fromElectronDetails(...)
• Add tests for injection using jsdom #163
• Cosmetic filtering performance improvements #163
  • Make use of DOM information to return subset of filters: ids, classes, hrefs
  • Make use of MutationObserver from content-script to return new DOM info
• Create integration benchmark to measure full extension #163
• Add Request parsing micro-benchmark #163
• Update bench/comparison to use adblock-rs instead of ad-block #163

• Optimize getCosmeticsFilters #158
  • [BREAKING] CosmeticFilterBucket.getCosmeticsFilters(...) now
    returns { injections: CosmeticFilter[]; stylesheet: string }
  • Internally CosmeticFilterBucket will do much less work to get
    cosmetic filters. In particular, we pre-compute a base stylesheet with
    all generic hide filters not having any corresponding unhide (#@#)
    rule in the bucket (that's most of the filters). This means we only need
    to apply exceptions and compute dynamic stylesheets for a minority of
    filters (~4%).
  • Add benchmark for getCosmeticsFilters.
• Drop RegExp in network filters #156

• prevent potential out-of-bound access #150
• bench: update comparison #148
  • Rename Ghostery to Cliqz
  • Update uBlock Origin to 1f8f616fafc0a3267cfe0796f0bbe29410fd6a71
  • Update adblockpluscore to 69118b828db0f6a53bc2306deacffc5361aeef0c
  • Update Brave to 4.1.7
• Punycode clean-up + tests #145
  • add extensive tests for data-view pushUTF8 and punycode toASCII/toUnicode
  • punycode: remove support for email addresses
  • punycode: remove mapDomain
  • punycode: replace for..of by bare for loops
  • punycode: inline helpers
• Add support for $elemhide + fix $generichide handling #143
• Speed-up parseFilters #142
  • clean-up filters parsing benchmark
  • speed-up parseFilters and detectFilterType (10-15% gain)
  • speed-up CosmeticFilter.parse (15% gain)
  • speed-up NetworkFilter.parse (10% gain)
• Introduce TokensBuffer to speed-up tokenization of filters and requests #141
  • results in a 25% speed-ups on NetworkFilters.getTokens which is one of the bottle-necks of ReverseIndex.update
• Implement variable-length encoding in StaticDataView to save space #138
  • saving of ~4% of the total size
  • fix issue with alignment of Uint32Array when serializing/deserializing (alignment should be done after persisting the size of the array, not before)
  • clean-up Node.js versions tested in Travis.
• Add AdblockFast (originally from Bluhell Firewall) to bench/comparison #133
• [BREAKING] Remove support for $bug option #131
• Add support for $badfilter option #127
  • add FiltersContainer class to store a list of filters
  • improve allocations by estimating required size of buffer from filters ahead of time
  • optimize hasUnicode using RegExp
  • include punycode package in source-tree to allow transpilation by tsc + minification + type checking
  • filters will now keep track of isUnicode which indicates if filter contains any unicode character. This allows to optimize storage + remove the need from checking later on.
  • StaticDataView's pushUTF8 now assumes the input is unicode and will not check
• Better formatting for injected stylesheets #126
• Fix python dependencies security alert #124
• Update locale assets #121
• [BREAKING] Remove support for script:contains(...) cosmetic filters #120
• Add support for $3p, $1p and $css options #119
• Fix example extension and README.md #118
• Update dev dependencies #117

• Support for cyrillic characters #115
• Implement generichide option + fix generic cosmetic matching #114
  • [BREAKING] Change arguments of FiltersEngine.getCosmeticsFilters
    getCosmeticsFilters({ url, hostname, domain }) is now expected
    instead of getCosmeticsFilters(hostname, domain). This allows to
    apply $generichide options which can match on arbitrary parts of the
    main_frame URL (not only hostname).
  • Add support for $generichide option in network filters
  • Fix matching of generic cosmetics when only negation was specified