spiderable-middleware

📦 NPM: @2.0.0
☄️ Packosphere @2.0.0

Major Changes:

• ⚠️ Removed request-libcurl dependency, replaced with https module
• ⚠️ Removed legacy url usage, replaced with new URL constructor
• ⚠️ Old requestOptions doesn't match http module options

Changes:

• ✨ new sanitizeUrls option (see docs)
• ✨ requestOptions now passed to http module .request method
• 📔 Docs update to match changes

Dependencies:

• 📦 [removed] [email protected]
• 📦 [dev] [email protected], was v4.3.6
• 📦 [dev] [email protected], was v4.18.1
• 📦 [dev] [email protected], was v10.0.0

• 🧹 Codebase cleanup
• 👷‍♂️ Avoid exception on .abort()

👷‍♂️ This is maintenance release

• Fix #13

📦 Dependencies update:

• 📦 [email protected], was v2.3.3

👷‍♂️ This is maintenance release

• 🤝 Compatibility with latest node LTS
• 🤝 Compatibility with [email protected]
• 👨‍🔬 Minor update of meteor.js tests

📦 Dependencies update:

• 📦 [email protected], was v2.3.2
• 📦 [dev] [email protected], was v4.3.4
• 📦 [dev] [email protected], was v4.17.1
• 📦 [dev] [email protected], was v9.1.1
• 📦 [dev] [email protected], was v1.13.1

• request-libcurl dependency update to v2.3.2, was v2.2.1
• Latest meteor compatibility
• Minor documentation enhancements
• Minor DEBUG flag support enhancements

Added:

• ✨ Add support for debug option

Changed:

• 👨‍💻 Add nel and report-to to ignored headers
• 👨‍💻 Improve HTTP response headers RegExp. Closing #6; Thanks to @arabidopsis for suggestion 👍
• 👷 Internal codebase micro-refactoring
• 📋 Minor docs update

This is maintenance release.

Fixes:

• 🐞 Fix edge case when query sin't and Object type;

Dependencies:

• 📦 request-libcurl updated to v2.2.1, was v2.2.0;
• 📦 dev-dependency mocha updated to v8.1.1, was v7.1.2.

New:

• ✨ Configuration option: opts.timeout {Number} - [Optional] Number, proxy-request timeout to rendering endpoint in milliseconds. Default: 180000

Changed:

• 👨‍💻 Codebase refactoring;
• 👷 Support for HEAD requests;
• 🔥 Fix for custom status codes on edge cases;
• 👷 Better client abort handling;
• 📋 Recommended nginx config;

Updated:

• 📦 [dev-dependency] [email protected], was 7.1.2.

• 👷 This is maintenance release;
• 🤝 Compatibility with [email protected];
• 📦 Update request-libcurl NPM dependency to v2.2.0, was v2.0.6;
• 📦 Update dev-dependencies.

New:

• 📱 Support for mobile crawlers and bots;
• 🐞 Fix regression introduced in latest release affecting some of the node.js setups;

Changed:

• 📋 Update documentation and examples;
• 📦 Updated NPM dependency request-libcurl to v2.0.6, was 2.0.3;
• 📦 Updated NPM [dev-dependency] mocha to v7.1.0, was 6.2.2;
• 📦 Updated NPM [dev-dependency] request to v2.88.2, was 2.88.0;
• 📦 Updated NPM [dev-dependency] underscore to v1.9.2, was 1.9.0.

New:

• 👨‍💻 requestOptions option, which is passed to request-libcurl package. It can be used to enable debug or adjust default timeout, read request-libcurl API docs for more details;
• 📋 Debugging documentation section

Changed:

• 👷‍♂️ Increase default timeout to 102400 ms; was 51200 ms; We know this number is kind of high, but its only purpose to serve edge cases like extra-large, extra-heavy, and extra-complex pages, as well as network issues
• 👷‍♂️ Increase retryDelay to 128 ms; was 64 ms; This is delay between failed requests to pre-rendering endpoint

Other Changes:

• 👨‍🔬 Minor update to test-suite for meteor
• 📦 Reduce Meteor.js (Atmosphere) bundle size with onTest (e.g. [dev]) dependencies
• 📋 Minor update/fixes to documentation

Changed:

• 👷‍♂️ List of "bad http statuses" (which would cause request retry) to [502, 503, 504, 599]; was [ 300, 303, 305, 400, 407, 408, 409, 410] && >= 500
• 👨‍💻 Retry delay between retries to 64ms; was 256ms
• 👷‍♂️ Added Accept: */* request header

Other changes:

• 👷‍♂️ Overall minor codebase enhancements and performance improvements
• 📦 Optimized package bundle size
• 🤝 Compatibility with [email protected]

Updates:

• 📦 [email protected]; was v2.0.0
• 📦 [dev-dependency] [email protected]; was v6.2.1
• 📦 Internal meteor dependencies update

• 👨‍💻 Minor overall codebase enhancements in stability and durability
• 👨‍💻 Better debugging logs and hints
• 📋 Minor documentation update
• 📦 Update request-libcurl to v2.0.0 (was 1.0.3)
• 📦 Update dev-dependency (test-suite) mocha to v6.2.1 (was v6.1.4)

• 👨‍💻 New option ignoredHeaders, see updated docs
• 👷‍♂️ Potential issue fix related to gzip and encoding-transfer
• 📦 NPM dependencies upgrade

• 📦 Dependencies update
• 👷‍♂️ This is maintenance release

• 👷‍♂️ Overall codebase enhancements
• 👷‍♂️ Switch from request to request-libcurl library
• 👨‍🔬 Test-suite dependencies update

• 🚧 Make sure library imported only on Node.js by throwing exception in Browser environment
• 👨‍💻 Minor tests update
• 📋 Add note about Node.js and Server environment to the documentation
• 📦 Internal and test (dev) dependencies update
• 🤝 Compatibility with [email protected]
• 👷‍♂️ This is regular maintenance relase

Looking for v1.3.5? We're very sorry v1.3.5 was broken due to network interruption during upload to NPM servers, to follow same release number we skipped v1.3.5 at Atmosphere and re-uploaded v1.3.6 to NPM.

• 👷‍♂️👷‍♀️ This is regular maintenance release
• 📦 Update dev dependencies
• 📦 Update Meteor internal dependencies
• 🤝 Compatibility with [email protected]
• 🤖 Run auto-tests, make sure package compatible with newest packages, node.js, and Meteor.js
• 📜 Docs update + minor fixes

• 📦 Dependencies update
• 👷‍♂️ Regular update
• 🤝 Tests and compatibility with [email protected]
• 🤝 Tests and compatibility with [email protected]

• Compatibility with [email protected]
• Dependencies update