facil.io

Your high performance web application C framework

MIT License

Stars
2.1K

Bot releases are hidden (Show)

facil.io - Patch Release (small bug fixes) Latest Release

Published by boazsegev over 2 years ago

Fixed a redirection bug on FIOBJ_T_NUMBER type testing which could have caused an 8 byte read from an unknown address (possibly a segfault) when testing usingFIOBJ_TYPE_IS(obj, FIOBJ_T_NUMBER).

This version also addresses some 32 bit compatibility concerns.

Please submit any issues or pull requests at https://github.com/boazsegev/facil.io.

facil.io - Security and fixes

Published by boazsegev over 4 years ago

v. 0.7.5 (2020-05-18)

Security: backport the 0.8.x HTTP/1.1 parser and it's security updates to the 0.7.x version branch. This fixes a request smuggling attack vector and Transfer Encoding attack vector that were exposed by Sam Sanoop from the Snyk Security team (snyk.io). The parser was updated to deal with these potential issues.

Fix: (http) fixes an issue with date calculation by backporting code from the 0.8.x branch.

Fix: (fio) call less signal handlers during shutdown.

from v. 0.7.4

Fix: (http) fixes an issue and improves support for chunked encoded payloads. Credit to Ian Ker-Seymer ( @ianks ) for exposing this, writing tests (for the Ruby wrapper) and opening both the issue boazsegev/iodine#87 and the PR boazsegev/iodine#88.

Fix: (http) requests will fail when the path contains a dangling ? (empty query). Credit to @adam12 for exposing this and opening issue boazsegev/iodine#86.

facil.io - Security, compatibility and minor fixes

Published by boazsegev about 5 years ago

Fix: (http) fixes a security issue in the static file name resolution logic, where a maliciously encoded request could invoke an arbitrary response.

Fix: (fio, fiobj) improved C++ compatibility. Credit to Joey (@joeyhoek) for PR #76.

Fix: (fio) fixes an issue where timer cleanup wasn't performed after fio_stop (or SIGINT/SIGTERM). No a "clean slate" will be provided if fio_start is called more then once. Note: this may break previous behavior, which should be considered undocumented and unexpected behavior. (this fax may be deferred to version 0.8.x, still undecided). Credit to @fbrausse for opening issue #72.

Fix: (fio) fixes an issue where timer cleanup would be performed after the AT_EXIT state callbacks. Now the timer cleanup callbacks will be performed before the AT_EXIT callback (as they should). (See issue #72).

Fix: (fio) fixes signal handler (re)establishment test to prevent recursive signal calling.

facil.io - Patch Release (fixes)

Published by boazsegev about 5 years ago

Fix: (fio_tls) fixes a memory leak in the trusted certificate chain. Credit to @fbrausse for opening PR #71.

Fix: (fio_tls) fixes compilation / linking flags (including a bug caused by the gcc optimizer -fipa-icf) and improves support for OpenSSL using pkg-config. Credit to @fbrausse for PR #71.

Fix: (http1) fixes a race-condition between the on_ready and on_data events, that could result in the on_data event being called twice instead of once (only possible with some clients). On multi-threaded workers, this could result in the CPU spinning while the task lock remains busy. Credit to Néstor Coppi (@Shelvak) for exposing the issue and providing an example application with detailed reports. Issue #75.

facil.io - Security Update

Published by boazsegev over 5 years ago

v. 0.7.1

Security: a heap-overflow vulnerability was fixed in the WebSocket parser. This attack could have been triggered remotely by a maliciously crafted message-header. Credit to Dane (4cad@silvertoque) for exposing this issue and providing a Python script demonstrating the attack.

facil.io - Stable API release

Published by boazsegev over 5 years ago

Stable API release. Future API updates will be wait for the 0.8.x release.

Fix: (fio, fiobj) fixed some gcc and clang compatibility issues and warnings.

Fix: (http) fixed HTTP date format to force the day of the month to use two digits. Credit to @ianks (Ian Ker-Seymer) for exposing this issue (iodine#64).

Compatibility: (http) updated time-zone compile-time tests with a safer fall-back.

facil.io - An ocean of details

Published by boazsegev over 5 years ago

We've been hard at work authoring this (hopefully last) beta release to the 0.7.x facil.io.

There's just one or two planned API updates for the HTTP module (which is why we're still in beta phase). These changes shouldn't effect any code, but they will effect ABI compatibility, so if you need ABI compatibility, you'll need to wait a bit longer.

Here's what this update includes:

Security: (fio) Slowloris mitigation is now part of the core library, where FIO_SLOWLORIS_LIMIT pending calls to write (currently 1,024 backlogged calls) will flag the connection as an attacker and either close the connection or ignore it. This protocol independent approach improves security.

Security: (http) HTTP/1.1 client throttling - new requests will not be consumed until pending responses were sent. Since HTTP/1.1 is a response-request protocol, this protocol specific approach should protect the HTTP application against slow clients.

Fix: (fio) fixed fallback implementation for fio_atomic_xchange when missing atomic primitives in compiler (older compilers). Credit to @Low-power for identifying and fixing the issue (PR #55).

Fix: (fio) fixed a possible unreleased lock when a memory allocation failed (no memory in the system). Credit to @Low-power for identifying and fixing the issue (PR #54).

Fix: (fio) fixed the fio_sock_sendfile_from_fd fall-back for a missing sendfile. Credit to @Low-power for identifying and fixing the typo (PR #49).

Fix: (fio) fixed fio_pending not decrementing packet count before reaching zero.

Fix: (fio) fixed logging message for overflowing log messages. Credit to @weskerfoot (Wesley Kerfoot) and @adam12 (Adam Daniels) for exposing the issue (issue iodine/#56).

Fix: (fio, fio_risky_hash) Florian Weber (@Florianjw) exposed a byte ordering error (last 7 byte reading order) and took time challenge the algorithm. The exposed errors were fixed and the exposed a possible attack on RiskyHash using a variation on a Meet-In-The-Middle attack, written by Hening Makholm (@hmakholm). This prompted an update and fixes to the function.

Fix: (fio) fixed fio_str_resize where data might be lost if data was written beyond the current size and the requested size is larger then the String's capacity (i.e., when fio_str_resize is (mis)used as an alternative to fio_str_capa_assert).

Fix: (json / redis) fixed JSON formatting error caused by buffer reallocation when multiple (more then 48) escape sequences were detected. This issue also effected the Redis command callback handler (which was using JSON for IPC).

Fix: (redis) fixed a potential double free call.

Fix: (redis) fixed a recursive endless loop when converting nested Hash Tables to Redis objects (which normally wouldn't happen anyway, since they would be processed as JSON).

Fix: (redis) fixed Redis reconnection. Address and port data was mistakingly written at the wrong address, causing it to be overwritten by incoming (non-pub/sub) data.

Fix: (redis) fixed a race condition in the Redis reconnection logic which might have caused more then a single pub/sub connection to be established and the first pending command to be sent again.

Fix: (fio) fix capacity maximization log to accommodate issues where getrlimit would return a rlim_max that's too high for rlim_cur (macOS).

Fix: (fio) fix uninitialized kqueue message in fio_poll_remove_fd.

Fix: (http) possible fix for http_connect, where host header length might have been left uninitialized, resulting in possible errors.

Fix: (fio) fixed logging error message for long error messages.

Update: (fio / makefile) improved detection for polling system call, sendfile, etc'.

Update: (fio) improved signal handling. Signal handling now propagates to pre-existing signal handlers. In addition, the fio_signal_handler_reset function was made public, allowing facil.io signal handlers to be removed immediately following startup (using fio_state_callback_add with FIO_CALL_PRE_START to call fio_signal_handler_reset).

Update: (fio) improved pub/sub memory usage to minimize message copying in cluster mode (same memory is used for IPC and local-process message publishing).

Update: (fio) updated the non-cryptographic PRG algorithm for performance and speed. Now the fio_rand functions are modeled after the xoroshiro128+ algorithm, with an automated re-seeding counter based on RiskyHash. This should improve performance for non cryptographic random requirements.

Compatibility: (fio) mitigate undefined MAP_ANONYMOUS on MacOS <= 10.10. Credit to @xicreative (Evan Pavlica) for iodine/PR#61.

Compatibility: (fio) various Solaris OS compatibility patches, courtesy of @Low-power (PR #52, #53).

facil.io - API and TLS updates, Hash Map security and more...

Published by boazsegev over 5 years ago

I spent a bunch of time learning about Hash Map and Hash function security concerns, realizing Hash Map security should be prioritized over a goof Hash function. I also Implemented a TLS client in iodine (the Ruby wrapper for facil.io) and learned a bunch of stuff that required some API changes.

On the positive side, it appears that the 0.7.x API is fairly finalized and facil.io could probably move to a 0.7.0 release soon.

The main changes in the this beta release are:

BREAK: (fio_tls) breaking API changes to the SSL/TLS API... I know, I'm sorry, especially since there's a small and misleading change in argument ordering for fio_tls_cert_add and fio_tls_new... but if we don't fix the API now, before the 0.7.0 release, bad design might ruin our Wednesday meditation for all eternity.

BREAK: (http) breaking API changes to http_connect were required in order to support Unix Socket connections in client mode.

Deprecation: (http) deprecating the http_url_parse in favor of fio_url_parse (moved the function to the core library and rewrote it in part).

Security: facil.io hash maps now limit the number of full-collisions allowed in a hash map. This mitigates the effects of hash flooding attacks. As a side effect, hash maps that are under attack might return false results for collision objects.

Fix: (websocket) fixed an issue with the WebSocket parser where network byte order for long message lengths wasn't always respected and integer bit size was wrong for larger payloads. Credit to Marouane Elmidaoui (@moxgeek) for exposing the issue.

Fix: (http) fixed udata in WebSocket client callback for failed WebSocket client connections.

Fix: (fio) logging message when listening to a Unix Socket.

Fix: (fio) numerous minor design fixes, such as Big-Endian string memory access, allowing fio.h to be used as a header only library (requires FIO_FORCE_MALLOC) and other adjustments.

Fix: (fio) fixed unaligned memory access in SipHash implementation and added secret randomization for each application restart.

Fix: (redis) fixed an issue where destroying the Redis engine and exiting pre-maturely, before running facio.io (fio_start), will cause a segmentation fault during cleanup.

Update: (fio) added Risky Hash, for fast hashing of safe data. This is a fast hashing function (about twice as fast as the default SipHash1-3 secure function) that wasn't tested for security. For this reason it should be limited to internal / safe data, such as CLI argument names.

facil.io - Security and TLS

Published by boazsegev almost 6 years ago

This is a security release with partial support for TLS, using OpenSSL.

TLS support is still under development and the API related to TLS should be considered fragile (I will release 0.7.0 once the API is solid enough and things work).

BREAK: (fio_tls) breaking API changes to the SSL/TLS API, adding support for password protected private key files. Note: The TLS API is still fragile and should only be considered stable once version 0.7.0 is released with SSL/TLS support.

Security / Fix: (http) fixed an issue with the HTTP/1.1 parser, where maliciously crafted white-space data could cause a segmentation fault, resulting a potential DoS.

Fix: (fio) fixed an issue exposed by implementing the TLS layer, where the highet fd for a connection that wasn't assigned a protocol_s object immediately after the connection was opened, might avoid timeout review or avoid cleanup during shutdown (which will be marked as a memory leak).

Update: (fio_tls) added experimental support for OpenSSL. This was only partially tested and should be considered experimental.

Update: (fio) added, the fio_rw_hook_replace_unsafe to allow r/w hook switching from within a r/w hook callback.

Update: (fio_cli) a common user-error is a a missing fio_cli_end, resulting in a memory leak notification. Now facil.io protects against this common error by automatically calling fio_cli_end during the exit stage, if fio_cli_start was called.

facil.io - Quick fix

Published by boazsegev almost 6 years ago

This is a quick fix release, fixing a minor issue introduced in the 0.7.0.beta4 version:

Fix: (fio_cli) fixed an issue introduced in version 0.7.0.beta4, where fio_cli_get_i would dereference NULL if the value wasn't set. Now fio_cli_get_i returns zero (0) for missing values, as expected. Note: this related to the new hex and binary base support in command line numerals.

facil.io - minor breaking changes and fixes + better Mustache support

Published by boazsegev almost 6 years ago

BREAK: (fio_cli) breaking API changes make this extension easier than ever to use... I do apologize for this, but part of the reason 0.7.0 is still in beta is to test the API itself for ease of use and readability.

Fix: (fio) fixed a minor memory leak in cluster mode, caused by the root process not freeing the hash map used for child process subscription monitoring.

Fix: (fio) fixed superfluous and potentially erroneous pub/sub engine callback calls to unsubscribe, caused by (mistakingly) reporting filter channel closure.

Fix: (mustache, FIOBJ) added support for dot notation in mustache templates.

Fix: (http/1.1) avoid processing further requests if the connection was closed.

Fix: (fio_test) fixed some memory leaks in the testing functions.

Update: (fio_cli) stylize and beautify FIO_CLI_PRINT_HEADER lines.

Update: (fio) updated the automatic concurrency calculations to leave resources for the system when a negative value is provided (was only available for worker count calculations, now available for thread count as well).

facil.io - Things we learned

Published by boazsegev almost 6 years ago

Fix: (fio) Fixed superfluous ping events that might occur after a fio_write (but before the scheduled write actually occurred).

Fix: (mustache) updated the mustache parser to fix an issue with template loading path names. The partial template path resolution logic was re-written, fixed and improved (I hope). This also adds support for text in mustache lambda, though not applicable when used with FIOBJ.

Fix: (fio) Prevent Read/Write Hooks from delaying fio_force_close when an error occures while polling a connection.

Fix: (fio) Deletes Unix sockets once done listening. Fixes an issue where the files would remain intact.

Fix: (fio) Replaced fio_malloc existing memory allocation / free-list implementation. This also fixes an issue with large memory pools being retained on multi-core systems with many reported CPU cores.

Fix: (fio) The FIO_FORCE_MALLOC flag was fixed to accommodate for the fact that fio_malloc returns zeroed data (all bytes are set to zero) vs. the system's malloc which might return junk data.

Fix: (http) fixes a possible memory leak in http_mimetype_register, where clearing the registry wouldn't free the FIOBJ Strings.

Update: (cli) updated handling of empty strings in CLI values by ignoring the argument rather than printing an error or experiencing an undefined value.

Optimization: (fio) pub/sub channel names appear to be (often) objects with a long life-span. Hence, these objects now use malloc (instead of fio_malloc). Also, temporary allocations in fio_subscribe were removed.

Optimization: (fio) pub/sub meta-data information and callbacks now use an Array (instead of link lists and a hash map). This should improve cache locality when setting and retrieving pub/sub meta-data.

Optimization: (fio) added an urgent task queue for outbound IO, possibly improving protection against non-evented / blocking user code.

Optimization: (http) WebSocket broadcasting optimizations are now automated.

facil.io - A Unified API with 0.7.0.beta2

Published by boazsegev almost 6 years ago

Breaking Changes!

A lot of the code was re-written and re-organized, minimizing the name space used by the core library and consolidating the core library into a two file library (fio.h and fio.c).

This translated directly to breaking the API and ABI and bumping the version number.

This should make the library easier to copy and use as well as minimize possible name collisions (at the price of maintaining a couple of monolithic files as the core library).

These are the main changes:

  • Extracted the FIOBJ library from the core library, making it an add-on that could used by the extensions (such as the HTTP extension) rather than a core requirement.

  • Condensed the core library and it's namespace to two files (fio.h and fio.c) - replaced all facil_ function names with fio_ to accommodate the new namespace.

    ...why?

    It's a choice to sacrifice ease of maintainability in favor of ease of use.

    Although two files are more difficult to maintain than 10 files (assuming it's the same amount of code)... it seems that two files are easier for other developers to copy and paste into their projects.

  • Added poll support to allow for performance testing and CYGWIN compatibility. The system epoll/kqueue calls should perform better for higher loads, but now you can see for yourself.

  • Timers are now in user space, allowing for more timers and less kernel dependencies.

  • The on_idle and on_finish settings in facil_run (now fio_start) were removed, replaced by the more flexible fio_state_callback_add approach.

  • The fio_listen and http_listen functions now return the listening socket's uuid (much like fio_connect and http_connect did).

  • The Protocol's on_shutdown callback is now expected to return a uint8_t, hinting at a requested timeout before the socket is forcefully closed. A return value of 0 will indicate immediate socket closure with an 8 second timeout for outgoing buffer flushing.

  • The cluster messaging system and the Pub/Sub system were both refactored, the API changed and the FIOBJ dependency was removed. This change cascades to effect all the Pub/Sub system elements.

  • The Pub/Sub system's use_pattern was replaced with the optional callback argument match (a function pointer), allowing for custom pattern matching approaches (such as implementing NATs and RabbitMQ pattern matching). The previous glob matching approach (Redis compatible) is available using the provided FIO_MATCH_GLOB function pointer.

  • The WebSocket upgrade (http_upgrade2ws) now matches the SSE upgrade function (starts with the handle http_s * and named arguments come later).

  • The CLI API and implementation was completely rewritten.

    The new code is slightly more "monolithic" (one long function does most of the work), but should waste less memory with a simpler API (also got rid of some persistent data).

  • The Read/Write socket hooks were redesigned.

  • An SSL/TLS API stub was designed for SSL/TLS library abstraction (not implemented yet).

    This API is experimental and might change as I author the first SSL/TLS library wrappers (roadmap includes OpenSSL and BearSSL).

Update: (fio_mem => fio.h) updated the allocator defaults to lower the price of a longer life allocation. Reminder: the allocator was designed for short/medium allocation life-spans or large allocations (as they directly map to mmap). Now 16Kb will be considered a larger allocation and the price of holding on to memory is lower (less fragmentation).

Fix: (fio) fixed a typo in the shutdown output. Credit to @bjeanes (Bo Jeanes) for the Iodine#39 PR.

Feature: (FIOBJ) added mustache template support.

Logo: Logo created by @area55git (Area55)

Documentation: A new website!

facil.io - Some birds eat bugs

Published by boazsegev over 6 years ago

This is a bug squashing release.

Fix: (sock) fixed an issue where calls to sock_write could potentially add data to the outgoing queue even after sock_close in cases where the outgoing queue isn't empty.

Fix: (facil.io) fixed a race condition between pre-scheduled tasks (defer calls) and the worker process initialization. The race condition might have resulted in some pre-scheduled tasks not running on all the workers.

Fix: (http) fixed an issue with the HTTP request logging, where the peer address wasn't shown.

Fix: (websocket) made sure that on_ready is never called before on_open returns.

Fix: (fio_mem, facil, http) fixed compatibility issues with Alpine Linux distro and older OS X versions (< 10.12).

Fix: (http) fixed the http_date2rfc2109 method where a space wasn't written to the buffer after the month name (resulting in a junk byte).

Fix: (pubsub) made sure that newly registered engines get the full list of existing subscriptions (no need to call pubsub_engine_resubscribe).

Fix: (facil) possible fix for protocol attachment with NULL protocol.

facil.io - EventSource (SSE) support + more bug fixes

Published by boazsegev over 6 years ago

Welcome EventSource (SSE) support into the mix - using the .on_upgrade callback with "sse" as the requested protocol identifier, it's now easy to support SSE on facil.io applications.

The details:

Fix: (fio_hashmap / fiobj_hash) fixed a possible issue that could occur when compacting a severely fragmented Hash (where the length of the new storage requirements is shorter than the fragmented ordered array of data).

Fix: (http / websocket) fixed an issue where the WebSocket's on_close callback wouldn't be called if certain errors prevented the upgrade. Now the on_close callback is always called.

Fix: (http) fixed an issue where the Content-Type header might be missing when sending unrecognized files. Now an additional best attempt to detect the content type (this time using the URL instead of the file name) will be performed. If no content type is detected, the default RFC content type will be attached (application/octet-stream).

Fix: (http1_parser) fixed a possible unaligned memory access concern.

Fix: (FIOBJ) fixed compiler compatibility concerns with the fiobj_num_new logic, removing some possibly undefined behavior.

Fix: (facil) a missing on_data protocol callback (missing during facil_attach) will now call facil_quite, preventing the event from firing endlessly.

Update: (http) the on_upgrade callback now supports SSE connections with sse protocol identifier and the http_upgrade2sse function.

Update: (sock) initial support for TCP Fast Open (TFO) when listening for connections.

facil.io - Major Fixes, Patch Release

Published by boazsegev over 6 years ago

This version fixes a number of issues, including a serious issue that prevented sockets from fully flushing their buffer.

This version also improved the shutdown and hot restart logic and fixes numerous issues with cluster mode an pub/sub services.

Note that the pubsub_engine_s struct was updated, which might break ABI for those that write custom extensions to the library (reminder: this is a source code library, I try to keep the ABI stable, but I take more liberty while still in development versioning).

It's recommended that all 0.6.0.beta, 0.6.0 and 0.6.1 upgrade to this version.

Security: (http1) added a hard-coded limit on the number of headers allowed per request (regardless of size). HTTP_MAX_HEADER_COUNT defaults to 128, which should be enough by all accounts.

Fix: (pubsub, facil, redis-engine) fixed numerous cluster and Pub/Sub issues, including support for new on_startup callback for pubsub_engine_s objects (to make handling forks that much easier. This fixes a memory leak, a reference counting error that freed memory prematurely, message parsing errors on fragmented messages, an obsolete ping formatting error, and more.

Fix: (sock, facil) fixed an issue where socket buffers wouldn't be completely cleared (the on_ready event wouldn't be properly re-armed). This was discovered as a serious issue and upgrading to 0.6.2 is recommended.

Fix: (http) fixed an HTTP status string output error, where status codes above 410 would degrade to status 500 (internal error) instead of printing the correct status string to the response.

Fix: (websockets) fixed a missing "close" packet signal that should have been sent immediately after the user's on_shutdown callback.

Fix: (FIOBJ) fixed the fiobj_str_tmp function to add thread safety (temp string should be stored in the thread's local storage, not globally accessible).

Fix: (redis) fixed a race condition in the Redis engine that could prevent publishing connections from being established in worker processes.

Fix: (facil) fixed an issue where facil_attach wouldn't call the on_close callback if the failure was due to the file descriptor being equal to -1.

Fix: (facil) fixed a signaling issue where a SIGUSR1 sent to a worker process might inadvertently shutdown the server instead or wind down the specific worker and re-spawn a new one.

Fix: (facil) fixed a signal handling logic to make it async-safe, protecting it against possible deadlocks or cluster stream corruption.

Update/Fix: (facil) the on_data event will no longer be fired for sockets that were flagged to be closed using sock_close.

Update: (FIOBJ) updated the fiobj_str_readfile to allow for a negative stat_at position (calculated from the end of file of the file backwards).

Update: (facil) strengthened the on_shutdown callback lock to prevent the on_shutdown callback from being called while the on_data callback (or other connection tasks) is running.

Update: (facil) shutdown logic provides more time for socket buffers to flush (only when required).

facil.io - Patches and cleanup

Published by boazsegev over 6 years ago

Fixed a few issues and cleaned up some code.

Fix: (pubsub) fixed a possible issue where a channel name might be freed before it's callback is handled. This was overlooked during the Hash caching logic update that prevented key hashing when the last item of the ordered Hash is removed.

Fix: (pubsub) pubsub will now compact the memory used for client and channel data if the storage becomes excessively fragmented.

Fix: (hashmap) fixed a possible memory reading issue that could occur when a Hash contains only one object and that object is removed (causing a memory read into the location just before the Hash map's address).

Fix: (defer) defer now prefers the statically allocated buffer over the dynamically allocated buffer when all tasks have completed, preventing the last allocated buffer from leaking during the shutdown stage.

Fix: (websocket) subscriptions created during the on_close callback (besides indicating that the API was being abused) are now properly removed.

facil.io - Version 0.6.0 is officially released

Published by boazsegev over 6 years ago

Version 0.6.0 is a major release, changing much of the extension API (HTTP, pub/sub, CLI) and some of the core API (i.e., moving the evio polling from level-triggered to one-shot polling, a rewrite to the facil.io dynamic object types FIOBJ, and more).

The following updates are included in this release (in addition to the beta updates):

Fix: (pubsub) Fixed an issue where deferred pub/sub messages would have udata2 set to udata1 instead of the actual correct value.

Fix: (facil) Fixed the facil_is_running() function, which could crash if facil.io wasn't initialized before the function was called.

Fix: (facil) Fix CPU limit detection. Negative values are now meaningful (fraction of CPU cores, so -2 == cores/2). Zero values are replaced by facil.io.

Update: (facil) Hot restart is now available for cluster mode. By sending the SIGUSR1 signal to the program, facil.io will shutdown any worker processes and re-spawn new workers, allowing for a hot restart feature. Disable using FACIL_DISABLE_HOT_RESTART

Update: (facil) Dedicated system mode can be toggled by setting the FIO_DEDICATED_SYSTEM macro during compile time. When FIO_DEDICATED_SYSTEM is set, facil.io will assume all the CPU cores are available and it will activate threads sooner. When FIO_DEDICATED_SYSTEM is defined as 0, facil.io will limit thread to protect against slow user code (rather than attempt concurrent IO).

Update: (fio_mem) replaced the double linked list logic with a single linked list to make the library more independent as well as reduce some operations.

As well as some refactoring, minor adjustments and the beta updates.

facil.io - Backport fix 0.5.10

Published by boazsegev over 6 years ago

This is a back ported fix to the facil.io cluster messaging system in the 0.5.x release branch.

Fix: (facil / pubsub) fixed an issue where cluster messages would be corrupted when passed in high succession. Credit to Dmitry Davydov (@haukot) for exposing this issue through the Iodine server implementation (the Ruby port).

It should be noted that this doesn't fix the core weakness related to large cluster or pub/sub messages, which is caused by a design weakness in the pipe implementation (in some kernels).

The only solution for large message corruption is to use the new pub/sub engine introduced in the facil.io 0.6.x release branch, which utilizes Unix Sockets instead of pipes.

facil.io - Sweetness

Published by boazsegev over 6 years ago

I still have a few features to roll out before 0.6.0 is officially released. Otherwise, this version would have been the actual release.

This version appears to be both more stable and more mature than the 0.5.9 release.

One of the best performance enhancers in this release is the custom memory allocator. It's tuned for the network style use and probably shouldn't be used for persistent data... it's also thread-friendly and fast.

The cookie implementation is also sweeter, now with easy cookie parsing (both for client and server modes) as well as cookie authoring.

Fix: (defer) the defer_free_thread symbol is now correctly marked as weak, allowing the function to be overridden.

Fix: (http) fixes an issue where cookies without an explicit age would be marked for immediate deletion (instead of the expected "session" lifetime).

Fix: (http) fixes a potential issue where a missing or corrupt accept-encoding header could cause a segmentation fault.

Fix: (http) fixes an issue where a cookie encoding errors would reported repeatedly.

Fix: (fio_hash) fixes an issue where resubmitting a removed object wouldn't increase the object count.

Fix: (fiobj) fixes an issue where testing the type of specific FIOBJ_T_NUMBER objects using FIOBJ_TYPE_IS would return a false positive for the types FIOBJ_T_HASH or FIOBJ_T_STRING.

Update: Added an experimental custom memory allocator (fio_mem.h) optimized for small concurrent short-lived allocations (anything over 16Kb and reallocations start to take a toll). It can replace the system's malloc function family when FIO_OVERRIDE_MALLOC is defined. To use tcmalloc or jemalloc, define FIO_FORCE_MALLOC to prevent fio_mem from compiling.

Update: (http) added cookie parsing.

Update: minor optimizations, fio_malloc incorporation and documentation updates.