Fast Elixir RSS feed parser, a NIF wrapper around the Rust RSS crate
APACHE-2.0 License
Parse RSS feeds very quickly
Speed
Currently this is already much faster than most of the pure elixir/erlang packages out there. In benchmarks there are speed improvements anywhere between 6.12x - 50.09x over the next fastest package (feeder_ex) that was tested.
Compared to the slowest elixir options tested (feed_raptor, elixir_feed_parser), FastRSS was sometimes 259.91x faster and used 5,412,308.17x less memory (0.00156 MB vs 8423.70 MB).
See full benchmarks below:
FastRSS requires a minimum combination of Elixir 1.6.0 and Erlang/OTP 20.0, and is tested with a maximum combination of Elixir 1.11.1 and Erlang/OTP 22.0.
This package is available on hex.
It can be installed by adding fast_rss
to your list of dependencies in mix.exs
:
def deps do
[
{:fast_rss, "~> 0.5.0"}
]
end
You also need the rust compiler installed: https://www.rust-lang.org/tools/install
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
There is only two functions, one for parsing rss parse_rss/1
and one for parsing atom feeds parse_atom/1
they takes a string and outputs an {:ok, map()}
with string keys.
iex(1)> {:ok, map_of_rss} = FastRSS.parse_rss("...rss_feed_string...")
iex(2)> Map.keys(map_of_rss)
["categories", "cloud", "copyright", "description", "docs", "dublin_core_ext",
"extensions", "generator", "image", "items", "itunes_ext", "language",
"last_build_date", "link", "managing_editor", "namespaces", "pub_date",
"rating", "skip_days", "skip_hours", "syndication_ext", "text_input", "title",
"ttl", "webmaster"]
The docs can be found at https://hexdocs.pm/fast_rss.
Reading from the following RSS versions is supported:
HTML: https://avencera.github.io/fast_rss/
Benchmark run from 2020-02-22 05:23:47.524699Z UTC
Benchmark suite executing on the following system:
Benchmark suite executing with the following configuration:
Input: anxiety
Run Time
Input: ben
Run Time
Input: daily
Run Time
Input: dave
Run Time
Input: sleepy
Run Time
Input: stuff
Run Time
Deploying rust NIFs can be a little bit annoying as you have to install the rust compiler. We try to alleviate this with rustler_precopmiled
, which will create precompiled assets for a number of targets (see release.yml
for the full list), but does not cover all environments. If you are having trouble deploying this package make an issue and I will try and help you out.
I will then add it to the FAQ below.
On the stages where you build all your deps, and build your release make sure to install build-base
and libgcc
:
# This step installs all the build tools we'll need
RUN apk update && \
apk upgrade --no-cache && \
apk add --no-cache \
git \
curl \
build-base \
libgcc && \
mix local.rebar --force && \
mix local.hex --force
Install the rust compiler and allow dynamic linking to the C library by setting the rust flag
# install rustup
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
ENV RUSTUP_HOME=/root/.rustup \
RUSTFLAGS="-C target-feature=-crt-static" \
CARGO_HOME=/root/.cargo \
PATH="/root/.cargo/bin:$PATH"
On the stage where you actually run your elixir release install libgcc
:
################################################################################
## STEP 4 - FINAL
FROM alpine:3.11
ENV MIX_ENV=prod
RUN apk update && \
apk add --no-cache \
bash \
libgcc \
openssl-dev
COPY --from=release-builder /opt/built /app
WORKDIR /app
CMD ["/app/my_app/bin/my_app", "start"]
FastRSS is released under the Apache License 2.0 - see the LICENSE file.