Port of ruby-readability's port of arc90's readability project to Crystal
MIT License
Crystal port of @cantino's port of arc90's readability project
Still a WIP!
document#images
is not implemented. Specs are not passing and some parts of them are probably incorrect...
There's also a monkey patch for LibXML
while waiting for the changes in https://github.com/crystal-lang/crystal/pull/6910) to be released.
Add this to your application's shard.yml
:
dependencies:
readability:
github: joenas/readability.cr
version: 0.2.0
require "readability"
require "http/client"
response = HTTP::Client.get "http://www.example.com"
document = Readability::Document.new(response.body)
puts document.content
puts document.meta_image
# With Options
options = Readability::Options.new(
tags: %w[article p span div document b strong em h1 h2 h3 h4],
remove_empty_nodes: true,
attributes: %w[],
blacklist: %w[figcaption figure]
)
document = Readability::Document.new(response.body, options)
You may provide options to Readability::Options.new
, including:
:retry_length
: how many times to retry getting the best content
:min_text_length
: the least number of characters in a paragraph for it to be scored
:tags
: the base whitelist of tags to sanitize, defaults to %w[div p]
;
:attributes
: whitelist of allowed attributes for tags
;
:blacklist
and :whitelist
allow you to explicitly scope to, or remove, CSS selectors.
:remove_unlikely_candidates
: whether to remove unlikely candidates or not
:weight_classes
: whether to use weight_classes
or not
:clean_conditionally
: whether to use clean conditionally or not
:return_nil_content
: if no decent match, return nil
for #content
:remove_empty_nodes
: remove <p>
tags that have no text content; also
removes <p>
tags that contain only images;
:debug
: provide debugging output, defaults false;
:ignore_image_format
: for use with .images. For example:
:ignore_image_format => ["gif", "png"]
;
:min_image_height
: set a minimum image height for #images
;
:min_image_width
: set a minimum image width for #images
.
git checkout -b my-new-feature
)git commit -am 'Add some feature'
)git push origin my-new-feature
)