LWP-Protocol-Net-Curl

let's make LWP great again!

Stars
14
Committers
4

=pod

=encoding UTF-8

=head1 NAME

LWP::Protocol::Net::Curl - the power of libcurl in the palm of your hands!

=head1 VERSION

version 0.026

=head1 SYNOPSIS

#!/usr/bin/env perl;
use common::sense;

use LWP::Protocol::Net::Curl;
use WWW::Mechanize;

...

=head1 DESCRIPTION

Drop-in replacement for L, LWWW::Mechanize and their derivatives to use LNet::Curl as a backend.

Advantages:

=over 4

=item *

support ftp/ftps/http/https/sftp/scp protocols out-of-box (secure layer require L<libcurl|http://curl.haxx.se/> to be compiled with TLS/SSL/libssh2 support)

=item *

support SOCKS4/5 proxy out-of-box

=item *

connection persistence and DNS cache (independent from LLWP::ConnCache)

=item *

lightning-fast L<HTTP compression|https://en.wikipedia.org/wiki/Http_compression> and redirection

=item *

lower CPU usage: this matters if you C<fork()> multiple downloader instances

=item *

asynchronous threading via LCoro::Select (see F<eg/async.pl>)

=item *

at last but not least: B<100% compatible> with both L and LWWW::Mechanize test suites!

=back

=head1 LIBCURL INTERFACE

You may query which L protocols are implemented through LNet::Curl by accessing C<@LWP::Protocol::Net::Curl::implements> or C<%LWP::Protocol::Net::Curl::implements>.

By default, B listed in that array will be implemented via LLWP::Protocol::Net::Curl. It is possible to import only specific protocols:

use LWP::Protocol::Net::Curl takeover => 0;
LWP::Protocol::implementor(https => 'LWP::Protocol::Net::Curl');

The default value of C option is I, resulting in exactly the same behavior as in:

use LWP::Protocol::Net::Curl takeover => 0;
LWP::Protocol::implementor($_ => 'LWP::Protocol::Net::Curl')
    for @LWP::Protocol::Net::Curl::implements;

Default L<curl_easy_setopt() options|http://curl.haxx.se/libcurl/c/curl_easy_setopt.html> can be set during initialization:

use LWP::Protocol::Net::Curl
    encoding    => '',  # use HTTP compression by default
    referer     => 'http://google.com/',
    verbose     => 1;   # make libcurl print lots of stuff to STDERR

Or during runtime, using special HTTP headers (prefixed by C):

use LWP::Protocol::Net::Curl;
use LWP::UserAgent;

my $ua = LWP::UserAgent->new;
my $res = $ua->get(
    'https://metacpan.org/',
    X_CurlOpt_Verbose => 1,
);

Options set this way have the lowest precedence. For instance, if LWWW::Mechanize sets the IReferer: by it's own, the value you defined above won't be used.

=head1 DEBUGGING

Quickly enable libcurl I mode via C environment variable:

PERL5OPT=-MLWP::Protocol::Net::Curl=verbose,1 perl your-script.pl

BBonus: it works even if you don't include the C line!

=for Pod::Coverage import request

=head1 TODO

=over 4

=item *

better implementation for non-HTTP protocols

=item *

more tests

=item *

expose the inner guts of libcurl while handling encoding/redirects internally

=item *

revise LNet::Curl::Multi "event loop" code

=back

=head1 BUGS

=over 4

=item *

sometimes still complains about I<Attempt to free unreferenced scalar: SV 0xdeadbeef during global destruction.>

=item *

in "async mode", each LLWP::UserAgent instance "blocks" until all requests finish

=item *

parallel requests via LCoro::Select are B; consider using L if you're into event-driven parallel user agents

=item *

LNet::Curl::Share support is disabled on threaded Perl builds

=back

=head1 SEE ALSO

=over 4

=item *

LLWP::Protocol::GHTTP - used as a reference for LLWP::Protocol implementation

=item *

LLWP::Protocol::AnyEvent::http - another LLWP::Protocol reference

=item *

L - LNet::Curl usage reference

=item *

LNet::Curl - backend for this module

=item *

LLWP::Curl - provides LLWP::UserAgent-compatible API via LWWW::Curl

=back

=head1 AUTHOR

Stanislaw Pusep [email protected]

=head1 COPYRIGHT AND LICENSE

This software is copyright (c) 2014 by Stanislaw Pusep.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

=head1 CONTRIBUTORS

=for stopwords José Joaquín Atria Nick Kostyria Peter Williams

=over 4

=item *

José Joaquín Atria [email protected]

=item *

Nick Kostyria [email protected]

=item *

Peter Williams [email protected]

=back

=cut