A multi-threaded worker pool for Curb
BSD-2-CLAUSE License
A multi-threaded worker pool for Curb.
ct = Curl::ThreadPool.new
10000.times { |i| ct[i] = "http://localhost/test" }
responses = ct.perform() # => contains keyed response hash
or
ct = Curl::ThreadPool.new
reqs = []
10000.times { reqs << "http://localhost/test" }
ct.get(reqs) # => contains responses in same order as reqs
In order to re-use connections opened by the pool, hang on to it:
Thread.current[:my_pool] ||= Curl::ThreadPool.new
Thread.current[:my_pool].get("http://localhost/test")
This way, if you frequently make requests to the same host (e.g., an API service) then the connections will be kept open as long as possible and re-used, speeding up your response time. Curl will automatically reconnected when necessary.
Copyright (c) 2012 Chetan Sarva. See LICENSE for further details.