Ruby: Web Scraper

An application designed to scrap the web and retrieve information from movie websites, such as the movie title, it's quality, year of release, download link etc, it retrieves data from movie websites like Gostream.

Web scraping is the process of using bots to extract content and data from a website. Unlike screen scraping, which only copies pixels displayed onscreen, web scraping extracts underlying HTML code and, with it, data stored in a database. The scraper can then replicate entire website content elsewhere.

Documentation

I don't know about you, but I have always wanted to surf the web and download movies or other public data freely without being annoyed by those pop-ups and ads from the internet. I'm glad I can do that now with this Web scraper that I've created using Ruby programming language and few Gems. This application was designed to simplify the process of retrieving movie information from a website and then display it in your terminal or command prompt. And then you decide if you want to download it or stream it, giving you the direct link to accomplish the task. This tool is very easy to use, please follow the instructions bellow to install it and enjoy scraping the web :D.

If you are on repl.it, please click here.

Built With

Ruby
Gems
- Nokogiri
- HTTParty
- Rspec
- colorize
Live Demo

click here**.

Getting Started

You have two options for using this program: running it locally or running it from the live demo.

Preparing the local environment

For running this program locally, you need to:

Note: as discussed below, downloading the source code without git is possible.

Once Ruby is installed, get a copy of this project by:

Cloning the repository using git
- git clone https://github.com/ixboy/web_scraper.git
Downloading a zip with the source code
- Click here
- Once downloaded, extract it in a place with easy access to a terminal

Once Ruby is installed, and you have a copy of the source code, open the terminal and navigate to the folder where the code is:

For Windows users, check this article.
For UNIX/UNIX-like users such as Linux and macOS users, check this article.

Install the Gems by running bundle install. You may need to provide your user password. For more information about Ruby's bundle, check here.

From the root directory(Web_Scraper), change directory into the bin directory by running the following command and start using the tool...

ruby bin/scraper.rb

To run the Rspec test suites, from the root directory(Web_Scraper) after installing ruby Gems and installing rspec, just run the following command.

rspec

Preparing repl.it

If you are running this program inside the live demo:

Click the green Run button centered at the top
Verify that the output shows no errors
Click the Shell tab in the right panel

The Run button will install any missing Gem and run a default demo command. For the interactive view use the Shell tab, because the Console view does not display it correctly.