🕷🕸 Spiders to crawl ads of houses and apartments. 🏠 🏢
AGPL-3.0 License
.. image:: https://img.shields.io/travis/andreoliwa/scrapy-tegenaria.svg :target: https://travis-ci.org/andreoliwa/scrapy-tegenaria
Scrapy spiders to collect and browse different items (so far, apartment ads).
TODO
.. image:: https://badge.waffle.io/andreoliwa/scrapy-tegenaria.svg?label=ready&title=Ready :alt: Stories in Ready :target: http://waffle.io/andreoliwa/scrapy-tegenaria
.. image:: https://graphs.waffle.io/andreoliwa/scrapy-tegenaria/throughput.svg :alt: Throughput Graph :target: https://waffle.io/andreoliwa/scrapy-tegenaria/metrics
First, set your app's secret key as an environment variable. For example, example add the following to .bashrc
or .bash_profile
.
.. code-block:: bash
export TEGENARIA_SECRET='something-really-secret'
Then run the following commands to bootstrap your environment.
::
git clone https://github.com/andreoliwa/scrapy-tegenaria
cd tegenaria
pip install -r requirements/dev.txt
flask run
You will see a pretty welcome screen.
Once you have installed your DBMS, run the following to create your app's database tables and perform the initial migration:
::
flask db init
flask db migrate
flask db upgrade
flask run
In your production environment, make sure the TEGENARIA_ENV
environment variable is set to "prod"
.
To open the interactive shell, run ::
flask shell
By default, you will have access to app
, db
, and the User
model.
To run all tests, run ::
flask test
Whenever a database migration needs to be made. Run the following commands: ::
flask db migrate
This will generate a new migration script. Then run: ::
flask db upgrade
To apply the migration.
For a full migration command reference, run flask db --help
.