Python-Crawling-Tutorial

slideshare , , Clone or download

2017 release, link

Anaconda ()

• Python 3.6 https://www.continuum.io/downloads
• Chrome Chrome
• webdriver
  • Chrome
  • Firefox
• jupyter notebook Anaconda jupyter notebook .ipynb
• Anaconda Anaconda

$ pip install selenium tldextract Pillow

pip

pip Python pip3 Python3 pip3 Python3

# ,  pip  pip3
$ pip install requests beautifulsoup4 lxml Pillow selenium tldextract

Optional:

code ( wordcloud visual studio warining )

# Anaconda
$ pip install jieba wordcloud

# pip
$ pip3 install numpy pandas matplotlib scipy scikit-learn jieba wordcloud

robots.txt,

robots.txt wiki google

Q&A

Q: API

API API API

• Facebook Graph API
• Youtube
• Yahoo YQL
• Instagram
• KKTIX
• Google Maps API
• Taipei Open Data API
• Imgur API