Popular repositories Loading

1. distribute_crawler distribute_crawler Public

   Forked from gnemoug/distribute_crawler

   使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现

   Python 1

2. scrapy scrapy Public

   Forked from scrapy/scrapy

   Scrapy, a fast high-level web crawling & scraping framework for Python.

   Python 1

3. zhihu-scrapy zhihu-scrapy Public

   Forked from immzz/zhihu-scrapy

   A scrapy zhihu crawler

   Python

4. movie-scrapper movie-scrapper Public

   Forked from codeadict/movie-scrapper

   Scrapy project to fetch movies data from IMDB and put on MySQL DB. It download movie covers also and more scrapers will be done on future.

   Python

5. dirbot-mysql dirbot-mysql Public

   Forked from rmax/dirbot-mysql

   Scrapy project based on dirbot to show how to use Twisted's adbapi to store the scraped data in MySQL.

   Python

6. scrapyd scrapyd Public

   Forked from scrapy/scrapyd

   A service daemon to run Scrapy spiders

   Python