Scrapy project

scrapy.org
The scrapy.org website

HTML 44 145 2 (1 issue needs help) 5 Updated Aug 27, 2021
scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.

Python 41,412 9,290 483 (31 issues need help) 292 Updated Aug 27, 2021
itemadapter
Common interface for data container classes

Python 26 BSD-3-Clause 5 2 3 Updated Aug 26, 2021
queuelib
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

Python 216 50 2 2 Updated Aug 26, 2021
xtractmime
https://mimesniff.spec.whatwg.org/ implementation for Python

Python 3 BSD-3-Clause 0 1 1 Updated Aug 24, 2021
w3lib
Python library of web-related functions

Python 334 85 15 (3 issues need help) 9 Updated Aug 24, 2021
cssselect
CSS Selectors for Python

Python 244 52 13 10 Updated Aug 18, 2021
parsel
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Python 686 106 27 (1 issue needs help) 14 Updated Aug 13, 2021
scrapyd-client
Command line client for Scrapyd server

Python 610 128 25 14 Updated Aug 11, 2021
protego
A pure-Python robots.txt parser with support for modern conventions.

DIGITAL Command Language 21 BSD-3-Clause 18 0 0 Updated Aug 3, 2021

Popular repositories