scraping

Changing the value of that setting has been seen to work around some bans, so it may be worth mentioning in https://docs.scrapy.org/en/master/topics/avoiding-bans.html#bypassing-web-browser-filters

If you're using proxies with requests-html and rendering JS sites is all good. Once you render a website pyppeteer don't know about this proxies and will expose your IP. This is an undesired behavior when scraping with proxies.

The idea is that whenever someone passes in proxies to the session object or any method call, make pyppeteer also use these proxies. #265

It's been awhile since I updated e2e tests and there are some of them that are filing (most of them are related to examples).

Also, we need to add e2e tests that cover headers and cookies for both drivers.

Main examples at Apify SDK webpage, Github repo and CLI templates should demonstrate how to manipulate with DOM and retrieve data from it.

Also add one example of scraping with Apify SDK + jQuery to https://sdk.apify.com/docs/examples/basiccrawler

Feedback from: https://medium.com/better-programming/do-i-need-python-scrapy-to-build-a-web-scraper-7cc7cac2081d

I lost an hour trying to make

My project have routing based on hosts. But web driver make request to http://127.0.0.1:9080.
How can i change host?

Hi,

I have read that it is possible to write extra parameters.
Where exactly do I do that?
Do I create a new file or can I add a line to my command in the terminal? Like --extra_info true?

### Optional parameters
*(For the 'get_posts' function)*.

- **group**: group id, to scrape groups instead of pages. Default is `None`.
- **pages**: how many pages of posts to request, usua