scraping-websites

It's been awhile since I updated e2e tests and there are some of them that are filing (most of them are related to examples).

Also, we need to add e2e tests that cover headers and cookies for both drivers.

Currently, in master, the logs are stored in a non-optional manner.

I think a good improvement for this would be to make the file logging optional, as not everyone has log rotation configured or needs logging (especially ad-hoc users).

Mentioned first here: elixir-crawly/crawly#155 (comment)

Some improvements i can think of is:

• optional switch to enabl

It would be beneficial to return the URL of the sitemap.xml file directly.

The idea is to have an option like 3 (Do a Google search, save the Urls found and search the emails), but search a list of phrases.

This list can be in a .txt
The option can ask for number of search results in Google

Is your feature request related to a problem? Please describe.
It would be interesting to capture if a listing is a Buy It Now or Auction listing and if Auction how many bids were on it.

Describe the solution you'd like
Another two columns on the df for "Listing Type" and "Number of Bids" where if BIN the latter should be 0. Really we could get away with just one column as all BIN wil

PR for download video feature will be welcome or the gem is only for audio