crawler fix: setting start_url_absolute to be the return value of do_request so that a trailing slash is added for directories. We need this…
Show more
crawler fix: setting start_url_absolute to be the return value of do_request so that a trailing slash is added for directories. We need this to resolve relative URLs
Show less
