Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Response never received errors #59

Open
mmotti opened this issue Dec 19, 2020 · 0 comments
Open

Response never received errors #59

mmotti opened this issue Dec 19, 2020 · 0 comments

Comments

@mmotti
Copy link

mmotti commented Dec 19, 2020

Does anyone else experience timeout errors, specifically immediately after redirects?

I've only set this up today, but specifically https://www.game.co.uk/en/hardware/xbox-series-x/?contentOnly=&inStockOnly=true&listerOnly=&pageSize=100

I can fetch it ok with Scrapy fetch, but if I try to use a spider that crawls the URL, I hit a 302 redirect and my crawl just completely errors out from that point with immediate failures "response never received". It's not long timeouts, it's literally just erroring immediately.

Please could somebody help me? I'm fairly new to this and I have no idea what the cause may be.

I'm using a pool of 10 http proxies on port 80

@mmotti mmotti changed the title Timeout errors Response never received errors Dec 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant