Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add crawler #513

Merged
merged 3 commits into from
Nov 11, 2023
Merged

add crawler #513

merged 3 commits into from
Nov 11, 2023

Conversation

IT-Joris
Copy link
Contributor

@IT-Joris IT-Joris commented Nov 2, 2023

No description provided.

@JayBizzle
Copy link
Owner

Failing test here @IT-Joris

I think you make need to escape the brackets in nbertaupete95(at)gmail.com

@IT-Joris
Copy link
Contributor Author

IT-Joris commented Nov 9, 2023

Hello @JayBizzle, I made the corrections on my branch.
Unfortunately the line Mozilla/5.0/Firefox/42.0 - nbertaupete95(at)gmail.com is present in the tests/devices.txt which makes the test user_agents_are_devices fail.

@JayBizzle
Copy link
Owner

Thanks, if you push those changes, ill rerun tests and get this merged 👍

@IT-Joris
Copy link
Contributor Author

Hello @JayBizzle, as mentioned above, the tests don't work because the user agent is present in the device.txt folder. How can I fix this?

Because the user agent is considered a crawler.

@JayBizzle
Copy link
Owner

Just remove that line from devices.txt as you have now added it to crawlers.txt 👍

@IT-Joris
Copy link
Contributor Author

The devices.txt file is read-only

@IT-Joris
Copy link
Contributor Author

Sorry @JayBizzle, my IDE was blocking editing of the file, as it was too large, so I pushed the correction and testing successful through on my end.

@JayBizzle
Copy link
Owner

Ha, yeah, that file is fairly large 😬

@JayBizzle JayBizzle merged commit 92ed056 into JayBizzle:master Nov 11, 2023
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants