Timeline for Cloudflare blocking Wayback Machine from archiving Stack Exchange questions
Current License: CC BY-SA 4.0
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 23 at 18:19 | comment | added | Josh Zhang StaffMod | @Vikki I just audited a weeks worth of traffic from Archive.org, no legitimate request was blocked or rate limited, which means it never hit the general site rate limiter or any rule. As a rule, we don't poke holes in the base rate limiter that keeps the app from crashing. | |
| May 23 at 1:56 | comment | added | Vikki | @JoshZhang: Wouldn't be possible, the Wayback Machine's archiver won't hit the same page again until it's been an hour since the last time it went there. | |
| May 21 at 13:11 | comment | added | Josh Zhang StaffMod | @Vikki no traffic is exempt from ALL rate limiters, otherwise people could abuse the archive function en masse to bring the site down. The general site rate limiters applies to all traffic regardless of source. | |
| May 20 at 23:24 | comment | added | Vikki | @JoshZhang I thought the Wayback Machine was exempted from the site rate-limiter? | |
| May 15 at 0:04 | comment | added | Josh Zhang StaffMod | @Starship it's hard to say for sure but it's possible their bot hit the main site rate limiter, without a RayID I can't tell for sure. | |
| May 14 at 22:33 | comment | added | Starship | @JoshZhang Don't see the RayID. Here's an example of what happens when wayback can complete the archive, but archives the captcha page instead and I don't see a RayID. And I have no idea how I'd look at the bottom of the page for a page which wayback machine never was able to visit (and hence record for me to see) | |
| May 14 at 21:17 | comment | added | Josh Zhang StaffMod | @Starship sorry I always call it RayID but it's actually CF-Ray, see meta.stackexchange.com/a/403462/784098. In the screen shot above, you see the RayID at the bottom. | |
| May 14 at 20:34 | comment | added | Starship | @JoshZhang How does one find RayID? | |
| May 14 at 20:34 | comment | added | Josh Zhang StaffMod | @Starship I'd need more details like a RayID in order to troubleshoot the errors you're getting. | |
| May 13 at 23:57 | comment | added | Starship | I'm still get a lot "Error! Job failed." when the Wayback Machine attempts to archive outlinks that are questions, tags, or users on SO. Any idea why? And on SE in general I get error "We’re currently facing some limitations when it comes to archiving this site. We apologize for any inconvenience this might cause and appreciate your understanding. Please email us at "[email protected]" if you would like to discuss this more." | |
| May 8 at 23:11 | comment | added | Vikki | Can confirm that Wayback Machine archiving is working now. Much appreciated! | |
| May 8 at 23:10 | vote | accept | Vikki | ||
| May 8 at 13:13 | history | answered | Josh ZhangStaffMod | CC BY-SA 4.0 |