0

I am playing around with a new tool for web dev, Arcjet, which allows you to do rate limiting and bot attack prevention basically, really easily.

So I plugged it in with their default settings, which are basically these:

  • refillRate: 5, // Refill 5 tokens per interval
  • interval: 10, // Refill every 10 seconds
  • capacity: 10, // Bucket capacity of 10 tokens

That looks like this, hard refreshing CMD+R 5-10 times quickly, and you see the error page:

enter image description here

That to me feels too intrusive, too harsh. It's like, as a normal user, you should never see this "rate limiting" error page, I don't think. At least I rarely ever see it on other websites, but when I have (Elsevier or CiteseerX sometimes will show a "limit reached" page). I can't find their error pages, but it is something like this (static page):

enter image description here

So my question is, what is the recommended UX for rate limiting and showing an error page to block spammers?

Rate limiting is only necessary to prevent machines from using the site too fast (scraping 10 pages per second sort of thing). But how can you tell the difference between an anonymous user in the browser hitting CMD+R 20 times fast, and a bot? I don't know.

So it seems to boil down to some degree to how quickly you run out of tokens. (Using the tokenBucket algorithm). They have other rate limiting algorithms:

  • Fixed window: This is the simplest algorithm. It tracks the number of requests made by a client over a fixed time window e.g. 60 seconds. If the client exceeds the limit, they are blocked until the window expires.
  • Sliding window: This algorithm tracks the number of requests made by a client over a sliding window so that the window moves with time.
  • Token bucket....

I think I like the token bucket approach, so every 10 seconds it gets refilled by say 20, and so if they use all 20 in 5 seconds, then they are locked out for only 5 seconds. Whereas the the fixed window is a little harsher, so if you hit 30 requests in 10 seconds, you might be locked out for a minute. I dunno.

So what is the UX here? How should the math work out? Should you show an error page at all?

You need to do rate limiting to prevent bots, but maybe try and avoid it for users in a browser in some creative way?

What are your suggestions?

Improve this question
2
  • What is the application about? Do you need real time connectivity between users? What is the security level expected – does it have to be super strict? Can bot pings not be handled in some other way so that users do not have to suffer? Commented Nov 6, 2024 at 5:21
  • Rule of thumb: 1-10 seconds require loading spinner, 10+ seconds need progress status bars or error messages if there is an error. Commented Nov 6, 2024 at 5:24

1 Answer 1

Reset to default
1

Arcjet bot protection and rate limiting tend to be combined to provide defense in depth. Bot detection can never be 100% perfect - some bots will be able to bypass it - so that's where rate limiting comes in. Additional protection.

The Arcjet Next.js example returns an error as part of an API route response. That can then be used in a form to show an error within the form design. If you're using an API route you can also return rate limit headers.

If you're using server actions then you can do something similar. Server components will require you to return an error component or throw an error to be handled by the global error handler.

So it depends if you're optimizing the rate limit error for humans or machines. With machine you can return an HTTP error code and optionally the headers if you want to be nice (e.g. if it's an API and you want your users to have useful info). For humans, an error message is probably sufficient.

For bot detection, a redirect to a generic error or throwing a global error would work. Humans should never see it and the goal is to block bots as quickly as possible.

Improve this answer
1
  • "Humans should never see it and the goal is to block bots as quickly as possible." Can you elaborate on that? How can you achieve both goals simultaneously? Commented Nov 9, 2024 at 4:00

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.