Ask HN: Protecting your Sites/Services from Unwanted Traffic?

So... If PoW (proof-of-work) schemes like Anubis ultimately don't' work in practise, because Bots/Crawlers are increasingly using headless browsers, are able to solve captures, proofs, etc;

What options do we have realistically to filter out or block unwanted traffic?

What clever schemes can we come up that don't rely on centralised serices like Cloudflare?

6 points | by prologic 6 hours ago

5 comments

  • mstaoru 24 minutes ago
    Browsing is absolutely intolerable these days, with almost every website having a block screen, "we're checking your browser...", stuff like that. Lately I just close these websites immediately.
  • wanderingpixel 1 hour ago
    It depends on what you define as unwanted traffic.

    Personally I do not have an issue with bots as long as they behave and are not straight up malicious, so I rely on a combination of rate limiting, a fine-tuned OWASP CRS ruleset and an aggressive Fail2ban enforcement (hit 2 triggers and you get a 24 hour ban, 2 bans and you get banned for 30 days).

    My sites also make extensive use of static elements and caching.

  • doolta 1 minute ago
    [flagged]
  • lpsatwork 31 minutes ago
    [flagged]
  • xchili 5 hours ago
    [dead]