LMM
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
fattyfoods@feddit.nl to Open Source@lemmy.ml · 1 day ago

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

external-link
message-square
101
link
fedilink
538
external-link

The Open-Source Software Saving the Internet From AI Bot Scrapers

www.404media.co

fattyfoods@feddit.nl to Open Source@lemmy.ml · 1 day ago
message-square
101
link
fedilink
Anubis, which block AI scrapers from scraping websites to death, has been downloaded almost 200,000 times.
  • unexposedhazard@discuss.tchncs.de
    link
    fedilink
    arrow-up
    102
    ·
    edit-2
    1 day ago

    Non paywalled link https://archive.is/VcoE1

    It basically boils down to making the browser do some cpu heavy calculations before allowing access. This is no problem for a single user, but for a bot farm this would increase the amount of compute power they need 100x or more.

    • exu@feditown.com
      link
      fedilink
      English
      arrow-up
      5
      ·
      9 hours ago

      It inherently blocks a lot of the simpler bots by requiring JavaScript as well.

    • Mubelotix@jlai.lu
      link
      fedilink
      arrow-up
      63
      ·
      1 day ago

      Exactly. It’s called proof-of-work and was originally invented to reduce spam emails but was later used by Bitcoin to control its growth speed

      • JackbyDev@programming.dev
        link
        fedilink
        English
        arrow-up
        4
        ·
        7 hours ago

        It’s funby that older captchas could be viewed as proof of work algorithms now because image recognition is so good. (From using captchas.)

        • Mubelotix@jlai.lu
          link
          fedilink
          arrow-up
          2
          ·
          edit-2
          2 hours ago

          Interesting stance. I have bought many tens of thousand of captcha soves for legitimate reasons, and I have now completely lost faith in them

      • RandomTester@lemmybefree.net
        link
        fedilink
        arrow-up
        3
        ·
        14 hours ago

        it wasn’t made for bitcoin originally? didn’t know that!

        • 0xebfe@lemmy.today
          link
          fedilink
          arrow-up
          9
          arrow-down
          1
          ·
          12 hours ago

          Originally called hashcash: http://hashcash.org/

          • RandomTester@lemmybefree.net
            link
            fedilink
            arrow-up
            8
            ·
            12 hours ago

            you know it’s old when it doesn’t have ssl

          • daw@feddit.org
            link
            fedilink
            arrow-up
            2
            ·
            9 hours ago

            TIL

    • lazynooblet@lazysoci.al
      link
      fedilink
      English
      arrow-up
      6
      ·
      1 day ago

      Thank you for the link. Good read

Open Source@lemmy.ml

opensource@lemmy.ml

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !opensource@lemmy.ml

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

  • Open Source Initiative
  • Free Software Foundation
  • Electronic Frontier Foundation
  • Software Freedom Conservancy
  • It’s FOSS
  • Android FOSS Apps Megathread

Rules

  • Posts must be relevant to the open source ideology
  • No NSFW content
  • No hate speech, bigotry, etc

Related Communities

  • !libre_culture@lemmy.ml
  • !libre_software@lemmy.ml
  • !libre_hardware@lemmy.ml
  • !linux@lemmy.ml
  • !technology@lemmy.ml

Community icon from opensource.org, but we are not affiliated with them.

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 567 users / day
  • 1.55K users / week
  • 3.29K users / month
  • 8.2K users / 6 months
  • 1 local subscriber
  • 38.7K subscribers
  • 531 Posts
  • 4.19K Comments
  • Modlog
  • mods:
  • Evan@lemmy.ml
  • kevincox@lemmy.ml
  • CrypticCoffee@lemmy.ml
  • Lettuce eat lettuce@lemmy.ml
  • BE: 0.19.12
  • Modlog
  • Legal
  • Instances
  • Docs
  • Code
  • join-lemmy.org