The rise and fall of robots.txt
www.theverge.com
As unscrupulous AI companies crawl for more and more data, the basic social contract of the web is falling apart.
  • mark@programming.devEnglish
    4·
    9 months ago

    This seems to interestingly prove the point made by the person this is in reply to. Breaking laws come with consequences. Not caring about a robots.txt file doesn’t. But maybe it should.

    • Tlaloc_TemporalEnglish
      31·
      9 months ago

      My angle was more about all rules being social contructs, and said rules being important for the continued operation of society, but that’s a good angle too.

      Lots of laws don’t come with real punishments either, especially if you have money. We can change this too.