Cool project: "Nepenthes" is a tarpit to catch (AI) web crawlers.

Paul Cantrell

@tante @dalias
Heh, https://wookieepedia.org has been functioning this way for me I opened up its robots.txt: it's dynamically generated on demand, ALL links work, and ALL pages exist. It’s generated so much sustained load that I may need to throttle it too!

Aaron

@altruios Hi, author of Nepenthes here.

I respect your discomfort, but honestly I'm angry enough about their behavior I want to see them burn. There's been far too much of this:

https://mastodon.social/@khobochka/113724300122190730

I cannot trust traffic to my site to be harmless, so I don't see any reason why something connecting to every site on the internet should be able to trust the site isn't harmful.

@tante

Marsh Gardiner 🌱🐝

@inthehands @tante @dalias OMG, that is a spectacular use of "hirsute." Bravo.

woe2you

@dalias @altruios @tante They're crawling the web, running code against it and providing a service based on the results. How's that any different to what search engines have been doing for the last 30 years?

altruios phasma

@woe2you @dalias @tante data rates. They be ddos’ing people like they don’t care. That’s the difference. One crawl a week vs 100,000 every hour, or whatever the actual rate is…

I fundamentally have no problem with a crawler. I do not approve of crawlers that are that aggressive and disrespectful.

woe2you

@altruios @dalias @tante I agree that a misbehaving crawler is annoying. I'm just trying to understand why people seem to have a knee jerk reaction against LLMs in general.

Cassandrich

@woe2you @altruios @tante Because if we gobbled up millions of copyrighted works and produced derivatives of them, we'd be facing decades in prison, billions in fines. But when they seize our stuff that way and enclose it and use it to manufacture slop, in gross violation of copyright, it's deemed legitimate business.

Cassandrich

@woe2you @altruios @tante You should understand that LLMs have no legitimate purposes. They do not produce intelligence or knowledge or information. They produce *information-shaped* slop. The only way they get better is getting better at deceiving fools that the slop is what it looks like.

If you don't understand what they are, what they're doing, and how they do it, then kindly stop calling the reactions by people who do "knee-jerk".

woe2you

@dalias @altruios @tante I can talk to my smart home in natural language instead of having to sound like a BASIC program and get the name of every device precisely right, and it can respond in natural language. How is that not a "legitimate purpose" per your definition?

If you could explain without calling me an asshole or a simp that would be appreciated too.

Cassandrich

@woe2you @altruios @tante You don't need LLMs to do any of that.

Amber

@[email protected] @[email protected] @[email protected] @[email protected] yeah. Microsoft Power Automate, Scratch and several other "languages" I can think of on the top of my head that allow you to more or less skip the BASIC syntax and program things by moving blocks containing logic around. You don’t need an LLM for that and you shouldn’t use one either. tell an LLM to do 1+2 and it won’t calculate that answer. Instead it’ll hallucinate, I’ve seen it spit out 5 for 2+2 because that’s a common reference with 1984 and the radio head song 2+2=5

Amber

@[email protected] @[email protected] @[email protected] @[email protected] basic voice commands do not need an LLM or a language (I mean, command line programs don’t need to compile their arguments). voice detection isn’t built on the same thing.