NodeBB anti spam

julian

Typically, I've been of the "if there's an npm module for it, go for it", rather than re-inventing the wheel.

bentael

url, ip, agent

@julian I will be using a module, but if you can get these in, why not add the rest for more accurate results, most of them are accessible from the req object - but then again, there is the sockets issue that @baris brought up.

@psychobunny I was hoping to get some sort of ACP panel to moderate the Posts and Registrations, along with augmenting the .post-tools to add a report as spam btn. but I have a feeling I'm going to hit some dead ends doing this outside of the core.
Honestly, I feel like this sort of structure should be built in core, on top of the Plugins infrastructure, just like the emailer "system", where nodebb would keep track of a property on each user as spammer == '1'/'0' (kinda like banned) and flag every post they post as spam = '1' automatically, but still fire off couple of hooks to notify the "plugins" that are installed to handle the Checking ONLY, but the decision of what to do with the spammers and spam post is up to NodeBB - similar to the WP structure, and again, the Emailer plugins-structures, so we would write something like these two instead.

nodebb-plugin-antispam-akismet
nodebb-plugin-antispam-stopforumspam

and the NodeBB.hooks would flag posts and users appropriately.

However, since the NodeBB structure is very flexible, and modular, I know that I can probably still DO most of that with a plugin, but I would have to Augment every user:{id} at each user.create, if flagged as spammer, but how would I gracefully prevent registration without throwing an error? or maybe let the user register, but only flag them as spammer.. but still let them know about it... same with post.create

psychobunny

tbh I like the idea of spammer flag in core. This way every other plugin doesn't have to reimplement this feature.

an upvote/downvote plugin could ideally flag someone as a spammer after they reached X amounts of downvotes and then hide their posts. but if that wasn't in the core already then I'd need your plugin for that plugin to work.

I was hoping to get some sort of ACP panel to moderate the Posts and Registrations, along with augmenting the .post-tools to add a report as spam btn. but I have a feeling I'm going to hit some dead ends doing this outside of the core.

So sounds like this should really be a core feature then. But imo isn't "banned" equivalent to "spammer"? What's the real distinction, maybe we can just use the banned flag since it's already implemented into user management.

finid

@psychobunny said:

an upvote/downvote plugin could ideally flag someone as a spammer after they reached X amounts of downvotes and then hide their posts. but if that wasn't in the core already then I'd need your plugin for that plugin to work.

Instead of depending on the number of down votes to hide a post, how about adding a Flag_This_Post feature that can be used to alert the admin that a post is spam and/or requires moderation. The up/down vote feature may then be reserved solely for judging the usefulness of a post to a topic or a topic itself. Does that make sense?

Xiph

A very simple solution to prevent a lot of automated signups without requiring a Captcha or Question/Answer form:

Honeypots. Simply have some input elements with the name attribute being something like username, password and the like and an id attribute either equal to or containing the same phrase. These inputs should not be visible to the user and contain no data, so whenever something is entered into these forms, you'll know it's a bot.
Ideally these elements or their container should also include the actual labels shown to the user, to fool the bots looking at the label rather than the input in an attempt to bypass the prevention described in 2.
Have the actual input elements be unique to each session. Simply make the name and id attributes of the actual input forms be unique to each session. Shouldn't be too difficult, yet still blocks a lot of scripts that do not take this into consideration. This should not apply to the honeypot inputs described in 1., for obvious reasons.
Randomize the order of the input elements. Some bots look at the order of the input elements rather than their name and id attributes. Simply randomizing it should prevent this.

bentael

@Xiph all this is good, but doesn't prevent a targeted-script

julian

@bentael @Xiph unfortunately, no amount of automated prevention will stop people from hiring people from the third world to register accounts and post spam... but at least it'll weed out the automated ones!

Xiph

@bentael said:

@Xiph all this is good, but doesn't prevent a targeted-script

Well, the theory of using those three methods of prevention combined is that only A. a regular, normal human (sorry, mutants and/or half-cyborg jellyfish-themed superheroes) or B. a bot rendering the entire page can register.
Having to actually render the page does massively increase the cost of creating spam accounts though and as @julian said, if you really really want to spam you can always just have a human register, or do it yourself. None of this removes the need for a kind-of Akismet-ish thing as another layer of protection, it just massively reduces the amount of automated registrations.

EDIT It seems like XenForo does in fact allow you to check all posts with Akismet.

fantapop

@bentael You still working on this?

bentael

@fantapop this plugin is currently blocked while waiting for some desgin descisions. i'll chat with NodeBB team to see whats the best way to go about this.

julian

req.ip passed into user registration now, and @bentael, you can use my honey pot plugin to query IPs for validity.

bentael

@fantapop I just released a sort of working version,
https://www.npmjs.org/package/nodebb-plugin-spam-be-gone
this one only work with the Honeypot Project, i would appreciate some testing help. i didn't get a chance to actually test real IPs, only tested with 127.0.0.1 which is close to useless. Tomorrow, I'll try to hardcode fake spammy IPs for better coverage.

@julian, I got the Honeypot plugin to work, nice job on that nodejs module, but that only works for User registration, I was aiming for post creations as well,

I need a hook on Post.create and/or Topic.post or Topic.reply to get a check with Akismet, but Akismet requires at least these 3

            user_ip: req.ip ,
            user_agent: req.get('User-Agent'),
            blog: req.host,

I could also use
postData.user.username and postData.content

let me know what you think is the best approach and I can add the hooks and submit a PR.

bentael

So, I don't know if you want to plumb through the req object, or maybe implement the filter on a higher level, maybe at the POST /api/{route} handler

julian

Seems legit.

Topic.post and Topic.reply are both standalone functions, but to follow DRY, Topic.post calls Topic.reply
Topic.reply calls Posts.create

So I believe putting a hook in at the Posts.create level (here, as the first waterfall function) would be easiest.

I suggest filter:posts.check or filter:posts.validate as a name. filter:posts.save is already there, but the post creation process has already begun, so it is used mostly for munging data after it is destined to enter the database.

bentael

@julian ok, Post.create sounds like a good place, but can I guarantee that the req object is going to be plumbed in from all the paths?

julian

Hm... since posting is done via web sockets, we don't have the same data we would have in req... would have to example the socket data and see what we have to work with...

bentael

ok i'll investigate and post back

<baris>

I don't think we should put it in Posts.create as that is the internal function used by importers as well, Topics.reply seems like a better place as it is the one that is called by users and Topics.post calls that too.

bentael

okay.png

julian

Haha good thing we have @baris watching out for the wrong things I say