NodeBB anti spam

[datagutt]

If the NodeBB developers choose to implement stopforumspam.com, here is a node module for it: https://github.com/dkleehammer/node-stopforumspam

[julian]

@datagutt :

here is a node module for it: https://github.com/dkleehammer/node-stopforumspam

Now that's what I like to hear...

[bentael]

@datagutt said:

If the NodeBB developers choose to implement stopforumspam.com, here is a node module for it: https://github.com/dkleehammer/node-stopforumspam

taking a shot at that today if no one beats me to it.

Stay tuned.

[bentael]

hmm ....

doesn't really provide post.content analysis, just username, email and IP, is that really what we need?

EDIT: unless you want this check on User creation. @julian can we sneak in a filter:user.create but where do I get the req.connection.remoteAddress from? is the express.req available anywhere at this point?

I was thinking something like that in User.create .. async.parallel call

			function(next) {
				plugins.fireHook('filter:user.create', {username: username, email: email, /* req: req, and probably leave the password out, or not? */}, function(err){
					next(err);
				});
			}

[bentael]

or should that filter live in here? where it would have a reference to the req object.

https://github.com/designcreateplay/NodeBB/blob/master/src/routes/authentication.js#L200

[psychobunny]

I think req.connection.remoteAddress could be passed into user.create, perhaps at some point in the future NodeBB will also store user's IP as well

[bentael]

@psychobunny that won't work if behind proxy, you need req.ip - BUT you also need express.enable('trust proxy'); if behind one

[bentael]

plugin pending this PR: https://github.com/designcreateplay/NodeBB/pull/820 needed that added hook and the req.ip to be plumbed in User.create

[baris]

Hopefully one of the early birds can take a look @julian @psychobunny

[psychobunny]

@julian is the early bird I'm going to bed (mooning)

[julian]

@bentael -- merged it in, but let me know what you think re: my comment, perhaps I am mistaken?

[bentael]

replied. Thanks.

[bentael]

@julian @baris @psychobunny
we may have a little problem, so I'm writing this anti-spam plugin, and I was able to successfully use the StopForumSpam.com api, and I wanted to use the filter:post.create hook to submit to Akismet as well, BUT, Akismet's API is very greedy, it wants all this stuff, i am not too sure we can easily plumb all that through, what do you think?

some we may not have, but still. I can get the req object in, but not too sure if you have something else in mind

blog url (required)
user_ip (required)
user_agent (required)
referrer (note spelling)
permalink
comment_type
comment_author
comment_author_email
comment_author_url
comment_content
comment_date_gmt
comment_post_modified_gmt
blog_lang
blog_charset

Should we also change the Post.create signature, but then we need to do the same for Topic.reply, Topic.post, and plumb all these info through.

thoughts?

[baris]

Might be better to do this before we call Topics.post or Posts.reply that way we won't have to change the signatures. Also all posting happens through sockets so not sure how you are going to access req.

[psychobunny]

I'm assuming a bot would submit spam from the same address as what they registered from, perhaps saving that to the user hash and then submitting that instead?

edit: don't quote me on this one, I have no idea how devious spammers can be

edit2 I'm actually looking forward to seeing how you pull off the spam/comment moderation system in ACP (ie. "here are your blocked/spammy messages please moderate the following"..) That definitely seems like a pretty decent undertaking compared to your problem above.

[Chris]

A week or so ago I was missing around with the akismet node npm I found (https://github.com/ryanscott/node-akismet) and if I recall it didn't actually require all of the above fields - it's just kind of "the more the better" sort of thing. I beleive I only used url, ip, agent and content in my testing and was able to get a valid return.

[julian]

Typically, I've been of the "if there's an npm module for it, go for it", rather than re-inventing the wheel.

[bentael]

url, ip, agent

@julian I will be using a module, but if you can get these in, why not add the rest for more accurate results, most of them are accessible from the req object - but then again, there is the sockets issue that @baris brought up.

@psychobunny I was hoping to get some sort of ACP panel to moderate the Posts and Registrations, along with augmenting the .post-tools to add a report as spam btn. but I have a feeling I'm going to hit some dead ends doing this outside of the core.
Honestly, I feel like this sort of structure should be built in core, on top of the Plugins infrastructure, just like the emailer "system", where nodebb would keep track of a property on each user as spammer == '1'/'0' (kinda like banned) and flag every post they post as spam = '1' automatically, but still fire off couple of hooks to notify the "plugins" that are installed to handle the Checking ONLY, but the decision of what to do with the spammers and spam post is up to NodeBB - similar to the WP structure, and again, the Emailer plugins-structures, so we would write something like these two instead.

nodebb-plugin-antispam-akismet
nodebb-plugin-antispam-stopforumspam

and the NodeBB.hooks would flag posts and users appropriately.

However, since the NodeBB structure is very flexible, and modular, I know that I can probably still DO most of that with a plugin, but I would have to Augment every user:{id} at each user.create, if flagged as spammer, but how would I gracefully prevent registration without throwing an error? or maybe let the user register, but only flag them as spammer.. but still let them know about it... same with post.create

[psychobunny]

tbh I like the idea of spammer flag in core. This way every other plugin doesn't have to reimplement this feature.

an upvote/downvote plugin could ideally flag someone as a spammer after they reached X amounts of downvotes and then hide their posts. but if that wasn't in the core already then I'd need your plugin for that plugin to work.

I was hoping to get some sort of ACP panel to moderate the Posts and Registrations, along with augmenting the .post-tools to add a report as spam btn. but I have a feeling I'm going to hit some dead ends doing this outside of the core.

So sounds like this should really be a core feature then. But imo isn't "banned" equivalent to "spammer"? What's the real distinction, maybe we can just use the banned flag since it's already implemented into user management.

[finid]

@psychobunny said:

an upvote/downvote plugin could ideally flag someone as a spammer after they reached X amounts of downvotes and then hide their posts. but if that wasn't in the core already then I'd need your plugin for that plugin to work.

Instead of depending on the number of down votes to hide a post, how about adding a Flag_This_Post feature that can be used to alert the admin that a post is spam and/or requires moderation. The up/down vote feature may then be reserved solely for judging the usefulness of a post to a topic or a topic itself. Does that make sense?