NodeBB anti spam

[bentael]

or should that filter live in here? where it would have a reference to the req object.

https://github.com/designcreateplay/NodeBB/blob/master/src/routes/authentication.js#L200

[psychobunny]

I think req.connection.remoteAddress could be passed into user.create, perhaps at some point in the future NodeBB will also store user's IP as well

[bentael]

@psychobunny that won't work if behind proxy, you need req.ip - BUT you also need express.enable('trust proxy'); if behind one

[bentael]

plugin pending this PR:

Changed User.create signature to accept a hash, added a couple of utilit... by akhoury · Pull Request #820 · NodeBB/NodeBB

...y functions as well, also added a new hook filter:user.create . EDIT: also added a new config use_proxy - I wasn't sure where exactly you would prefer that config to live, feel free to either move it or suggest a better place (db maybe?) - but it does require a restart, so I figured the config.json is not a bad idea.

GitHub (github.com)

needed that added hook and the req.ip to be plumbed in User.create

[baris]

Hopefully one of the early birds can take a look @julian @psychobunny

[psychobunny]

@julian is the early bird I'm going to bed (mooning)

[julian]

@bentael -- merged it in, but let me know what you think re: my comment, perhaps I am mistaken?

[bentael]

replied. Thanks.

[bentael]

@julian @baris @psychobunny
we may have a little problem, so I'm writing this anti-spam plugin, and I was able to successfully use the StopForumSpam.com api, and I wanted to use the filter:post.create hook to submit to Akismet as well, BUT, Akismet's API is very greedy, it wants all this stuff, i am not too sure we can easily plumb all that through, what do you think?

some we may not have, but still. I can get the req object in, but not too sure if you have something else in mind

blog url (required)
user_ip (required)
user_agent (required)
referrer (note spelling)
permalink
comment_type
comment_author
comment_author_email
comment_author_url
comment_content
comment_date_gmt
comment_post_modified_gmt
blog_lang
blog_charset

Should we also change the Post.create signature, but then we need to do the same for Topic.reply, Topic.post, and plumb all these info through.

thoughts?

[baris]

Might be better to do this before we call Topics.post or Posts.reply that way we won't have to change the signatures. Also all posting happens through sockets so not sure how you are going to access req.

[psychobunny]

I'm assuming a bot would submit spam from the same address as what they registered from, perhaps saving that to the user hash and then submitting that instead?

edit: don't quote me on this one, I have no idea how devious spammers can be

edit2 I'm actually looking forward to seeing how you pull off the spam/comment moderation system in ACP (ie. "here are your blocked/spammy messages please moderate the following"..) That definitely seems like a pretty decent undertaking compared to your problem above.

[Chris]

A week or so ago I was missing around with the akismet node npm I found (https://github.com/ryanscott/node-akismet) and if I recall it didn't actually require all of the above fields - it's just kind of "the more the better" sort of thing. I beleive I only used url, ip, agent and content in my testing and was able to get a valid return.

[julian]

Typically, I've been of the "if there's an npm module for it, go for it", rather than re-inventing the wheel.

[bentael]

url, ip, agent

@julian I will be using a module, but if you can get these in, why not add the rest for more accurate results, most of them are accessible from the req object - but then again, there is the sockets issue that @baris brought up.

@psychobunny I was hoping to get some sort of ACP panel to moderate the Posts and Registrations, along with augmenting the .post-tools to add a report as spam btn. but I have a feeling I'm going to hit some dead ends doing this outside of the core.
Honestly, I feel like this sort of structure should be built in core, on top of the Plugins infrastructure, just like the emailer "system", where nodebb would keep track of a property on each user as spammer == '1'/'0' (kinda like banned) and flag every post they post as spam = '1' automatically, but still fire off couple of hooks to notify the "plugins" that are installed to handle the Checking ONLY, but the decision of what to do with the spammers and spam post is up to NodeBB - similar to the WP structure, and again, the Emailer plugins-structures, so we would write something like these two instead.

nodebb-plugin-antispam-akismet
nodebb-plugin-antispam-stopforumspam

and the NodeBB.hooks would flag posts and users appropriately.

However, since the NodeBB structure is very flexible, and modular, I know that I can probably still DO most of that with a plugin, but I would have to Augment every user:{id} at each user.create, if flagged as spammer, but how would I gracefully prevent registration without throwing an error? or maybe let the user register, but only flag them as spammer.. but still let them know about it... same with post.create

[psychobunny]

tbh I like the idea of spammer flag in core. This way every other plugin doesn't have to reimplement this feature.

an upvote/downvote plugin could ideally flag someone as a spammer after they reached X amounts of downvotes and then hide their posts. but if that wasn't in the core already then I'd need your plugin for that plugin to work.

I was hoping to get some sort of ACP panel to moderate the Posts and Registrations, along with augmenting the .post-tools to add a report as spam btn. but I have a feeling I'm going to hit some dead ends doing this outside of the core.

So sounds like this should really be a core feature then. But imo isn't "banned" equivalent to "spammer"? What's the real distinction, maybe we can just use the banned flag since it's already implemented into user management.

[finid]

@psychobunny said:

an upvote/downvote plugin could ideally flag someone as a spammer after they reached X amounts of downvotes and then hide their posts. but if that wasn't in the core already then I'd need your plugin for that plugin to work.

Instead of depending on the number of down votes to hide a post, how about adding a Flag_This_Post feature that can be used to alert the admin that a post is spam and/or requires moderation. The up/down vote feature may then be reserved solely for judging the usefulness of a post to a topic or a topic itself. Does that make sense?

[Xiph]

A very simple solution to prevent a lot of automated signups without requiring a Captcha or Question/Answer form:

1. Honeypots. Simply have some input elements with the name attribute being something like username, password and the like and an id attribute either equal to or containing the same phrase. These inputs should not be visible to the user and contain no data, so whenever something is entered into these forms, you'll know it's a bot.
   Ideally these elements or their container should also include the actual labels shown to the user, to fool the bots looking at the label rather than the input in an attempt to bypass the prevention described in 2.
2. Have the actual input elements be unique to each session. Simply make the name and id attributes of the actual input forms be unique to each session. Shouldn't be too difficult, yet still blocks a lot of scripts that do not take this into consideration. This should not apply to the honeypot inputs described in 1., for obvious reasons.
3. Randomize the order of the input elements. Some bots look at the order of the input elements rather than their name and id attributes. Simply randomizing it should prevent this.

[bentael]

@Xiph all this is good, but doesn't prevent a targeted-script

[julian]

@bentael @Xiph unfortunately, no amount of automated prevention will stop people from hiring people from the third world to register accounts and post spam... but at least it'll weed out the automated ones!

[Xiph]

@bentael said:

@Xiph all this is good, but doesn't prevent a targeted-script

Well, the theory of using those three methods of prevention combined is that only A. a regular, normal human (sorry, mutants and/or half-cyborg jellyfish-themed superheroes) or B. a bot rendering the entire page can register.
Having to actually render the page does massively increase the cost of creating spam accounts though and as @julian said, if you really really want to spam you can always just have a human register, or do it yourself. None of this removes the need for a kind-of Akismet-ish thing as another layer of protection, it just massively reduces the amount of automated registrations.

EDIT It seems like XenForo does in fact allow you to check all posts with Akismet.