Data model of Chat / Purge history

floufille

Hi,

I'm trying to understand the database model of chat (with MongoDB) because I've a need to purge sometime the global chat.

I understand they are this kinds of object :

// The message with an id, a timestamp and the chat room id

{
"_id" : ObjectId("5a9a80d85626e2b836154d78"),
"_key" : "message:535",
"content" : "Moshi moshi !!!!",
"timestamp" : 1520074968489.0,
"fromuid" : 1,
"roomId" : 5
}

//The link between a message and a user

{
"_id" : ObjectId("5a9a8a945626e2b836155b22"),
"_key" : "uid:10:chat:room:5:mids",
"value" : "539",
"score" : 1520077460459.0
}

//The link between an instance of room and a user

{
"_id" : ObjectId("5a9a80915626e2b836154c46"),
"_key" : "uid:11:chat:rooms",
"value" : "5",
"score" : 1520077460459.0
}

So far so good.

And then, i don't understand the database model with notification.

They are this kind of objects, but I don't understand what i have to delete

{
"_id" : ObjectId("5a9a80d85626e2b836154d99"),
"_key" : "uid:10:chat:rooms:unread",
"value" : "5",
"score" : 1520077460467.0
}

//difference between the : {_key" : "uid:11:chat:rooms", "value" : "5",} ??

{
"_id" : ObjectId("5a9851e6ca66c7f2d7e75c87"),
"_key" : "chat:room:2:uids",
"value" : "9",
"score" : 1519992614981.0
}

And finally :

{
"_id" : ObjectId("5a994004907ed4104cd982bb"),
"_key" : "uid:12:notifications:read",
"value" : "chat_15_2",
"score" : 1519992629517.0
}

Someone can explains me what are the puposes of the three last objects ?

Thanks.

PitaJ

@floufille so each of those objects is a single entry in how the sorted sets work. A sorted set is queried by telling mongo to fetch a certain number of those objects.

The unread and read ones are entries on the sorted sets which store the state of the user's chat rooms. So if a room had no unread messages for a specific user, it shows up in :read otherwise :unread.

floufille

I take some time to analyse source code and I understand the difference between chat:room:2:uids and uid:11:chat:rooms

uid:11:chat:rooms is the link between a user and a room

chat:room:2:uids is used to know the number of user in a room

Messaging.getUserCountInRoom = function (roomId, callback) {
	db.sortedSetCard('chat:room:' + roomId + ':uids', callback);

And this, seems to notify people of the creation of a new room :

"_key" : "uid:12:notifications:read",
"value" : "chat_15_2",

I activate by error the global chat plugin : https://github.com/adlerosn/nodebb-plugin-chats-global#readme instead of the "good" global chat plugin : https://github.com/nodebb/nodebb-plugin-global-chat#readme

But, with the previous information, i succeed to migrate from plugin to another.
I know it is not advised to remove elements in MongoDB directly but it seems everyting is OK with the following mongodb Command

db.getCollection('objects').remove({_key: /chat:room:2:mids$/})
WriteResult({ "nRemoved" : 19246 })
db.getCollection('objects').remove({_key: /chat:rooms:unread$/, value: "2"})
WriteResult({ "nRemoved" : 19 })
db.getCollection('objects').remove({_key: "chat:room:2:uids"})
WriteResult({ "nRemoved" : 23 })
db.getCollection('objects').remove({_key: /chat:rooms$/, value: "2"})
WriteResult({ "nRemoved" : 23 })

(of course, 2 was the room id)

floufille

PS :

To anonymize global chat, this command seems to be good (if "2" is the room)

db.getCollection('objects').updateMany({roomId:"2", timestamp: { '$lte': new Date("2018-03-01").getTime()}}, {$set: {content:"Logical delete"}})
db.getCollection('objects').updateMany({roomId:2, timestamp: { '$lte': new Date("2018-03-01").getTime()}}, {$set: {content:"Logical delete"}})

(i don't know why, sometimes, roomId is integer, somtimes is a string)

faizanzahid

Is global chat live on node bb latest version? Are you reffering global chat = node bb private inbox chat?

or global chat is like shoutbox but its now in node bb core?

PitaJ

@faizanzahid I think he's referring to https://github.com/nodebb/nodebb-plugin-global-chat

faizanzahid

@pitaj Wow. How does this compare with shoutbox?

Is this better alternative to shoutbox? Coz our shoutbox works with latest version of node bb but it still has some small issues sometimes and warnings in logs.

@fais3000 we can try this global chat.

PitaJ

@faizanzahid the plugins have different purposes. Shoutbox provides the global chat widget for embedding, whereas this global chat plugin uses the internal NodeBB chat to implement a global room accessible anywhere, just like any vanilla chat room.

floufille

Yes, it is this plugin : https://github.com/nodebb/nodebb-plugin-global-chat

Some night, tongues are loosened, and it seems to be interesting to keep a "short history" (such as 24h) of the global chat.