How to avoid data inconsistency

NodeBB Development
  • Hello. I wonder if nodebb structure does not favor data inconsistency

    What i see in user/create.js file

    Screenshot from 2019-10-11 18-24-56.png

    Different waterfall and parallel async calls, but why not use a single batch request directly to the database?
    Anything can happen during the execution of a function. If the server crash before the end of a function we will have data inconsistency, with incremented counters and inexistent new users or similar things.

    p.s: With MongoDB 4.0 is possible to execute ACID transactions. Postgre SQL can do the same, i have no idea about redis DB.

  • Should be possible but I am not sure if mongodb transactions work on standalone servers yet, I think they are only for sharded clusters and replica sets.

    In version 4.0, MongoDB supports multi-document transactions on replica sets.
    In version 4.2, MongoDB introduces distributed transactions, which adds support for multi-document transactions on sharded clusters and incorporates the existing support for multi-document transactions on replica sets.
    To use transactions on MongoDB 4.2 deployments(replica sets and sharded clusters), clients must use MongoDB drivers updated for MongoDB 4.2.

    More info here

    Redis has multi for transactions so we could aim for something like this in our dbal layer.

    const tx = db.transaction();
    tx.setObject('user:' + uid, data);
    tx.sortedSetAdd('users:joindate',, uid);
    await tx.exec();

    So both commands are executed atomically.

  • @baris yeah it's true, but you can use transaction on a multicluster with only one node, also if with a lot of limitations.
    About your command example is a good idea also if it would work only with redis and postgre, with mongodb can be used like a normal batch.

    I took a look to postgreSQL module, seems it support transactions, mongodb emulate the same thing with non ACID batch. I prefer to avoid SQL databases but the way data is organized seem more suitable to a relational DB

Suggested Topics