How to avoid data inconsistency



  • Hello. I wonder if nodebb structure does not favor data inconsistency

    What i see in user/create.js file

    Screenshot from 2019-10-11 18-24-56.png

    Different waterfall and parallel async calls, but why not use a single batch request directly to the database?
    Anything can happen during the execution of a function. If the server crash before the end of a function we will have data inconsistency, with incremented counters and inexistent new users or similar things.

    p.s: With MongoDB 4.0 is possible to execute ACID transactions. Postgre SQL can do the same, i have no idea about redis DB.



  • Should be possible but I am not sure if mongodb transactions work on standalone servers yet, I think they are only for sharded clusters and replica sets.

    In version 4.0, MongoDB supports multi-document transactions on replica sets.
    
    In version 4.2, MongoDB introduces distributed transactions, which adds support for multi-document transactions on sharded clusters and incorporates the existing support for multi-document transactions on replica sets.
    
    To use transactions on MongoDB 4.2 deployments(replica sets and sharded clusters), clients must use MongoDB drivers updated for MongoDB 4.2.
    

    More info here https://docs.mongodb.com/manual/core/transactions/

    Redis has multi for transactions so we could aim for something like this in our dbal layer.

    const tx = db.transaction();
    tx.setObject('user:' + uid, data);
    tx.sortedSetAdd('users:joindate', Date.now(), uid);
    await tx.exec();
    

    So both commands are executed atomically.



  • @baris yeah it's true, but you can use transaction on a multicluster with only one node, also if with a lot of limitations.
    About your command example is a good idea also if it would work only with redis and postgre, with mongodb can be used like a normal batch.

    I took a look to postgreSQL module, seems it support transactions, mongodb emulate the same thing with non ACID batch. I prefer to avoid SQL databases but the way data is organized seem more suitable to a relational DB



Suggested Topics

  • 9
  • 1
  • 24
  • 3
| |