Help Fine Tuning Advanced Cache Settings

  • NodeBB

    Can you post your stats? How many categories/groups do you have?

    The group cache is used to store the membership of users in groups(and privileges). For example checking if a user is an administrator is done by checking if they are member of the administrators group. The results are cached since group membership rarely changes.
    If the hit ratio is low you might have lots of active registered users and/or only a single nodebb instance. Each nodebb process has it's own cache and if users are distributed to these nodebb instances evenly the group cache will have a higher hit ratio.


  • Sure, here you go.

    Screen Shot 2021-02-24 at 10.02.12 AM.png

    Right now, we have 12 active categories, 10 groups.


  • I toggled the radio buttons. I guess that deleted the cache. I haven't done it for Group Cache.


  • or maybe I hit trash when I was tinkering with it.

  • NodeBB

    The toggle lets you enable/disable the cache, the delete button just clears it's contents.

    You can download the contents of the cache as well.

    When you clear the group cache how long does it take to make that many misses?

    How many users are on your nodebb? You can check /admin/development/info for more detailed stats.

    My guess is there is some code that is making a lot of group membership checks maybe a plugin. Or you just have a lot of active users.

  • NodeBB

    You can download the contents of the group cache and examine it to get more info about what is being cached as well.


  • @baris I deleted the Group Cache and it's full after 2 secs.

  • NodeBB

    @baris said in Help Fine Tuning Advanced Cache Settings:

    /admin/development/info

    Did you check this page? Share that one as well.


  • @baris here's our development/info stats.

    -deleted image link-

    The image dimensions were too big to upload.


  • ip-10-111-250-67:4567	 / 	7985	v10.17.0	82 / 4 / 93	[email protected]	1244.30 / 39.95	292.63 mb / 10635.03 mb	1.05, 1.64, 1.87	57m
    ip-10-111-250-67:4568	 / 	7986	v10.17.0	89 / 7 / 106	[email protected]	1176.16 / 38.71	340.14 mb / 10650.05 mb	1.05, 1.64, 1.87	57m
    ip-10-111-250-67:4569	 / 	7988	v10.17.0	106 / 12 / 150	[email protected]	1395.36 / 43.34	369.73 mb / 10650.05 mb	1.05, 1.64, 1.87	57m
    ip-10-111-250-67:4570	 / 	8005	v10.17.0	97 / 5 / 123	[email protected]	1431.64 / 45.03	312.77 mb / 10650.05 mb	1.05, 1.64, 1.87	57m
    
    [
        {
            "process": {
                "port": "4567",
                "pid": 7985,
                "title": "/usr/bin/node",
                "version": "v10.17.0",
                "memoryUsage": {
                    "rss": 331522048,
                    "heapTotal": 270622720,
                    "heapUsed": 191571224,
                    "external": 23467064,
                    "humanReadable": "316.16"
                },
                "uptime": 3516.246,
                "cpuUsage": {
                    "user": "1258.65",
                    "system": "40.50"
                },
                "uptimeHumanReadable": "58m"
            },
            "os": {
                "hostname": "ip-10-111-250-67",
                "type": "Linux",
                "platform": "linux",
                "arch": "x64",
                "release": "5.4.0-1029-aws",
                "load": "1.70, 1.72, 1.89",
                "freemem": "10625.66",
                "totalmem": "16481.63"
            },
            "nodebb": {
                "isCluster": true,
                "isPrimary": true,
                "runJobs": true,
                "jobsDisabled": false
            },
            "git": {
                "hash": "d7044a5e8623ae6b95360ff58e8f6dbc0e90629b",
                "hashShort": "d7044a",
                "branch": "v1.16.x"
            },
            "stats": {
                "onlineGuestCount": 4,
                "onlineRegisteredCount": 82,
                "socketCount": 93,
                "users": {
                    "categories": 0,
                    "recent": 1,
                    "unread": 0,
                    "topics": 39,
                    "category": 41
                },
                "topics": [
                    {
                        "tid": "26200",
                        "count": 5
                    },
                    {
                        "tid": "26202",
                        "count": 4
                    },
                    {
                        "tid": "26203",
                        "count": 3
                    },
                    {
                        "tid": "26747",
                        "count": 3
                    },
                    {
                        "tid": "26359",
                        "count": 2
                    },
                    {
                        "tid": "26199",
                        "count": 2
                    },
                    {
                        "tid": "26211",
                        "count": 2
                    },
                    {
                        "tid": "26901",
                        "count": 2
                    },
                    {
                        "tid": "26634",
                        "count": 1
                    },
                    {
                        "tid": "25866",
                        "count": 1
                    }
                ]
            },
            "id": "ip-10-111-250-67:4567"
        },
        {
            "process": {
                "port": "4568",
                "pid": 7986,
                "title": "/usr/bin/node",
                "version": "v10.17.0",
                "memoryUsage": {
                    "rss": 370384896,
                    "heapTotal": 302641152,
                    "heapUsed": 261548072,
                    "external": 22358009,
                    "humanReadable": "353.23"
                },
                "uptime": 3516.239,
                "cpuUsage": {
                    "user": "1204.29",
                    "system": "39.61"
                },
                "uptimeHumanReadable": "58m"
            },
            "os": {
                "hostname": "ip-10-111-250-67",
                "type": "Linux",
                "platform": "linux",
                "arch": "x64",
                "release": "5.4.0-1029-aws",
                "load": "1.70, 1.72, 1.89",
                "freemem": "10625.66",
                "totalmem": "16481.63"
            },
            "nodebb": {
                "isCluster": true,
                "isPrimary": false,
                "runJobs": false,
                "jobsDisabled": false
            },
            "git": {
                "hash": "d7044a5e8623ae6b95360ff58e8f6dbc0e90629b",
                "hashShort": "d7044a",
                "branch": "v1.16.x"
            },
            "stats": {
                "onlineGuestCount": 5,
                "onlineRegisteredCount": 93,
                "socketCount": 110,
                "users": {
                    "categories": 2,
                    "recent": 2,
                    "unread": 0,
                    "topics": 41,
                    "category": 48
                },
                "topics": [
                    {
                        "tid": "26747",
                        "count": 7
                    },
                    {
                        "tid": "26201",
                        "count": 6
                    },
                    {
                        "tid": "26200",
                        "count": 3
                    },
                    {
                        "tid": "26202",
                        "count": 2
                    },
                    {
                        "tid": "26922",
                        "count": 1
                    },
                    {
                        "tid": "26359",
                        "count": 1
                    },
                    {
                        "tid": "26794",
                        "count": 1
                    },
                    {
                        "tid": "25999",
                        "count": 1
                    },
                    {
                        "tid": "26210",
                        "count": 1
                    },
                    {
                        "tid": "26136",
                        "count": 1
                    }
                ]
            },
            "id": "ip-10-111-250-67:4568"
        },
        {
            "process": {
                "port": "4569",
                "pid": 7988,
                "title": "/usr/bin/node",
                "version": "v10.17.0",
                "memoryUsage": {
                    "rss": 365465600,
                    "heapTotal": 299364352,
                    "heapUsed": 262696736,
                    "external": 23254567,
                    "humanReadable": "348.54"
                },
                "uptime": 3516.235,
                "cpuUsage": {
                    "user": "1411.07",
                    "system": "43.92"
                },
                "uptimeHumanReadable": "58m"
            },
            "os": {
                "hostname": "ip-10-111-250-67",
                "type": "Linux",
                "platform": "linux",
                "arch": "x64",
                "release": "5.4.0-1029-aws",
                "load": "1.70, 1.72, 1.89",
                "freemem": "10625.66",
                "totalmem": "16481.63"
            },
            "nodebb": {
                "isCluster": true,
                "isPrimary": false,
                "runJobs": false,
                "jobsDisabled": false
            },
            "git": {
                "hash": "d7044a5e8623ae6b95360ff58e8f6dbc0e90629b",
                "hashShort": "d7044a",
                "branch": "v1.16.x"
            },
            "stats": {
                "onlineGuestCount": 12,
                "onlineRegisteredCount": 105,
                "socketCount": 151,
                "users": {
                    "categories": 2,
                    "recent": 4,
                    "unread": 0,
                    "topics": 69,
                    "category": 48
                },
                "topics": [
                    {
                        "tid": "26200",
                        "count": 8
                    },
                    {
                        "tid": "26747",
                        "count": 8
                    },
                    {
                        "tid": "26201",
                        "count": 3
                    },
                    {
                        "tid": "26656",
                        "count": 3
                    },
                    {
                        "tid": "26927",
                        "count": 3
                    },
                    {
                        "tid": "26533",
                        "count": 2
                    },
                    {
                        "tid": "26209",
                        "count": 2
                    },
                    {
                        "tid": "26199",
                        "count": 2
                    },
                    {
                        "tid": "26727",
                        "count": 2
                    },
                    {
                        "tid": "26901",
                        "count": 2
                    }
                ]
            },
            "id": "ip-10-111-250-67:4569"
        },
        {
            "process": {
                "port": "4570",
                "pid": 8005,
                "title": "/usr/bin/node",
                "version": "v10.17.0",
                "memoryUsage": {
                    "rss": 336445440,
                    "heapTotal": 272719872,
                    "heapUsed": 218833992,
                    "external": 22805176,
                    "humanReadable": "320.86"
                },
                "uptime": 3516.232,
                "cpuUsage": {
                    "user": "1454.85",
                    "system": "45.98"
                },
                "uptimeHumanReadable": "58m"
            },
            "os": {
                "hostname": "ip-10-111-250-67",
                "type": "Linux",
                "platform": "linux",
                "arch": "x64",
                "release": "5.4.0-1029-aws",
                "load": "1.70, 1.72, 1.89",
                "freemem": "10616.37",
                "totalmem": "16481.63"
            },
            "nodebb": {
                "isCluster": true,
                "isPrimary": false,
                "runJobs": false,
                "jobsDisabled": false
            },
            "git": {
                "hash": "d7044a5e8623ae6b95360ff58e8f6dbc0e90629b",
                "hashShort": "d7044a",
                "branch": "v1.16.x"
            },
            "stats": {
                "onlineGuestCount": 5,
                "onlineRegisteredCount": 101,
                "socketCount": 128,
                "users": {
                    "categories": 1,
                    "recent": 2,
                    "unread": 0,
                    "topics": 63,
                    "category": 46
                },
                "topics": [
                    {
                        "tid": "26200",
                        "count": 10
                    },
                    {
                        "tid": "26747",
                        "count": 6
                    },
                    {
                        "tid": "26201",
                        "count": 6
                    },
                    {
                        "tid": "26199",
                        "count": 5
                    },
                    {
                        "tid": "26359",
                        "count": 4
                    },
                    {
                        "tid": "26203",
                        "count": 4
                    },
                    {
                        "tid": "26210",
                        "count": 3
                    },
                    {
                        "tid": "26728",
                        "count": 2
                    },
                    {
                        "tid": "26656",
                        "count": 2
                    },
                    {
                        "tid": "26204",
                        "count": 2
                    }
                ]
            },
            "id": "ip-10-111-250-67:4570"
        }
    ]
    
  • NodeBB

    Your stats look fine, you have around ~100-150 users per nodebb instance.

    Do you have custom plugins? I suggest clearing the group cache and then downloading it after it is full in 2 seconds. Then you can check what is being cached.


  • @baris no custom plugins.

    Screen Shot 2021-02-24 at 10.22.19 AM.png

    Okay, i'll do what you suggested.


  • @baris Can I increase group cache size? just curious.

    I'm reviewing what's on the cache dump. Looks like lots of permission stuff. We have multiple moderators per category. Maybe that's adding up to that list. I can probably clean that up and see how it looks after.

  • NodeBB

    Sure you can change it in code, https://github.com/NodeBB/NodeBB/blob/master/src/groups/cache.js#L8.

    Modify the 40000 and restart all instances.


  • @baris Thanks! I'll try to cleanup our privileges first. If that doesn't help, I'll tweak the value in code.

  • Community Rep

    @baris said in Help Fine Tuning Advanced Cache Settings:

    @baris said in Help Fine Tuning Advanced Cache Settings:

    /admin/development/info

    New one for me. And pretty nice. Is this documented somewhere? Or is it one of those top seekrit dev thangs...

    Thx-- ;-)


  • @baris Okay, I did 1M and it's almost full. Hit ratio is a nice 0.9397.

    Thanks!

  • NodeBB

    Make sure to check your memory usage on the other page. Not sure how much it will increase it with a million entries.

  • NodeBB

    @gotwf yeah it's meant for devs


  • @baris Thanks! Yes, memory still looks good.

Suggested Topics

| |