Unsolved Is Google complaining about nodebb or something else?


  • Google is constantly complaining about our nodebb forums and I'm at a loss trying to fix this. I've looked online and for months have been trying to find the cause and simply cannot.

    This is a snippet of some of the complaints but I don't see anything causing blocking on the site. I've been over this countless times.

    7e17a1f6-ab1c-4842-b2b0-84fafa810e9d-image.png

    I've tried different settings in /admin/settings/web-crawler, adding, removing .xml files, nothing seems to work.

    There is no robots.txt file anywhere on the server itself so nothing is overriding the settings in nodebb.

    Looking at the Google information, where it shows pages that were not found, 404, I click on those links and I can get to most of those pages.

    However, all of them (shown below for example) lead to the top page and not individual pages that the URL's are showing.

    viewtopic.php?f=13&t=133
    user/seo-crawler/followers
    viewtopic.php?p=193
    user/sergrid/posts
    viewtopic.php?p=70
    user/ibm-research-bot/groups
    viewtopic.php?p=80
    viewtopic.php?p=208
    user/toliklus/topics?sort=lastpost
    groups/registered_coppa

    Is this a lead perhaps?

  • Community Rep

    I, and many others, feel your pain. I boycott evil these days the best I am able. My two bit opinion, ymmv.

    Matomo is one possible option Google Analytics alternative that protects your data and your customers' privacy

    Works well. Maybe non trivial to set up for non systems admin types but there are SaaS options as well. Iirc, they also offer free trials so you can test drive w/o incurring monetary costs.

    Does not solve your specific problem but at least one alternative to avoid having the problem in the first place. Managing self hosted Matomo may be an even larger "problem" for you depending on background and available resources. It was not for me and I've never looked back.


  • @gotwf I second this. I've been using Matomo since inception, and moved away from GA. If you need any assistance setting up Matomo @NodeHam, I'm happy to assist with that.


  • We already use Matomo for all our sites but it's not showing any problems yet Google does.
    In fact, I can search for a specific title that should have been indexed from the forums and never find it on Google.
    This is a problem I've seen since day one of using nodebb and I have an old post that didn't lead to much back then but to keep seeing this is very frustrating considering how hard we all try to get our forums found/seen. Not sure it's a nodebb 'problem' but what ever it is, I'd sure like to solve it so we can get properly indexed.

    BTW, I fully agree about boycotting the big tech companies. They really do control the Internet and keep us constantly having to work to have any footprint on the net.

    Anyhow, I have to believe there must be some good SEO folks in these forums. Hopefully, one of them will see these posts and try to lend a hand.


  • @NodeHam what output do you get if you go to google, and type site:yourforumdomain.com and search? I'd expect results to be returned, but they may not rank as high as you think.


  • Good call.

    Yes, some things do show up when searching using site.
    However, if I change the time frame to one month for example, not even one page shows up on Google.
    I have to change the range to one year to get anything and that's only two pages.
    We're definitely not getting indexed which is an absolute shame.


  • Another odd thing is that the very first result is Google suggesting to use Google search console.
    When clicking on that, it shows stats for our main site, not the support forums which does have its own asset in Google search console.


  • @NodeHam Yes, I've seen this myself with another forum. Sadly, with GA, you have to follow their recommendations in order to fix. Have you evaluated your site against an SEO testing tool ?


  • Yes, we also use SemRush and others and all looks fine there. We used to spend a great deal of time on SEO but found it was a never ending game with Google and could not justify spending so much time at it. Now we just do the best we can but getting indexed is one thing we very much count on.

    This is just frustrating and weird. Really don't know how to fix it.

    image.png

    Those 1078 pages it claims to have indexed are super old and not all very relevant or posts.


  • @NodeHam said in Google constantly complains about nodebb forums:

    We used to spend a great deal of time on SEO but found it was a never ending game with Google and could not justify spending so much time at it.

    And that one statement speaks a multitude. That's exactly what it is, and it's very easy to focus all of your effort on that rather than creating compelling content. I turned away from SEO years ago, but do post regularly on my own forum (https://sudonix.com) and it's ranking has improved over time - all organic, and no "tricks".


  • Their recommendations don't make any sense and are usually very circular. I've tried going through each error one at a time and most end up being dead ends or having to wait days to get re-indexed only to see the same problem. To me, this is something related with nodebb since I can see that other sites of ours are indexed as they should be but the forums aren't.


  • @NodeHam Can you list the plugins you are using ?


  • @phenomlab said in Google constantly complains about nodebb forums:

    @NodeHam said in Google constantly complains about nodebb forums:

    We used to spend a great deal of time on SEO but found it was a never ending game with Google and could not justify spending so much time at it.

    And that one statement speaks a multitude. That's exactly what it is, and it's very easy to focus all of your effort on that rather than creating compelling content. I turned away from SEO years ago, but do post regularly on my own forum (https://sudonix.com) and it's ranking has improved over time - all organic, and no "tricks".

    Yes but compelling content that cannot be found is exactly where we are now :). We decided to do just that, spend more time on relevant and good content but if we're not being indexed, no one is finding that great content anyhow.


  • @NodeHam I know the feeling. It's hard to find a niche that hasn't already been "done to death".


  • The niche is not the problem, it's being found on the Internet that is. I suggested nodebb because it seemed to have a better shot at being indexed than the previous forums that were being used but now I find this.

    Here are the plugins on the site. Pretty basic.

    nodebb-plugin-2factor - Installed 5.0.2 | Latest 5.0.2. Not active
    nodebb-plugin-composer-default - Installed 9.1.1 | Latest 9.1.1. Active
    nodebb-plugin-cppnet-markdown - Installed 0.7.0 | Latest 0.7.0. Not active
    nodebb-plugin-dbsearch - Installed 5.1.5 | Latest 5.1.5. Active
    nodebb-plugin-emoji - Installed 4.0.4 | Latest 4.0.4. Active
    nodebb-plugin-emoji-android - Installed 3.0.0 | Latest 3.0.0. Active
    nodebb-plugin-markdown - Installed 10.1.0 | Latest 10.1.0. Active
    nodebb-plugin-mentions - Installed 3.0.11 | Latest 3.0.11. Active
    nodebb-plugin-soundpack-default - Installed 2.0.1 | Latest 2.0.1. Not active
    nodebb-plugin-spam-be-gone - Installed 1.0.0 | Latest 1.0.0. Not active
    nodebb-rewards-essentials - Installed 0.2.1 | Latest 0.2.1. Active

    nodebb-theme-lavender - Installed 6.0.0 | Latest 6.0.0.
    nodebb-theme-persona - Installed 12.1.1 | Latest 12.1.1
    nodebb-theme-slick - Installed 2.0.2 | Latest 2.0.2
    nodebb-theme-vanilla - Installed 12.1.18 | Latest 12.1.18
    nodebb-widget-essentials - Installed 6.0.0 | Latest 6.0.0


  • @NodeHam Yes, nothing of any concern there. I also use the Q&A plugin, and I know that creates better SEO as mine increased after I installed it - I have a use case for this because my site is a consultancy where members can ask questions and get assistance around technology.


  • Robots in admin/settings/web-crawler is set as follows. At least this is the most current testing since finding this problem.

    User-Agent: *
    Sitemap: https://xxx.com/sitemap/topics.xml
    Sitemap: https://xxx.com/sitemap/topics.1.xml
    Sitemap: https://xxx.com/sitemap/categories.xml
    Sitemap: https://xxx.com/sitemap/pages.xml


  • @phenomlab, thanks very much. I'll install that plugin for sure if it helps even a little bit 🙂


  • I tried that plugin but it just breaks the editing tools. The site is fully updated, up to date, no other issues that I know of.
    Guess I can't use it.


  • @NodeHam Odd - seems fine for me


Suggested Topics

| | | |