Search unreliable



  • The search function doesn't work reliably.

    I've got two Threads with the (same) subject Semmelknödel a la Tom. If I search for knödel both will be found, however, if I search for Semmel only one of them will be shown.

    NodeBB version is 1.8.1.



  • Ah, there's a pattern, it doesn't find older posts!

    Nope, it finds the old, but not the new one. But there's an older posting "jobangebot" and if I search for "job", it wont be found.

    😞

    Also, subjects are not being searched at all. It seems, only post bodies are included - but not all, see above.

    So, here are the two threads:

    0_1522865617176_semmel.png

    Search for Semmel in subject and body returns 1 result:

    0_1522865659174_semmel1result.png

    When searching only the subject, nothing will be found:

    0_1522865685471_semmel0result.png


  • Admin

    Hm... have you tried re-indexing your forum? That might fix the missing search key...



  • I did but it didn't help.


  • Admin

    Funny, searching knödel here returns 1 post, but Semmel returns two! 😆

    @baris



  • wrong thread, sorry.



  • Is there any solution to this? Because if there's no solution, I must migrate the forum to some other software, as a reliable search function is key to my users application: they log recipes to the forum and use search to check for duplicates. But since the search doesn't work, they actually produce duplicates instead of avoiding them!


  • Global Moderator

    You can redirect the search function to Google, but it won't work for non-public posts. There is also an elasticsearch plugin you might try.


  • Admin

    Are you on mongodb or redis?



  • @PitaJ thanks for your reply, but it's not what I'm asking for 😞 I can't use google, of course, and I won't add another plugin which would just add to the already annoying load problem I'm experiencing.

    So, I think this is clearly a bug. Whatever the dbsearch plugin is trying to do- it is doing it completely wrong. In fact, I think the index it creates on MongoDB is wrong, since I can reproduce the invalid search results from mongo shell as well:

    > db.searchtopic.find( { $text: { $search: "Semmel" }})
    [no output]
    
    > > db.searchpost.find( { $text: { $search: "Semmel" }})
    > { "_id" : ObjectId("5a9d818cd19a243c1a362c28"), "id" : 1201, [..]
    

    So, only searching topics returns no result and posts returns 1 result. However, in reality there are 3 topics matching the string Semmel:

    > db.objects.find( { _key: "topic:178" });
    { "_id" : ObjectId("5a9d818cd19a243c1a362c15"), "_key" : "topic:178", "tid" : 178, "uid" : 3, "cid" : 24, "mainPid" : 1201, "title" : "Semmelknödel ala Tom", "slug" : "178/semmelknödel-ala-tom", "timestamp" : 1520271756137, "lastposttime" : 1520277378816, "postcount" : 7, "viewcount" : 12, "locked" : 0, "deleted" : 0, "pinned" : 0, "upvotes" : 2, "downvotes" : 0, "teaserPid" : "1212" }
    
    > db.objects.find( { _key: "topic:362" });
    { "_id" : ObjectId("5ac49e3cd19a243c1a36e9f3"), "_key" : "topic:362", "tid" : 362, "uid" : 6, "cid" : 29, "mainPid" : 2826, "title" : "Semmelknödelteig", "slug" : "362/semmelknödelteig", "timestamp" : 1522835004299, "lastposttime" : 1522847285369, "postcount" : 4, "viewcount" : 10, "locked" : 0, "deleted" : 0, "pinned" : 0, "teaserPid" : "2840" }
    
    > db.objects.find( { _key: "topic:367" });
    { "_id" : ObjectId("5ac4f7ced19a243c1a36ed50"), "_key" : "topic:367", "tid" : 367, "uid" : 6, "cid" : 23, "mainPid" : "2852", "title" : "Semmelknödel a la Tom...", "slug" : "367/semmelknödel-a-la-tom", "timestamp" : 1522857934277, "lastposttime" : 1522866479077, "postcount" : 20, "viewcount" : 12, "locked" : 0, "deleted" : 0, "pinned" : 0, "teaserPid" : "2907", "thumb" : "", "upvotes" : 2, "downvotes" : 0 }
    

    (and many more posts)

    What is weird as well is the index language setting: "default_language" : "english", but I'm using german as forum language.


  • Admin

    @tlinden what does the following query return? db.searchtopic.find({id: 178});



  • I'm using chinese as forum language and I'm experiencing the same problem.



  • @baris said in Search unreliable:

    @tlinden what does the following query return? db.searchtopic.find({id: 178});

    @baris this::

    > db.searchtopic.find({id: 178})
    { "_id" : ObjectId("5a9d818cd19a243c1a362c1c"), "id" : 178, "content" : "Semmelknödel ala Tom", "cid" : "24", "uid" : "3" }
    

  • Admin

    @tlinden ok so it is in the searchtopic collection, does searching for it like this find it?

    db.searchtopic.find( { $text: { $search: "Semmelknödel" }}

    I don't think full text search supports partial word matches right now. https://stackoverflow.com/questions/44833817/mongodb-full-and-partial-text-search?rq=1

    I will hopefully add language support to dbsearch plugin if mongodb is used https://github.com/barisusakli/nodebb-plugin-dbsearch/issues/33



  • Ah, that explains it! Searching for Semmelknödel reveals 1 topic and 2 posts (which is correct). So, we have to seach for whole words until partial matches are supported sometime.

    Ok, still a little annoying for the girls, but I'm going to blame you 🙂

    However, thanks for clarifying @baris !

    best,
    Tom


  • Admin

    @tlinden Yeah I agree it's a bit annoying, there is an issue for it here https://jira.mongodb.org/browse/SERVER-15090 so it might be supported in the future.


  • Admin

    nodebb-plugin-dbsearch@2.0.14 adds support for different languages. You can switch to German and see if it improves your results.



  • You can also try Solr search for NodeBB.

    First install solr-jetty (jetty is lighter and has better performance than tomcat):

    sudo apt-get install solr-jetty
    

    (This also automatically installs the many needed dependencies, basically Java packages).

    Optional: Temporarily open port 8080 if you want to see Solr admin (not needed):

    http://example.com:8080/solr/

    (Port 8080 worked for me; if not, try 8983).

    Then install NodeBB's Solr plugin:

    NodeBB Admin > Extend > Plugins > Find Plugins > nodebb-plugin-solr > Install

    Then activate the plugin, restart NodeBB, reload admin page, and go to:

    Plugins > Apache Solr

    Configuration:

    Host: localhost
    Port: 8080
    Path: /solr

    Save Settings

    Reload NodeBB and admin page.

    Connected

    Indexing Enabled

    Rebuild Search Index

    Disable port 8080 (or 8983) on firewall for security.

    It works well. 👍



  • Thanks for the tutorial, but I won't do that, it's like we say in Germany: "mit Kanonen auf Spatzen schiessen" (shooting with canons on sparrows). Also, I can't install software, because I'm on a shared system w/o root.

    best,
    Tom


 

| |

Looks like your connection to NodeBB was lost, please wait while we try to reconnect.