[nodebb-plugin-solr] Solr search plugin

[mani]

@a_5mith I think it's because I didn't install Solr, so I'll install and try again.

[trevor]

Okay apparently it WASN'T updated to latest, lol. The following changes were made that fixed it; https://github.com/NodeBB/NodeBB/commit/7212f95582eb1e828ad6845a1b7f7f4dbb8b9e39

Thanks @main

0 barisusakli committed to NodeBB/NodeBB

small fix to routes

[mani]

@trevor Thank you too, I just installed Solr by your instruction and make everything works!

[trevor]

Note: Installing and running Solr takes up a good bit of your CPU so if you're using DigitalOcean, I suggest you use the 10$/mo plan instead of the $5/mo.

[Guiri]

How does this plugin compare to the DB Search as far as features and accuracy?

[Haowei]

It works fine on our http://nodebb.org.cn

[julian]

Hi @Guiri, good question!

This plugin is closer to a traditional full-text search plugin you'd see in use on other sites. It's more powerful than a simple fulltext search, because we can do thing like word exclusions (e.g. forum -NodeBB), boosts (e.g. forum +punBB), and can even sort relevancy better by ranking documents higher if the searched terms are closer together, or are sequential (e.g. A search for Farmer John would rank This is a topic about Farmer John higher than John is a Farmer)

The existing search plugin uses phonetics to determine matches, which would cause problems when things had the same phonetic sounds, but were completely different words. (e.g. forum and farm are very different, but have the same consonants).

[julian]

Some development notes re: Solr

The default configuration saves topic titles in title_t and post content in description_t. These are defined in the default schema.xml shipped with Solr as:

<dynamicField name="*_t"  type="text_general"    indexed="true"  stored="true"/>

Setting stored to false would decrease the index size (and increase search efficiency by a small amount), as the actual post content would not be saved on-disk or in-memory.

Before, the index for this forum was about 7.5 MB. By setting stored to false, the index size is now 2.6 MB.

Would look into further customisation of the plugin (so you can define your own fields in schema.xml if you'd like) for the next version...

[Guiri]

Awesome, thanks for the quick reply. Is there a min-version, (e.g., nodebb 0.5.0) that's required or can I install it on my 0.4.3? I didn't see a min-version of nodebb specified in https://github.com/julianlam/nodebb-plugin-solr/blob/master/package.json

[julian]

@Guiri The Solr plugin relies on changes present in the latest versions of NodeBB. If you are up to date on the master branch, you should be fine. If you are using the v0.4.x branch, it is not compatible.

[Guiri]

@julian Given my history of attempting to install updates and breaking everything, I'll wait until the 0.5.0 with the Idiots Guide. Please tailor it especially to my level of bad luck.

[julian]

@Guiri Good call -- the latest version only works with NodeBB commits from one day ago

[scottalanmiller]

Having issues with Solr on 0.6.0, even after the latest update. Is this a known issue or just me?

[julian]

Hey @scottalanmiller, there should be no problems with this plugin -- let me know the error?

Give v1.0.0 a try, though there are changed with the schema, so you'll want to flush/reindex your data.

[julian]

v1.0.0 Released

• Schema has been updated, so please flush and re-index your search index
• Progress bar now shows up in ACP, so you won't have to guess when the indexing is completed

[shady2k]

Why this plugin show only topic search data?
I see in Solr log, that it search for description_t and title_t. description_t returns 37 results, title_t - 0.
But user see nothing.

2447575 [qtp507584865-14] INFO org.apache.solr.core.SolrCore – [collection1] webapp=/solr path=/select params={start=0&q=дизайн&qf=description_t^1&wt=json&rows=500&defType=dismax} hits=37 status=0 QTime=1
2447575 [qtp507584865-16] INFO org.apache.solr.core.SolrCore – [collection1] webapp=/solr path=/select params={start=0&q=дизайн&qf=title_t^1&wt=json&rows=500&defType=dismax} hits=0 status=0 QTime=1

nodebb log:
9/2 12:42 [30014] - verbose: [plugin/solr] Search (topic) for "дизайн" returned 0 results
9/2 12:42 [30014] - verbose: [plugin/solr] Search (post) for "дизайн" returned 38 results

https://github.com/julianlam/nodebb-plugin-solr/issues/20

shady2k created this issue in julianlam/nodebb-plugin-solr

closed No search data for description_t shown #20

[scottalanmiller]

Updated to the latest Solr this morning and it returned zero results. We moved back to DBSearch and it worked fine, although the Regex in DBSearch is not nearly as nice as Solr. At least we have search again. Can't do any more testing with Solr until next week, so at the moment we are at a standstill there. But as DBSearch is working, it is not critical.

[julian]

Have you tried flushing and re-indexing?

[scottalanmiller]

A little bit ago we uninstalled it and completely reinstalled and it started working. So Solr is working for us again (as are emoji.)

[julian]

Hoorah! Glad to hear it