[nodebb-plugin-dbsearch] DB Search Plugin

Scuzz

@julian I'm on MongoDB, thought it might be different.

julian

I thought it would be, actually, since Mongo uses fulltext search. However, keep in mind you have to explicitly enable it per mongo install, it is not enabled by default:

(github.com)

^ Specifically, step 7.

Scuzz

Yeah I already have that.
I thought it would have been some sanitation within the db plugin that stopped searching for symbols.
Ok, with more testing it only happens with # and ?
Could this be something to do with URLs?

julian

@scuzz I think you may be right about that one.....

search term needs URL escaping · Issue #1195 · NodeBB/NodeBB

Terms including # and ? end up messing up the URL, and search fails.

GitHub (github.com)

theon

@julian Hi! I was interested to know more details about why Redis is not compatible with non-latin characters. I though Redis just takes arbitrary bytes as values and you store text encoded however you like (UTF-8 for example).

theon

Ah, I think I understand now. It doesn't support searching for non-latin chars, but it does support storing them?

julian

@theon Redis does, yes, but by itself, the server does not come with fulltext search capability. To do that, we use a package called Reds, which catalogues and indexes information by phonetic sounds, and only covers the latin charset.

theon

@julian Makes sense. It is a key/value store after all and not search index. I think I would prefer a Redis/ElasticSearch combo over MongoDB for my use case. Are your plans to still support Redis alongside MongoDB?

julian

@theon So far, yes, I don't see us dropping support for Redis, since NodeBB is already highly optimised for it.

Definitely would advocate for a third party solution like elasticache in conjunction with NodeBB

Nicekiwi

Could we get search enabled for guest users?

finid

@Ezra-Sharp

In the ACP, there's an option in the General Settings > General page that says Allow guests to search without logging in. Or is that different from what you are asking?

julian

Thanks @planner That seems to be what he's asking. Glad we have you around!

finid

@julian

No, no no, I'm not trying to apply for a Community Manager's job, but, but, but...

Nicekiwi

@planner that is what I ask asking,

But was also suggesting it be enabled on this website.

Leonardo Galani

Hi.. im having some "issues" with searching..

some content isnt showing on the search result.. .. like.. i got a thread with a word that repeats on the content.. but i look for that word.. and i dont get that thread as result...

any thoughts?

agiletesters .com .br -> search for 'cucumber'

julian

@Leonardo-Galani Does it work if you go to the plugin admin page and click "ReIndex"?

Giorgio Chiodi

@baris

When do I have to reindex? After every install / udpate?

<baris>

You don't have to reindex after upgrades or new installs.

a_5mith

@baris Hey buddy, I'm unable to re-index my data, there's a lot of it, and it basically brings the server to a standstill, nodebb crashes and nothing appears indexed after a restart. 500 error teaser doesn't exist.

Stats:

28,000 posts
2000 topics from 490 users.

While it's indexing I get a 504 Gateway Time-out. Is there a way to break up the indexing?

Just had a thought, it says new content is indexed automatically, if I re-index before I import my data, would that work?

<baris>

Can you update the plugin and nodebb core and give it another try? I reindex 1.6k
topics and 12.5k posts in 30 seconds. I added some logging and timing into reindex so you should see the progress now. Also changed the reindex to use eachLimit so it doesn't try to reindex everything at once.

Let me know if it changes anything.