[nodebb-plugin-dbsearch] DB Search Plugin


  • @baris Hey buddy, I'm unable to re-index my data, there's a lot of it, and it basically brings the server to a standstill, nodebb crashes and nothing appears indexed after a restart. 500 error teaser doesn't exist.

    Stats:

    28,000 posts
    2000 topics from 490 users.

    While it's indexing I get a 504 Gateway Time-out. Is there a way to break up the indexing?

    Just had a thought, it says new content is indexed automatically, if I re-index before I import my data, would that work?

  • NodeBB

    Can you update the plugin and nodebb core and give it another try? I reindex 1.6k
    topics and 12.5k posts in 30 seconds. I added some logging and timing into reindex so you should see the progress now. Also changed the reindex to use eachLimit so it doesn't try to reindex everything at once.

    Let me know if it changes anything.


  • @baris Thanks, will give it a go and let you know. 👍

  • Global Moderator Plugin & Theme Dev

    @baris What versions have compatibility with 0.5.x?

  • NodeBB

    0.0.15 should be compatible with 0.5.x


  • Tried this out (is it the only search plugin?!) for v0.6.x and searches have an issue with broken templates I think?
    dbsearch.JPG
    You get no search results, and a broken page (as above)
    As well as a console error in JS of "Uncaught TypeError: Cannot read property 'split' of undefined" but that's probably more a symptom of the template render issue.

  • GNU/Linux Admin

    @Drew-Llewellyn we also support solr and elasticsearch 😄

    ... but for those you have to have some and elasticsearch installed, respectively.


  • @julian Is that instead of redis, or as well as?
    Also, any clue what's up with the search page? Or where I'd find the template file for that fragment to begin looking into it myself?


  • @Drew-Llewellyn said:

    Tried this out (is it the only search plugin?!) for v0.6.x and searches have an issue with broken templates I think?
    dbsearch.JPG
    You get no search results, and a broken page (as above)
    As well as a console error in JS of "Uncaught TypeError: Cannot read property 'split' of undefined" but that's probably more a symptom of the template render issue.

    This is my compiled search.tpl file, and I can't seem to find what the problem is:

    <ol class="breadcrumb">
    	<!-- BEGIN breadcrumbs -->
    	<li itemscope="itemscope" itemtype="http://data-vocabulary.org/Breadcrumb" <!-- IF @last -->class="active"<!-- ENDIF @last -->>
    		<!-- IF [email protected] --><a href="{breadcrumbs.url}" itemprop="url"><!-- ENDIF [email protected] -->
    			<span itemprop="title">
    				{breadcrumbs.text}
    				<!-- IF @last -->
    				<!-- IF !feeds:disableRSS -->
    				<!-- IF rssFeedUrl --><a target="_blank" href="{rssFeedUrl}"><i class="fa fa-rss-square"></i></a><!-- ENDIF rssFeedUrl --><!-- ENDIF !feeds:disableRSS -->
    				<!-- ENDIF @last -->
    			</span>
    		<!-- IF [email protected] --></a><!-- ENDIF [email protected] -->
    	</li>
    	<!-- END breadcrumbs -->
    </ol>
    
    <div class="search">
    	<div class="row">
    
    		<div class="well col-md-12">
    			<form id="advanced-search">
    				<div class="form-group">
    					<label>[[global:search]]</label>
    					<input type="text" class="form-control" id="search-input" placeholder="[[global:search]]">
    				</div>
    				<div class="form-group">
    					<label>[[search:in]]</label>
    					<select class="form-control">
    						<option value="posts">[[global:posts]]</option>
    						<option value="users">[[global:users]]</option>
    						<option value="tags">[[tags:tags]]</option>
    					</select>
    				</div>
    				<div class="form-group by-container<!-- IF hidePostedBy --> hide<!-- ENDIF hidePostedBy -->">
    					<label>[[search:by]]</label>
    					<input type="text" class="form-control" id="posted-by-input" placeholder="[[search:posted-by]]">
    				</div>
    				<button type="submit" class="btn btn-default">[[global:search]]</button>
    			</form>
    		</div>
    
    		<div id="results" class="col-md-12" data-search-query="{search_query}">
    			<!-- IF matchCount -->
    			<div class="alert alert-info">[[search:results_matching, {matchCount}, {search_query}, {time}]] </div>
    			<!-- ELSE -->
    			<div class="alert alert-warning">[[search:no-matches]]</div>
    			<!-- ENDIF matchCount -->
    
    			<!-- BEGIN posts -->
    			<div class="topic-row panel panel-default clearfix">
    				<div class="panel-body">
    
    					<a href="{relative_path}/topic/{posts.topic.slug}/{posts.index}" class="search-result-text">
    						<h4>{posts.topic.title}</h4>
    					</a>
    					<div class="search-result-text">
    						{posts.content}
    						<p class="fade-out"></p>
    					</div>
    
    					<small>
    						<span class="pull-right footer">
    							<a href="{relative_path}/users/{posts.user.userslug}"><img class="user-img" title="{posts.user.username}" src="{posts.user.picture}"/></a>
    							[[global:posted_in_ago, <a href="{relative_path}/category/{posts.category.slug}"><i class="fa {posts.category.icon}"></i> {posts.category.name}</a>, <span class="timeago" title="{posts.relativeTime}"></span>]]
    						</span>
    					</small>
    				</div>
    			</div>
    			<!-- END posts -->
    
    			<!-- IF users.length -->
    			<ul id="users-container" class="users-container">
    <!-- BEGIN users -->
    <li class="users-box registered-user" data-uid="{users.uid}">
    	<a href="{relative_path}/user/{users.userslug}"><img src="{users.picture}" class="img-thumbnail"/></a>
    	<br/>
    	<div class="user-info">
    		<span>
    			<i class="fa fa-circle status {users.status}" title="[[global:{users.status}]]"></i>
    			<a href="{relative_path}/user/{users.userslug}">{users.username}</a>
    		</span>
    		<br/>
    		<div title="reputation" class="reputation">
    			<i class='fa fa-star'></i>
    			<span class='formatted-number'>{users.reputation}</span>
    		</div>
    		<div title="post count" class="post-count">
    			<i class='fa fa-pencil'></i>
    			<span class='formatted-number'>{users.postcount}</span>
    		</div>
    	</div>
    </li>
    <!-- END users -->
    			</ul>
    			<!-- ENDIF users.length -->
    
    			<!-- IF tags.length -->
    <!-- BEGIN tags -->
    <h3 class="pull-left tag-container">
    	<a href="{relative_path}/tags/{tags.value}" data-value="{tags.value}"><span class="tag-item" data-tag="{tags.value}" style="<!-- IF tags.color -->color: {tags.color};<!-- ENDIF tags.color --><!-- IF tags.bgColor -->background-color: {tags.bgColor};<!-- ENDIF tags.bgColor -->">{tags.value}</span><span class="tag-topic-count">{tags.score}</span></a>
    </h3>
    <!-- END tags -->
    			<!-- ENDIF tags.length -->
    		</div>
    	</div>
    </div>
    

  • @Drew-Llewellyn What version of templates.js are you using? It's in your node_modules folder. Also make sure vanilla and lavender are up to date.


  • @a_5mith
    templates.js 0.1.9
    nodebb-theme-vanilla 0.2.30
    nodebb-theme-lavender 0.2.13


  • Fixed by an npm install in nodebb root folder. Must've got the wrong version of something somewhere.


  • @Drew-Llewellyn Templates.js likely out of date. Will have updated to 0.2.10 or so. 👍


  • installed the plugin today, but it think it mistakenly installed the older version and I guess it's not working, says 'No matches found'.

    Capture.JPG


  • Go to the Plugin control panel (/admin/plugins/dbsearch) and tap on "Re Index".

    Bildschirmfoto 2015-04-17 um 09.54.43.png


  • @Hägar

    Thank you very much, it worked 👍


  • I have several comments/requests:

    If any of these are doable? I would like to carry them to GitHub...

    1. In Turkish, we have some letters with accents such as ı,ç,ö,ş,ü,ğ... however since our English keyboard does not contain these letters, it is commonly used with english letters, and for a normal Turkish person the word "cesme" would be easily understood as "çeşme" in a sentence; however dbsearch is making difference with these two words, and therefore we -most of the time- cannot find what we are searching for on the form.

    So, I wonder if it would be possible for this letter to be recognized as their counterpart without accent? Such as

    ı can be also i
    ç can be also c
    ö can be also o
    ş can be also s
    ü can be also u
    ğ can be also g

    It would help us a lot.

    2. When you search a word on the forum it is quite important to write the word correct, however sometimes people make mistakes in their comments/posts/titles and it is not edited later. So for example, when you search for "importan" or "importat" it is never showing you the word "important" or vice versa. I believe a wobbly search would be quite nice, so maybe a long word with one letter mistake is tolerated; so people can find what they are looking for.

    3. Would it be possible to implement a hashtag system to the forum? Unlike most forums, our forum is based on accumulating experiences and knowledge under related title, so creating new topic for something already in the forum is discouraged. This of course causes very long topics, however since tag system only works for the first post and we cannot tag other keywords in the following posts, it is getting harder to find other related stuff (especially #1 issue causes problems with our dbsearch). It would be cool to have a hashtag in or below the post so for example, if people are giving information about "high school" in a "New York" topic, they can tag their post with #highschool and later this can be found very easily with dbsearch or just by clicking the hashtag. I think people also like using hashtags because of social media.

  • Global Moderator Plugin & Theme Dev

    @crazycells this plugin is called "db search" because all it does it use the database's internal search functionality. That's why your first two requests are not possible to implement in this plugin.

    Unfortunately other search plugins are difficult to set up since they require entirely new programs on your server.

    Your third request is just entirely off topic, nothing really to do with the plugin. Personally I think the tagging system deserves a lot of help. I think removing the actual tag fields and instead just parsing post content could be a better system.


  • @PitaJ said in [nodebb-plugin-dbsearch] DB Search Plugin:

    @crazycells this plugin is called "db search" because all it does it use the database's internal search functionality. That's why your first two requests are not possible to implement in this plugin.

    Unfortunately other search plugins are difficult to set up since they require entirely new programs on your server.

    Your third request is just entirely off topic, nothing really to do with the plugin. Personally I think the tagging system deserves a lot of help. I think removing the actual tag fields and instead just parsing post content could be a better system.

    I see, but people are using this to find relevant info on the forum. Right now, google search in the website gives more useful results than the search function of the forum. I think it should be other way around. I would try to do brainstorming if I knew how to do it but unfortunately I am not competent to make plugins. 🙂


  • Additionally, I have forgot to add, so this is #4... A lot of people in our forum are complaining that there is no "search in the topic" function.

    In other old style forums, there is usually a function at the top or below first post where you can only search that topic, however it is missing in NodeBB.

    I think this definitely should be added to NodeBB.

Suggested Topics

| |