In core now. 👍
Thanks @baris for helping.
Currently I see there are only 63 latest urls that appear in /sitemap.xml file
How can i increase this number?
You know, this is a good question.
NodeBB used to display ALL topics on sitemap.xml, except we were finding that it would take many seconds to load, effectively locking the Node process (if you only had 1 process).
We restricted it to load recent topics only, although I can imagine that this isn't good for SEO.
Does anybody know of a standard for paginated sitemaps?
https://github.com/NodeBB/NodeBB/blob/master/src/sitemap.js#L52 you can increase the value 49.
Does it take several seconds to build it? Maybe caching the sitemap in memory would help? Or simply write all new topics to a static file and rebuild it at startup.
The sitemap is cached, although the process to build it takes a couple seconds (if you have many thousand topics).
Might not be too bad if we optimized it a bit...
Optimize sitemap? Inb4: https://github.com/NodeBB/NodeBB/commit/006322f3863f5dabbc9b4051cac6b0f466e37916 @baris
Guys thank you so much for your reply.
Since I mostly look into Online Marketing Stuffs, that's why asked this question.
From our team @vatsal is taking care of tech., he would be happy to give feedback.
Yes, according to me maintaining a list of sitemap record is quite important for search engines.
Have a look at sitemap samples from here: http://www.amazon.com/robots.txt ; at the end of file sitemap location is given.
For large websites and forums .xml.gz format can be followed.
There's some info here about sitemap indexes, which we might put into action: http://www.sitemaps.org/protocol.html#index
Yep, this is quite important.
Did some test googling... and most of our forum thread content has fallen out of the google index. There are a wealth of information in those threads with lots of technical mumbo jumbo keywords which would drive more sensor-nerds to us if they were searchable.
OT: NodeBB gets appreciation from the members: http://forum.mysensors.org/topic/472/push-notifications
Thanks for all the effort you put into this!
Good workaround @ankitpr89. We'll try to change up the sitemap, let me issue it now
@julian Thank you so much Julian!
Hi guys, sorry to bump such an old thread but couldn't find anything more relevant.
Has there been any change in the way NodeBB generates sitemaps since back when this was originally posted? We are not seeing a full list of URLs in our sitemap either (using Lavender theme if it makes any difference) and a lot of should-be-popular threads aren't being indexed as a result.
Yes, the current sitemap should list all publicly available topics/categories.
hey @julian thanks for your quick reply. I checked the sitemap for our forum, but it shown only the categories & topics headers. Is this how it's meant to be shown or am I missing something?
@Vamsi-Krishna what you see there are links to other sitemap files. They are essentially imported when the main sitemap is parsed by Google et al.