[nodebb-plugin-rss] RSS Feed plugin

<baris>

@Bryant-Young published a new version can you try that one? v0.1.9.

Bryant Young

@baris Alright, updated to 0.1.9, restarted nodebb, I'll report back when the next interval goes through and pulls in new comments if there's further issues. On a sidenote does it show plugin version anywhere?

Bryant Young

@baris Just got some new content pulled in - worked great!

Now I just needa figure out how to do bulk deletions of old content! Thanks again for fixing that!

a_5mith

@Bryant-Young Click the first topic, then hold shift and click the last one, and it will tick all the ones in between a la Google Mail.

psychobunny

@a_5mith said:

@Bryant-Young Click the first topic, then hold shift and click the last one, and it will tick all the ones in between a la Google Mail.

Haha wow, you're a power user. So much so that even I didn't know we could do that. I guess kudos to @baris for building that in

<baris>

Updated with tags input and topic timestamp. v0.1.11

Silvina

Was playing around with this plugin, its working great with posts and all, yet i felt something was missing.

-URL Source (URL of the Original Content)

Title Prefix (RSS ?)

Tanner

An optional URL source at the end of the post would be nice.

Silvina

hi @baris

Recently i tried pulling this RSS feed, and it is failing, any idea why?

feed url: http://www.rssitfor.me/getrss?name=%40AmazonAppADay

running via nodebb dev,

the error shown is: error:
[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null

omar5687

Anyway to get the feed to post as read as to avoid spam in the unread tab?

julian

@Silvina Can you provide Baris with the full stack trace from that error?

Silvina

@julian I can probably do that. where can i find the full stack trace?
what ever that is shown on the ./nodebb dev console window?

julian

@Silvina said:

what ever that is shown on the ./nodebb dev console window?

Yep, that's it

Silvina

@Silvina said:

Rssitfor.me

(www.rssitfor.me)

[www-data@vultr forums-gameredge]$ ./nodebb dev
Launching NodeBB in "development" mode.
To run the production build of NodeBB, please use "forever".
More Information: https://docs.nodebb.org/en/latest/running/index.html
info: NodeBB v0.5.0 Copyright (C) 2013-2014 NodeBB Inc.
info: This program comes with ABSOLUTELY NO WARRANTY.
info: This is free software, and you are welcome to redistribute it under certain conditions.
info:
info: Time: Wed Sep 24 2014 23:20:12 GMT+0800 (SGT)
info: Initializing NodeBB v0.5.0
info: * using configuration stored in: /srv/forums-gameredge/config.json
info: * using redis store at 127.0.0.1:6379
info: * using themes stored in: /srv/forums-gameredge/node_modules
info: Base Configuration OK.
warn: You have no redis password setup!
morgan deprecated morgan(options): use morgan("default", options) instead src/logger.js:128:22
morgan deprecated default format: use combined format src/logger.js:128:22
info: [notifications.init] Registering jobs.
info: [user.startJobs] Registering User Jobs
info: Enabling 'trust proxy'
info: [plugins] Initializing plugins system
info: [themes] Using theme nodebb-theme-lavender
info: Middlewares loaded.
info: [plugins] Found 1 CSS file(s) for plugin nodebb-plugin-youtube

warn: [plugins/nodebb-plugin-mentions] This plugin may not be compatible with your version of NodeBB. This may cause unintended behaviour or crashing.
warn: [plugins/nodebb-plugin-mentions] In the event of an unresponsive NodeBB caused by this plugin, run ./nodebb reset plugin="nodebb-plugin-mentions".

info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-mentions
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-mentions
warn: [plugins/nodebb-plugin-spoilers] staticDir is deprecated, use staticDirs instead
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-spoilers
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-spoilers
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-spam-be-gone
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-spam-be-gone
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-linkcheck
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-linkcheck
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-twitter
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-emoji-extended
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-emoji-extended
info: [plugins] Found 1 js file(s) for plugin nodebb-theme-blacknred
info: [plugins] Found 1 CSS file(s) for plugin nodebb-widget-essentials
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-markdown
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-markdown
warn: [plugins/nodebb-plugin-featured-threads] staticDir is deprecated, use staticDirs instead
info: [plugins] Found 1 CSS file(s) for plugin nodebb-plugin-featured-threads
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-desktop-notifications
info: [plugins] Found 3 js file(s) for plugin nodebb-theme-lavender
info: [plugins] Loaded plugin: nodebb-plugin-rss
info: [plugins] Loaded plugin: nodebb-plugin-soundcloud
info: [plugins] Loaded plugin: nodebb-plugin-twitchtv
info: [plugins] Loaded plugin: nodebb-plugin-emailer-mailgun
info: [plugins] Loaded plugin: nodebb-plugin-youtube
info: [plugins] Loaded plugin: nodebb-plugin-mentions
info: [plugins] Loaded plugin: nodebb-plugin-spoilers
info: [plugins] Loaded plugin: nodebb-plugin-spam-be-gone
info: [plugins] Loaded plugin: nodebb-plugin-linkcheck
info: [plugins] Loaded plugin: nodebb-plugin-twitter
info: [plugins] Loaded plugin: nodebb-plugin-soundpack-default
info: [plugins] Loaded plugin: nodebb-plugin-emoji-extended
info: [plugins] Loaded plugin: nodebb-theme-blacknred
info: [plugins] Loaded plugin: nodebb-widget-essentials
info: [plugins] Loaded plugin: nodebb-plugin-imgur
info: [plugins] Loaded plugin: nodebb-plugin-markdown
info: [plugins] Loaded plugin: nodebb-plugin-slack
info: [plugins] Loaded plugin: nodebb-plugin-featured-threads
info: [plugins] Loaded plugin: nodebb-plugin-desktop-notifications
info: [plugins] Loaded plugin: nodebb-plugin-dbsearch
info: [plugins] Loaded plugin: nodebb-theme-lavender
info: [plugins] Sorting hooks to fire in priority sequence
info: [plugins] Plugins OK
info: [meta/css] Minifying LESS/CSS
warn: [plugins] The action:app.load hook is deprecated in favour of static:app.load, please notify the developers of the following plugins:

nodebb-plugin-featured-threads
warn: [meta/css] File not found! nodebb-plugin-featured-threads/css/main.css
info: [plugins/spam-be-gone] Settings loaded
info: [sounds] Sounds OK
info: [meta/css] Done.
info: [themes] Compiling templates
info: [themes] Successfully compiled templates.
info: NodeBB attempting to listen on: 0.0.0.0:4567
info: NodeBB is now listening on: 0.0.0.0:4567
info: [meta/js] Successfully minified.
info: [meta/js] Retrieved Mapping.
info: [meta/js] Compilation complete
info: NodeBB Ready
info: [socket] uid 1 (1JqnG1PdkeqC-cOkP5ac) has successfully reconnected.
warn: Route requested but not found: /css/assets/vendor/masonry.js
warn: Route requested but not found: /css/assets/vendor/imagesLoaded.js
warn: Route requested but not found: /css/assets/vendor/masonry.js
warn: Route requested but not found: /css/assets/vendor/imagesLoaded.js
warn: Route requested but not found: /css/assets/vendor/imagesLoaded.js
warn: Route requested but not found: /css/assets/vendor/masonry.js
error: [[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null

Silvina

[www-data@vultr forums-gameredge]$ ./nodebb log
at /srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:167:9
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at async.waterfall.content (/srv/forums-gameredge/src/topics/create.js:117:1 3)
at fn (/srv/forums-gameredge/node_modules/async/lib/async.js:641:34)
at Object._onImmediate (/srv/forums-gameredge/node_modules/async/lib/async.j s:557:34)
at processImmediate [as _immediateCallback] (timers.js:345:15)
info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
info: [app] Database connection closed.
info: [app] Web server closed to connections.
info: [app] Shutdown complete.

After upgrading to nodebb 0.5.1, i noticed that my nodebb instance been dying at random times.
Is this plugin causing the issue?

<baris>

Can you post the full log, should be at logs/output.log.

Silvina

Hi @baris
Running via ./nodebb dev

info: [notifications.prune] Removing expired notifications from the database.
warn: Route requested but not found: /forum/index.php?topic=256.90
error: [[error:too-many-posts, 10]]
error: TypeError: Cannot read property 'topicData' of undefined
at setTimestampToFeedPublishedDate (/srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:198:23)
at /srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:167:9
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at /srv/forums-gameredge/src/user.js:190:12
at /srv/forums-gameredge/node_modules/async/lib/async.js:592:17
at done (/srv/forums-gameredge/node_modules/async/lib/async.js:135:19)
at /srv/forums-gameredge/node_modules/async/lib/async.js:32:16
at /srv/forums-gameredge/node_modules/async/lib/async.js:589:21
at /srv/forums-gameredge/src/database/redis/sets.js:44:4
[cluster] Child Process (9827) has exited (code: 1)
[cluster] Spinning up another process...

From Output log

[32minfo[39m: Time: Wed Oct 15 2014 00:28:06 GMT+0000 (UTC)
[32minfo[39m: Initializing NodeBB v0.5.1
[32minfo[39m: * using configuration stored in: /srv/forums-gameredge/config.json
[32minfo[39m: * using redis store at 127.0.0.1:6379
[32minfo[39m: * using themes stored in: /srv/forums-gameredge/node_modules
[32minfo[39m: [user.startJobs] Registering User Jobs
[32minfo[39m: Enabling 'trust proxy'
[cluster] Child Process (8950) listening for connections.
[33mwarn[39m: [plugins/nodebb-plugin-spoilers] staticDir is deprecated, use staticDirs instead
[32minfo[39m: [hotswap] Router with id plugins replaced successfully
[32minfo[39m: [plugins] All plugins reloaded and rerouted
[32minfo[39m: [meta/css] Minifying LESS/CSS
[32minfo[39m: [hotswap] Router with id auth replaced successfully
[32minfo[39m: NodeBB attempting to listen on: 0.0.0.0:4567
[32minfo[39m: NodeBB is now listening on: 0.0.0.0:4567
[cluster] Child Process (8950) listening for connections.
[32minfo[39m: [sounds] Sounds OK
[32minfo[39m: [meta/templates] Compiling templates
[32minfo[39m: [themes] Successfully compiled templates.
[32minfo[39m: [meta/css] Done.
[32minfo[39m: [meta/css] Committing stylesheet (acpCache) to disk
[32minfo[39m: [meta/css] Stylesheet (acpCache) committed to disk.
[32minfo[39m: [meta/css] Done.
[32minfo[39m: NodeBB Ready
[32minfo[39m: [meta/css] Committing stylesheet (cache) to disk
[32minfo[39m: [meta/css] Stylesheet (cache) committed to disk.
[32minfo[39m: [meta/js] Successfully minified.
[32minfo[39m: [meta/js] Retrieved Mapping.
[32minfo[39m: [meta/js] Compilation complete
[32minfo[39m: [meta/js] Committing minfile to disk
[32minfo[39m: [meta/js] Client-side minfile and mapping committed to disk.
TypeError: Cannot read property 'topicData' of undefined
at setTimestampToFeedPublishedDate (/srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:198:23)
at /srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:167:9
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at async.waterfall.content (/srv/forums-gameredge/src/topics/create.js:117:13)
at fn (/srv/forums-gameredge/node_modules/async/lib/async.js:641:34)
at Object._onImmediate (/srv/forums-gameredge/node_modules/async/lib/async.js:557:34)
at processImmediate [as _immediateCallback] (timers.js:345:15)
[32minfo[39m: [app] Shutdown (SIGTERM/SIGINT) Initialised.
[32minfo[39m: [app] Database connection closed.
[32minfo[39m: [app] Web server closed to connections.
[32minfo[39m: [app] Shutdown complete.

Silvina

From Error.log

{"level":"error","message":"[meta/css] Syntax Error: .bg-variant is undefined - header.less on line 124","timestamp":"2014-09-24T15:16:06.561Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:23:05.307Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:41:00.746Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:42:00.466Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:43:00.691Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:44:00.683Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:45:00.394Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox connect ETIMEDOUT","timestamp":"2014-10-08T01:37:03.232Z"}
{"level":"error","message":"[[error:no-privileges]]","timestamp":"2014-10-08T02:00:00.602Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:56:40.082Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:57:40.087Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:58:40.088Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:59:40.096Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.theverge.com/rss/frontpage getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.073Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://isthereanydeal.com/rss/deals/us/ getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.073Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.polygon.com/rss/index.xml getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.074Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://robertsspaceindustries.com/comm-link/rss getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.074Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:01:20.105Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:01:40.102Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:02:40.108Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:03:40.112Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:04:40.118Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:05:40.122Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:06:40.127Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:07:40.132Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:08:40.136Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:09:40.141Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:10:40.146Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:11:40.150Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:12:40.151Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:13:40.158Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:14:40.162Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:15:40.165Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:16:40.170Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:17:40.169Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:18:40.173Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:19:40.182Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:20:40.186Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:21:40.187Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:22:40.186Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:23:40.195Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:24:40.199Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:25:40.202Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:26:40.207Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:27:40.209Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:28:31.073Z"}
{"level":"error","message":"[[error:too-many-posts, 10]]","timestamp":"2014-10-13T13:00:00.878Z"}

<baris>

Published 0.1.20 which fixes the crash.

error: You can only post once every 10 seconds - please wait before posting again

Looks like you regular user weren't able to post because of the post delay, I fixed that as well, let me know when you update.

{"level":"error","message":"error Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:28:31.073Z"}```

Not sure about those errors.

Silvina

Hi @baris
Installed the latest rss plugin and left it running over the night.

Some issues encountered. It is duplicating the posts.