[nodebb-plugin-rss] RSS Feed plugin

Silvina

@julian I can probably do that. where can i find the full stack trace?
what ever that is shown on the ./nodebb dev console window?

julian

@Silvina said:

what ever that is shown on the ./nodebb dev console window?

Yep, that's it

Silvina

@Silvina said:

Rssitfor.me

(www.rssitfor.me)

[www-data@vultr forums-gameredge]$ ./nodebb dev
Launching NodeBB in "development" mode.
To run the production build of NodeBB, please use "forever".
More Information: https://docs.nodebb.org/en/latest/running/index.html
info: NodeBB v0.5.0 Copyright (C) 2013-2014 NodeBB Inc.
info: This program comes with ABSOLUTELY NO WARRANTY.
info: This is free software, and you are welcome to redistribute it under certain conditions.
info:
info: Time: Wed Sep 24 2014 23:20:12 GMT+0800 (SGT)
info: Initializing NodeBB v0.5.0
info: * using configuration stored in: /srv/forums-gameredge/config.json
info: * using redis store at 127.0.0.1:6379
info: * using themes stored in: /srv/forums-gameredge/node_modules
info: Base Configuration OK.
warn: You have no redis password setup!
morgan deprecated morgan(options): use morgan("default", options) instead src/logger.js:128:22
morgan deprecated default format: use combined format src/logger.js:128:22
info: [notifications.init] Registering jobs.
info: [user.startJobs] Registering User Jobs
info: Enabling 'trust proxy'
info: [plugins] Initializing plugins system
info: [themes] Using theme nodebb-theme-lavender
info: Middlewares loaded.
info: [plugins] Found 1 CSS file(s) for plugin nodebb-plugin-youtube

warn: [plugins/nodebb-plugin-mentions] This plugin may not be compatible with your version of NodeBB. This may cause unintended behaviour or crashing.
warn: [plugins/nodebb-plugin-mentions] In the event of an unresponsive NodeBB caused by this plugin, run ./nodebb reset plugin="nodebb-plugin-mentions".

info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-mentions
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-mentions
warn: [plugins/nodebb-plugin-spoilers] staticDir is deprecated, use staticDirs instead
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-spoilers
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-spoilers
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-spam-be-gone
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-spam-be-gone
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-linkcheck
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-linkcheck
info: [plugins] Found 1 js file(s) for plugin nodebb-plugin-twitter
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-emoji-extended
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-emoji-extended
info: [plugins] Found 1 js file(s) for plugin nodebb-theme-blacknred
info: [plugins] Found 1 CSS file(s) for plugin nodebb-widget-essentials
info: [plugins] Found 1 LESS file(s) for plugin nodebb-plugin-markdown
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-markdown
warn: [plugins/nodebb-plugin-featured-threads] staticDir is deprecated, use staticDirs instead
info: [plugins] Found 1 CSS file(s) for plugin nodebb-plugin-featured-threads
info: [plugins] Found 2 js file(s) for plugin nodebb-plugin-desktop-notifications
info: [plugins] Found 3 js file(s) for plugin nodebb-theme-lavender
info: [plugins] Loaded plugin: nodebb-plugin-rss
info: [plugins] Loaded plugin: nodebb-plugin-soundcloud
info: [plugins] Loaded plugin: nodebb-plugin-twitchtv
info: [plugins] Loaded plugin: nodebb-plugin-emailer-mailgun
info: [plugins] Loaded plugin: nodebb-plugin-youtube
info: [plugins] Loaded plugin: nodebb-plugin-mentions
info: [plugins] Loaded plugin: nodebb-plugin-spoilers
info: [plugins] Loaded plugin: nodebb-plugin-spam-be-gone
info: [plugins] Loaded plugin: nodebb-plugin-linkcheck
info: [plugins] Loaded plugin: nodebb-plugin-twitter
info: [plugins] Loaded plugin: nodebb-plugin-soundpack-default
info: [plugins] Loaded plugin: nodebb-plugin-emoji-extended
info: [plugins] Loaded plugin: nodebb-theme-blacknred
info: [plugins] Loaded plugin: nodebb-widget-essentials
info: [plugins] Loaded plugin: nodebb-plugin-imgur
info: [plugins] Loaded plugin: nodebb-plugin-markdown
info: [plugins] Loaded plugin: nodebb-plugin-slack
info: [plugins] Loaded plugin: nodebb-plugin-featured-threads
info: [plugins] Loaded plugin: nodebb-plugin-desktop-notifications
info: [plugins] Loaded plugin: nodebb-plugin-dbsearch
info: [plugins] Loaded plugin: nodebb-theme-lavender
info: [plugins] Sorting hooks to fire in priority sequence
info: [plugins] Plugins OK
info: [meta/css] Minifying LESS/CSS
warn: [plugins] The action:app.load hook is deprecated in favour of static:app.load, please notify the developers of the following plugins:

nodebb-plugin-featured-threads
warn: [meta/css] File not found! nodebb-plugin-featured-threads/css/main.css
info: [plugins/spam-be-gone] Settings loaded
info: [sounds] Sounds OK
info: [meta/css] Done.
info: [themes] Compiling templates
info: [themes] Successfully compiled templates.
info: NodeBB attempting to listen on: 0.0.0.0:4567
info: NodeBB is now listening on: 0.0.0.0:4567
info: [meta/js] Successfully minified.
info: [meta/js] Retrieved Mapping.
info: [meta/js] Compilation complete
info: NodeBB Ready
info: [socket] uid 1 (1JqnG1PdkeqC-cOkP5ac) has successfully reconnected.
warn: Route requested but not found: /css/assets/vendor/masonry.js
warn: Route requested but not found: /css/assets/vendor/imagesLoaded.js
warn: Route requested but not found: /css/assets/vendor/masonry.js
warn: Route requested but not found: /css/assets/vendor/imagesLoaded.js
warn: Route requested but not found: /css/assets/vendor/imagesLoaded.js
warn: Route requested but not found: /css/assets/vendor/masonry.js
error: error Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null

Silvina

[www-data@vultr forums-gameredge]$ ./nodebb log
at /srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:167:9
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at async.waterfall.content (/srv/forums-gameredge/src/topics/create.js:117:1 3)
at fn (/srv/forums-gameredge/node_modules/async/lib/async.js:641:34)
at Object._onImmediate (/srv/forums-gameredge/node_modules/async/lib/async.j s:557:34)
at processImmediate [as _immediateCallback] (timers.js:345:15)
info: [app] Shutdown (SIGTERM/SIGINT) Initialised.
info: [app] Database connection closed.
info: [app] Web server closed to connections.
info: [app] Shutdown complete.

After upgrading to nodebb 0.5.1, i noticed that my nodebb instance been dying at random times.
Is this plugin causing the issue?

<baris>

Can you post the full log, should be at logs/output.log.

Silvina

Hi @baris
Running via ./nodebb dev

info: [notifications.prune] Removing expired notifications from the database.
warn: Route requested but not found: /forum/index.php?topic=256.90
error: [[error:too-many-posts, 10]]
error: TypeError: Cannot read property 'topicData' of undefined
at setTimestampToFeedPublishedDate (/srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:198:23)
at /srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:167:9
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at /srv/forums-gameredge/src/user.js:190:12
at /srv/forums-gameredge/node_modules/async/lib/async.js:592:17
at done (/srv/forums-gameredge/node_modules/async/lib/async.js:135:19)
at /srv/forums-gameredge/node_modules/async/lib/async.js:32:16
at /srv/forums-gameredge/node_modules/async/lib/async.js:589:21
at /srv/forums-gameredge/src/database/redis/sets.js:44:4
[cluster] Child Process (9827) has exited (code: 1)
[cluster] Spinning up another process...

From Output log

[32minfo[39m: Time: Wed Oct 15 2014 00:28:06 GMT+0000 (UTC)
[32minfo[39m: Initializing NodeBB v0.5.1
[32minfo[39m: * using configuration stored in: /srv/forums-gameredge/config.json
[32minfo[39m: * using redis store at 127.0.0.1:6379
[32minfo[39m: * using themes stored in: /srv/forums-gameredge/node_modules
[32minfo[39m: [user.startJobs] Registering User Jobs
[32minfo[39m: Enabling 'trust proxy'
[cluster] Child Process (8950) listening for connections.
[33mwarn[39m: [plugins/nodebb-plugin-spoilers] staticDir is deprecated, use staticDirs instead
[32minfo[39m: [hotswap] Router with id plugins replaced successfully
[32minfo[39m: [plugins] All plugins reloaded and rerouted
[32minfo[39m: [meta/css] Minifying LESS/CSS
[32minfo[39m: [hotswap] Router with id auth replaced successfully
[32minfo[39m: NodeBB attempting to listen on: 0.0.0.0:4567
[32minfo[39m: NodeBB is now listening on: 0.0.0.0:4567
[cluster] Child Process (8950) listening for connections.
[32minfo[39m: [sounds] Sounds OK
[32minfo[39m: [meta/templates] Compiling templates
[32minfo[39m: [themes] Successfully compiled templates.
[32minfo[39m: [meta/css] Done.
[32minfo[39m: [meta/css] Committing stylesheet (acpCache) to disk
[32minfo[39m: [meta/css] Stylesheet (acpCache) committed to disk.
[32minfo[39m: [meta/css] Done.
[32minfo[39m: NodeBB Ready
[32minfo[39m: [meta/css] Committing stylesheet (cache) to disk
[32minfo[39m: [meta/css] Stylesheet (cache) committed to disk.
[32minfo[39m: [meta/js] Successfully minified.
[32minfo[39m: [meta/js] Retrieved Mapping.
[32minfo[39m: [meta/js] Compilation complete
[32minfo[39m: [meta/js] Committing minfile to disk
[32minfo[39m: [meta/js] Client-side minfile and mapping committed to disk.
TypeError: Cannot read property 'topicData' of undefined
at setTimestampToFeedPublishedDate (/srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:198:23)
at /srv/forums-gameredge/node_modules/nodebb-plugin-rss/index.js:167:9
at /srv/forums-gameredge/node_modules/async/lib/async.js:544:30
at async.waterfall.content (/srv/forums-gameredge/src/topics/create.js:117:13)
at fn (/srv/forums-gameredge/node_modules/async/lib/async.js:641:34)
at Object._onImmediate (/srv/forums-gameredge/node_modules/async/lib/async.js:557:34)
at processImmediate [as _immediateCallback] (timers.js:345:15)
[32minfo[39m: [app] Shutdown (SIGTERM/SIGINT) Initialised.
[32minfo[39m: [app] Database connection closed.
[32minfo[39m: [app] Web server closed to connections.
[32minfo[39m: [app] Shutdown complete.

Silvina

From Error.log

{"level":"error","message":"[meta/css] Syntax Error: .bg-variant is undefined - header.less on line 124","timestamp":"2014-09-24T15:16:06.561Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:23:05.307Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:41:00.746Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:42:00.466Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:43:00.691Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:44:00.683Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.rssitfor.me/getrss?name=%40AmazonAppADay Cannot read property 'feed' of null","timestamp":"2014-09-24T15:45:00.394Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox connect ETIMEDOUT","timestamp":"2014-10-08T01:37:03.232Z"}
{"level":"error","message":"[[error:no-privileges]]","timestamp":"2014-10-08T02:00:00.602Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:56:40.082Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:57:40.087Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:58:40.088Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T10:59:40.096Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.theverge.com/rss/frontpage getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.073Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://isthereanydeal.com/rss/deals/us/ getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.073Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed http://www.polygon.com/rss/index.xml getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.074Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://robertsspaceindustries.com/comm-link/rss getaddrinfo ESRCH","timestamp":"2014-10-13T11:00:40.074Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:01:20.105Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:01:40.102Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:02:40.108Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:03:40.112Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:04:40.118Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:05:40.122Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:06:40.127Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:07:40.132Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:08:40.136Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:09:40.141Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:10:40.146Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:11:40.150Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:12:40.151Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:13:40.158Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:14:40.162Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:15:40.165Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:16:40.170Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:17:40.169Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:18:40.173Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:19:40.182Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:20:40.186Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:21:40.187Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:22:40.186Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:23:40.195Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:24:40.199Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:25:40.202Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:26:40.207Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:27:40.209Z"}
{"level":"error","message":"[[nodebb-plugin-rss:error]] Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:28:31.073Z"}
{"level":"error","message":"[[error:too-many-posts, 10]]","timestamp":"2014-10-13T13:00:00.878Z"}

<baris>

Published 0.1.20 which fixes the crash.

error: You can only post once every 10 seconds - please wait before posting again

Looks like you regular user weren't able to post because of the post delay, I fixed that as well, let me know when you update.

{"level":"error","message":"error Error pulling feed https://rssfeeds.s3.amazonaws.com/goldbox getaddrinfo ESRCH","timestamp":"2014-10-13T11:28:31.073Z"}```

Not sure about those errors.

Silvina

Hi @baris
Installed the latest rss plugin and left it running over the night.

Some issues encountered. It is duplicating the posts.

<baris>

You can try with version 0.1.21, although if the publishedDate of the articles are changing then they will get reposted. The plugin checks the latest published date and posts articles that are newer. So if an articles publishedDate is getting updated to something more recent it gets reposted.

Tanner

I had to remove the plugin because of the duplicate post issue.

peter

Plugin looks good.

@baris is there a way to set each feed post to go to draft?

trevor

@Tanner said:

I had to remove the plugin because of the duplicate post issue.

Just thought of a good idea for core integration; Duplicate post checking.

charles

@trevor said:

@Tanner said:

I had to remove the plugin because of the duplicate post issue.

Just thought of a good idea for core integration; Duplicate post checking.

+1 for duplicate check

Silvina

@charles said:

@trevor said:

@Tanner said:

I had to remove the plugin because of the duplicate post issue.

Just thought of a good idea for core integration; Duplicate post checking.

+1 for duplicate check

+2 for duplicate checks

trevor

For a bonus, duplicate Title checks as well.

Chris

This working in the latest release? Seems to stop after one post for me.

Also seems to be parsing markdown or something improperly but I think that's another issue altogether.

Chris

Ah yes, duplicate post issue here as well. +1 Duplicate post.

Chris

$30 paypal bounty to the first person that can fix the duplicate posting issue

<baris>

@Pyrax sent a PR for the duplicate post issue, merged and published 0.2.1

Fix duplicate post issue by Pyrax · Pull Request #6 · barisusakli/nodebb-plugin-rss

This should fix the known issue which resulted in posting the last entry of a RSS feed multiple times.

GitHub (github.com)