Performance and strange increase pageviews (google analytics vs dashboard)
-
Month ago, 9 sep 2016 I import posts from phpbb3 and run nodebb in production. Three weeks later, 29 sep 2016 something strange happens, suddenly pageviews increase 4-5 times, but unique visitors leave at same level. I can't remember, but I think I did't install any plugins or make some changes.
CPU load also increase and now 20%-30% (DigitalOcean, 1CPU/512mb). But when I look at Google Analytics, nothing changed, still same amount sessions/users/pageviews.
Any ideas why? I can't say that my forum is popular now, only 1-2 topic/posts per day. I think its some web crowlers, but why its create so much load?
Some statistics:
Users 2,839 (>90% not active anymore)
Topics 7,051
Posts 116,061Dashboard (unique visitors amount 900-5500):
-
GA will actively filter out bots from their report, because they're bots
Best bet is to investigate the nginx logs and see who's crawling your site. For a new import, it is not unexpected to see a flurry of new activity from bots as they discover that you have new, fresh pages to index.
While GA will throttle back their crawler if your site slows down, there are no such guarantees with other spiders, and so this may be the cause.
-
I know Awstats (http://awstats.org) is very old but it works well for basic weblog parsing. The other alternative is to push your weblog files to a Piwik (https://piwik.org) site daily (I have mine set to do that as the daily log rotate occurs.) Both of those will allow you to examine your real traffic in greater detail.
I have noticed what I believe are connections from RSS feeders that inflate my forum activity.