NodeBB Assets - Object Storage

PitaJ

@phenomlab what does crashing the admin console look like? And if you had any CDN stuff in your nginx config you'll need to clear you're cache again. Try a different browser if you haven't already.

The CDN plugin doesn't store anything in the database except for the settings key you've already deleted. You will need to restart NodeBB after deleting the key in order for that to take effect.

phenomlab

@PitaJ It looks like this

6Fetch finished loading: GET "<URL>".
admin:46          GET https://sudonix.dev/assets/plugins/nodebb-plugin-markdown/styles/obsidian.css net::ERR_ABORTED 404
18Fetch failed loading: GET "<URL>".
admin:45          GET https://sudonix.dev/assets/plugins/nodebb-plugin-emoji/emoji/styles.css?v=kccgr37fq3g net::ERR_ABORTED 404
admin:49          GET https://sudonix.dev/assets/admin.css?v=kccgr37fq3g net::ERR_ABORTED 404
admin:61          GET https://sudonix.dev/assets/admin.min.js?v=kccgr37fq3g net::ERR_ABORTED 404
admin:987 Uncaught ReferenceError: $ is not defined
    at HTMLDocument.prepareFooter (admin:987:4)
prepareFooter @ admin:987
13The FetchEvent for "<URL>" resulted in a network error response: the promise was rejected.
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
service-worker.js:14          Uncaught (in promise) TypeError: Failed to fetch
    at service-worker.js:14:11
(anonymous) @ service-worker.js:14
admin:34          GET https://sudonix.dev/assets/src/modules/composer.js?v=kccgr37fq3g net::ERR_FAILED
admin:35          GET https://sudonix.dev/assets/src/modules/composer/uploads.js?v=kccgr37fq3g net::ERR_FAILED
admin:36          GET https://sudonix.dev/assets/src/modules/composer/drafts.js?v=kccgr37fq3g net::ERR_FAILED
admin:37          GET https://sudonix.dev/assets/src/modules/composer/tags.js?v=kccgr37fq3g net::ERR_FAILED
admin:38          GET https://sudonix.dev/assets/src/modules/composer/categoryList.js?v=kccgr37fq3g net::ERR_FAILED
admin:39          GET https://sudonix.dev/assets/src/modules/composer/resize.js?v=kccgr37fq3g net::ERR_FAILED
admin:40          GET https://sudonix.dev/assets/src/modules/composer/autocomplete.js?v=kccgr37fq3g net::ERR_FAILED
admin:41          GET https://sudonix.dev/assets/templates/composer.tpl?v=kccgr37fq3g net::ERR_FAILED
admin:42          GET https://sudonix.dev/assets/language/en-GB/topic.json?v=kccgr37fq3g net::ERR_FAILED
admin:46          GET https://sudonix.dev/assets/plugins/nodebb-plugin-markdown/styles/obsidian.css net::ERR_FAILED
admin:44          GET https://sudonix.dev/assets/language/en-GB/tags.json?v=kccgr37fq3g net::ERR_FAILED
admin:43          GET https://sudonix.dev/assets/language/en-GB/modules.json?v=kccgr37fq3g net::ERR_FAILED
admin:47          GET https://sudonix.dev/assets/language/en-GB/markdown.json?v=kccgr37fq3g net::ERR_FAILED
touchicon-144.png:1          GET https://sudonix.dev/assets/uploads/system/touchicon-144.png 404
admin:1 Error while trying to use the following icon from the Manifest: https://sudonix.dev/assets/uploads/system/touchicon-144.png (Download error or resource isn't a valid image)

The site can no longer be loaded. I see traffic building on the CDN, but even with removing NGINX and just using NodeJS to serve the site, it's the same. The site is not rendered correctly, and I see a stack of errors in the console

One of particular interest is this

(blocked:NotSameOrigin)

The odd thing is that I'm pretty sure I enabled this in terms of CORS settings.

I've tried 5 different browsers and they all do the same thing.

phenomlab

What's more odd here is that the critical js files NodeBB needs are not being redirected to the CDN - see

Whereas others are

More concerning is that the assets still pointing to the origin server are now serving a 404

I think there is an issue with this plugin which prevents it from working correctly.

PitaJ

Yeah looks like I'll need to take another look at it in action. Maybe the upgrade Baris did changed something it wasn't supposed to. Unfortunately I'm away on vacation at the moment.

phenomlab

@PitaJ no problem at all. Thanks

phenomlab

@baris @PitaJ any plans to look into why this plugin does not work ?

phenomlab

@PitaJ any update please ?

Many thanks

PitaJ

Sorry I haven't had a chance yet. It's on my to-do list but I haven't had much time to dig into something like this recently.

What CDN service were you using? I'd like to try it with the same one.

phenomlab

@PitaJ I was testing with BunnyCDN. They have a free trial also. I also have an account with credit so happy to for you use that to test.

phenomlab

@PitaJ @baris any update at all with this plugin?

Many thanks

PitaJ

@phenomlab sorry I haven't had a chance yet. Been busy with travel lately but I'll try to make some time this month.

phenomlab

@PitaJ many thanks

razibal

You can achieve this relatively easily with Cloudflare R2 or AWS Cloudfront S3 storage. It would just require modifying the nodeBB build process to copy static assets to the appropriate bucket(s). The routing of requests for static assets to the various buckets would be delegated to cloudflare workers or cloudfront lambda@edge functions. In both cases, the workers are location aware and can route to the nearest replicated bucket. It should also be essentially free, unless you're hosting a huge set of files/media. R2 provides 10 GB/ month of storage for free with no egress charges. As an extra benefit, you would no longer need nginx in the pipeline as only API requests would be inbound to nodeBB.

phenomlab

@razibal to me this sounds quite invasive in terms of modifying the build process. I think it makes more sense for this to be handled as a plugin given that Cloudflare for instance isn't a cdn in the traditional sense.

There are a number of cdn providers who are by order of magnitude cheaper than the lowest paid plan on Cloudflare per month. Even using R2 which is technically freemium, there are going to be various limits and restrictions.

razibal

I guess it depends on your objectives, using a nodeBB plugin from my perspective is less than optimal because it requires a round trip to the origin. I assumed that the primary objective of the exercise is to serve assets from the edge to ensure minimum latency. I've been using cloudflare for some use cases for quite a while and their caching is quite robust (and cost-effective) if you leverage their cache reserve technology. As for the build process, there is no need to modify the core build process. Just a simple post-build step that copies all static assets to the R2/S3 bucket. Then its just a simple worker function to route all asset requests to R2 (after verifying that the asset is not in cache )

async function handleRequest(request) {
        const url = new URL(request.url);
        const {
            pathname,
            search
        } = url;
        if (  pathname.includes('/assets/') ) {
...

razibal

Here's a simple implementation using Cloudflare R2 storage

Create a new bucket and attach it to your domain as custom domain r2-static.yourdomain.com

Create a worker with the script

const host = 'nodebb.yourdomain.com';
const bucket = 'r2-static.yourdomain.com';
async function handleRequest(request) {
    const url = new URL(request.url);
    const { pathname, search } = url;
        const bucketUrl = request.url.replace(host, bucket)
        const response = await fetch(bucketUrl);
        return response;
}                                                                                                                                                         addEventListener('fetch', async event => {
    event.respondWith(handleRequest(event.request));
});;

Add a trigger to this worker that uses the route yourdomain/assets/*

Modify the scripts in your package.json in the nodebb folder:

    "scripts": {                                                                                                                                                  "start": "node loader.js",
        "debug": "NODE_ENV=dev DEBUG=* node loader.js",
        "build": "./nodebb build",
        "postbuild": "node postbuild.js",
...

Create a postbuild.js file in the nodebb root folder ( change the 'Your Cloudflare Account ID' to your actual account id)

const AWS = require('aws-sdk');
const mime = require('mime-types');
var ep = new AWS.Endpoint('[Your Cloudflare Account ID].r2.cloudflarestorage.com');                                                                       const { S3Client } = require('@aws-sdk/client-s3');
const S3SyncClient = require('s3-sync-client');
const client = new S3Client({ region: 'auto', endpoint: ep });
const { sync } = new S3SyncClient({ client: client });

const EventEmitter = require('events');
const { TransferMonitor } = require('s3-sync-client');

const monitor = new TransferMonitor();
monitor.on('progress', (progress) => console.log(progress));
setTimeout(() => monitor.abort(), 300000);

async function syncStaticFiles() {
    await sync('./build/public', 's3://nodebb-static/assets', {
        monitor,
        maxConcurrentTransfers: 1000,
        commandInput: {
            ACL: 'public-read',
            ContentType: (syncCommandInput) => mime.lookup(syncCommandInput.Key) || 'text/html'
        }
    });
    process.exit()
}
syncStaticFiles()

And that should do it. Every time you perform a build using the command yarn build, a nodebb build will be executed and the static assets should get copied to the R2 bucket. The cloudflare workers will ensure that they are served from the bucket.

A typical nodeBB installation has asssets of less than 50MB, the free tier of R2 includes 10 GB with no egress charges.
The free tier for cloudflare workers includes 100,000 requests per day.

phenomlab

@razibal Interesting. Would this work with the standard ./nodebb build ?

razibal

It is using the standard ./nodebb build just leveraging the package.json to automatically execute a postbuild step

phenomlab

@razibal said in NodeBB Assets - Object Storage:

Create a new bucket and attach it to your domain as custom domain r2-static.yourdomain.com

This isn't entirely clear on CF. If I attempt to connect a custom domain, it fails and tells me

DNS record for this domain already exists on zone. (Code: 10056)

razibal

@phenomlab you are probably using the root domain, you need to specify a subdomain that will get mapped as dns record. For example r2-static.yourdomain.com instead of yourdomain.dom