NodeBB Assets - Object Storage

[yasas]

Hi there,

Greetings!

I have been thinking about serving assets from an S3 bucket or another object storage. (I came across the plugin https://github.com/NodeBB-Community/nodebb-plugin-s3-uploads but yet to check)

I was wondering whether it's possible to "effectively" serve from S3 buckets/a different object storage in different availability zones depending on the client (US/UK/Europe/Asia) to reduce the time that the assets take to load.

Also, I would love to hear about any other possible configurations anyone is using/suggesting with "serving different geo-locations" in mind.

Thank you

[pkuanvil]

I guess you want to return different S3 URL link (i.e. return different HTML) based on geolocation IP? Sounds very complicated

[pkuanvil]

If you use multiple S3 geolocations, you force the S3 plugin to upload one resource to multiple S3 endpoints. I think it's not a great idea for the server to handle this kind of duplication; it's best left for a CDN to do this. CDN does a better job at delivering static file than different S3 geolocations do.

[julian]

@pkuanvil is correct, I'm personally of the opinion that one should use other well-known solutions to problems, and this is one that CDNs have effectively solved... albeit at the cost of money

[phenomlab]

@julian I'm interested in how any CDN could work with NodeBB as there is no plugin to control what is in the push or pull zone - unless you are referring to Cloudflare which isn't a CDN in the typical sense.

[julian]

Hmm, you're right. Perhaps I was thinking too high-level there.

There's no plugin to my knowledge that integrates with a CDN, but that's not to say there couldn't be one.

[phenomlab]

@julian it would be great if one could be created. On the CDN side, there's BunnyCDN which is very cheap and has an event expanding network. I personally would sponsor development of this.

[PitaJ]

There's https://github.com/NodeBB/nodebb-plugin-cdn but it's probably incompatible with newer NodeBB versions

[phenomlab]

@PitaJ yes, I saw that but haven't tried it. @julian @baris any appetite to make this work with v2 and v3 ? Even if only v3 I'd be happy.

[baris]

I am updating that plugin to work on ^2.x

[baris]

Published @nodebb/[email protected]

[phenomlab]

@baris Thanks !! Really appreciated. Will give this a whirl..

[phenomlab]

@baris Trying this now, and see traffic building on my CDN, but according to the network tab in Chrome, it's still pulling from my server ?

[baris]

What is the url of the requested file? And is privateUploads turned on in your settings? I think this plugin doesn't work if uploaded files are private. https://github.com/NodeBB/nodebb-plugin-cdn/commit/59a069827340e718d860a3ecf458b9a01ffc4711

[phenomlab]

@baris I've set the URL to https://cdn.sudonix.dev and uploads are not private. The URL of the requested assets in this case should be https://cdn.sudonix.dev but are in fact https://sudonix.dev

[baris]

What about the instructions on https://github.com/NodeBB/nodebb-plugin-cdn/blob/master/docs/nginx.md?

[phenomlab]

@baris Tried those, and landed up hosing my DEV environment have had to remove this plugin. I'll try it again at some point, but it's way more complex than it should be in my view sadly.

[PitaJ]

The CDN plugin uses redirects. It doesn't need the nginx stuff, that's just optimization. (nginx is much better at a simple redirect rule than nodejs) I'm not sure exactly what you're doing in the browser console but it's likely you will still get initial requests to your NodeBB domain, especially if your cache is disabled like the browser tools often does when open.

[phenomlab]

@PitaJ thanks, but the "redirects" don't seem to work. After making changes to nginx it completely skewered my forum and I had to use the cli to disable.

[PitaJ]

@phenomlab yeah I'd get it working without nginx first. What exactly do you see in your browser network tab?