I'm sorry, it took *how* many servers to post a single long message from Ghost to 5k fediverse accounts and handle some replies?
-
@kissane @fediversereport this also means incoming activities are likely holding connections open longer triggering auto scaling because usually that happens based on request count and CPU/Memory
-
@kissane @fediversereport sure!
-
@kissane ugh so I believe this is a mastodon fail and def not a protocol fail.
(at risk of being wrong, and sorry if I am)
Mastodon only shows things that are pushed into it from other servers.
Push is only important for real-time synchronization.
IMO it's obviously a good idea for any AP Server to check like once a day or on-demand to make a single HTTP Request to an Actor's outbox and show its recent stuff.
But mastodon doesn't do this.
-
@kissane so for a publisher to ensure that all their followers see a thing, and because in practice so many people use mastodon, a publisher has to spend all this compute/bandwidth/energy to make sure that they push the post into each followers inbox.
It would be better for everyone for mastodon and everyone else to just fetch a followee's outbox once a day (aka 'what RSS does') and just progressively enhance into receiving pushes from publishing actors that want to do it. but masto requires it
-
@kissane @4d3fect I used it for thousands of subscribers years ago and it was like $9/mo. activitypub may only make sense for self hosting if having thousands of masto followers is burning that much juice. I'd be interested to benchmark 5thousand followers on 100 masto servers versus 5000 on 100 threads versus 100 friendica versus 100 pleroma etc etc... someone write a grant proposal!
️ -
@kissane @fediversereport @thisismissem I read that post as “we set this up using our standard systems and they are definitely not right for this, so we’re going to have to figure out a different system that fits and that’ll take a while, but we’ll get there.” But I’m just watching and enjoying the messy honesty of these posts, not necessarily understanding the details.
-
@Viss @kissane @fediversereport ghost is using fedify! which is fairly modern JS AFAIK.
-
@Viss @kissane @fediversereport might be more networking intensive than compute intensive, fwiw
-
@by_caballero @kissane @fediversereport yeah the issue here isn't the front end js stuff - its whats running under the hood, powering any given mastodon instance. its a ton of ruby on rails, which twitter showed the world "isnt exactly the best language/framework to do huge, high volume. small bandwidth transactions and queueing systems". its why twitter scrambled to get off RoR and moved to scala as they grew.
i was VERY SURPRISED, to say the least, learning that mastodon still uses RoR.
-
@Viss @kissane @fediversereport right but ghost doesn't run mastodon, it runs fedify which interoperable with mastodon API. the 10 servers in question are fedify servers:
https://fedify.dev/ -
@pete @fediversereport @thisismissem I think that's a reasonable take!
It's also partly the information that to federate Ghost posts to a group that's equivalent in size to the number of people who follow me on Mastodon will cost at least $155/month—not counting email subscribers—is somewhat startling to me.
Like, I fully understand that infra isn't free, but Wordpress (which I am not going to use) charges $25/mo for a plan that includes AP federation.
-
@by_caballero @kissane @fediversereport the repo that project links to appears to be a node/npm project with dirs indicating it runs inside a docker container.
so this is javascript running 'as a server' inside docker, which is very different than ruby on rails, but in its own way, super topheavy in terms of layers of abstraction, and also entirely unsurprising it gets loaded down easily.
-
@by_caballero @kissane Did it support comments sections, notifications &c?
-
@polotek @kissane @fediversereport I remember when twitter first launched and everyone in that community were discovering the work needed to get RoR to scale to a global “firehose” size. Feels like we're back there again but in different ways.
-
Ziggy the Hamster :whyfox:🐹🌻replied to Viss on last edited by
@Viss @by_caballero @kissane @fediversereport
That was literally 16 years ago.
Most of the world still ran Windows XP and Internet Explorer.
Ruby and Rails are plenty fast today, and one could argue was when Twitter changed, but they were unwilling to make large changes to their code to be able to upgrade.
If you use GitHub, buy something with Shopify, or listen to a lot of podcasts, you are using Rails apps that perform just fine.
-
@kissane @fediversereport @thisismissem Yeah, that’s a downer. I wonder (based on total ignorance!) if there’s an incremental way to do this sort of thing? Like RSS broadcast at one end, full Masto ping-fest at the other, and Ghost could be somewhere inbetween?
-
Ziggy the Hamster :whyfox:🐹🌻replied to Ziggy the Hamster :whyfox:🐹🌻 on last edited by
@Viss @by_caballero @kissane @fediversereport oh shit, I also forgot about GitLab.
Mastodon isn’t a resource hog due to Rails, it’s a resource hog due to ActivityPub being somewhat badly implemented across the board, meaning that most actions result in amplification (ping N servers about a reply) and this amplification can sometimes be severe.
-
@pete @kissane @fediversereport many activitypub servers use queues for processing incoming activities, along with caches
-
@poswald @kissane @fediversereport I should be more clear about which issue I'm referring to. There are many technical issues to solve. I'm talking specifically about an issue that is specific to the decentralized nature of the fediverse. There are many optimizations that a company like twitter can do to scale fanouts, because they control both the source and the destination. Fediverse servers do not have that advantage. And more so, many servers are likely to be under resourced.
-
@poswald @kissane @fediversereport As far as I understand, mastodon's implementation is particularly naive today. If you have 5000 followers, every post creates 5000 jobs. There are many optimizations they can make today that will help. But I believe the problem of scaling of decentralized message delivery is going to be a huge bugbear if the fediverse keeps growing.
