I'm sorry, it took *how* many servers to post a single long message from Ghost to 5k fediverse accounts and handle some replies?
-
@polotek @kissane @fediversereport I remember when twitter first launched and everyone in that community were discovering the work needed to get RoR to scale to a global “firehose” size. Feels like we're back there again but in different ways.
-
Ziggy the Hamster :whyfox:🐹🌻replied to Viss on last edited by
@Viss @by_caballero @kissane @fediversereport
That was literally 16 years ago.
Most of the world still ran Windows XP and Internet Explorer.
Ruby and Rails are plenty fast today, and one could argue was when Twitter changed, but they were unwilling to make large changes to their code to be able to upgrade.
If you use GitHub, buy something with Shopify, or listen to a lot of podcasts, you are using Rails apps that perform just fine.
-
@kissane @fediversereport @thisismissem Yeah, that’s a downer. I wonder (based on total ignorance!) if there’s an incremental way to do this sort of thing? Like RSS broadcast at one end, full Masto ping-fest at the other, and Ghost could be somewhere inbetween?
-
Ziggy the Hamster :whyfox:🐹🌻replied to Ziggy the Hamster :whyfox:🐹🌻 on last edited by
@Viss @by_caballero @kissane @fediversereport oh shit, I also forgot about GitLab.
Mastodon isn’t a resource hog due to Rails, it’s a resource hog due to ActivityPub being somewhat badly implemented across the board, meaning that most actions result in amplification (ping N servers about a reply) and this amplification can sometimes be severe.
-
@pete @kissane @fediversereport many activitypub servers use queues for processing incoming activities, along with caches
-
@poswald @kissane @fediversereport I should be more clear about which issue I'm referring to. There are many technical issues to solve. I'm talking specifically about an issue that is specific to the decentralized nature of the fediverse. There are many optimizations that a company like twitter can do to scale fanouts, because they control both the source and the destination. Fediverse servers do not have that advantage. And more so, many servers are likely to be under resourced.
-
@poswald @kissane @fediversereport As far as I understand, mastodon's implementation is particularly naive today. If you have 5000 followers, every post creates 5000 jobs. There are many optimizations they can make today that will help. But I believe the problem of scaling of decentralized message delivery is going to be a huge bugbear if the fediverse keeps growing.
-
@kissane @pete @fediversereport @thisismissem You can federate with all of our WordPress.com plans, including the free plan! I assure you our infra can handle a lot more than the fediverse has been able to throw at it thus far.
-
@mattwiebe @pete @fediversereport @thisismissem Ah, I must have been reading old docs!
-
@kissane You do need that plan for installing plugins but many features, like ActivityPub, are available on all plans.
-
@mattwiebe So I went back to see what docs I'd run into as I searched while making dinner, in case it's useful: I googled "activitypub wordpress free" (not in quotes) bc I thought I remembered that it was free. The first few results offered no obvious answer, but the fourth had one that was accurate at the time, but isn't now.
(Might I have checked another post if I hadn't been stirring multiple pots of noodles? Very likely yes.)
-
@kissane Aha! That was the post from when we acquired the plugin and brought @pfefferle on board. This is when we launched it for everyone: https://wordpress.com/blog/2023/10/11/activitypub/
We should probably add a follow-up note to the older post
-
Jenniferplusplusreplied to Erin Kissane on last edited by
@kissane @fediversereport @thisismissem
This is armchair engineering, but I suspect there's an architecture issue here. I suspect ghost is organized around the assumption that secondary work is fast and easy. Like sending emails is mostly an API call to mailgun for them. But there's no mailgun for activitypub, so they're doing it themselves, and it happens in a blocking way. -
@mattwiebe @pfefferle Honestly this largely is a search results problem, but that's a problem we all live with forever somehow. I keep meaning to check out your implementation!
-
@bengo @kissane I'm telling everyone, push model was a mistake!!! (https://icosahedron.website/@greg/113222459291481648)
-
-
Jenniferplusplusreplied to Jenniferplusplus on last edited by
@kissane @fediversereport @thisismissem also worth noting ghost is built in nodejs, so it's more or less single threaded. 10 servers might very well have been 10 cpu cores in a different stack.
Anyway, this is to say that activitypub is very resource intensive, but this seems like there are complicating factors that can be worked through over time.
-
@polotek @poswald @kissane @fediversereport No, that’s not true. A post is only delivered once per domain. And we use keep-alive connections to shave off request setup time for repeat deliveries. If you have 5k followers from 2 domains, 2 requests will be made. Only if you have one follower per domain does it become 5k requests.
-
@jenniferplusplus @kissane @fediversereport
True, node.js is single threaded, however due to the async i/o you can usually process a fair number of requests simultaneously, because a single request doesn't have to finish before another is processed — you only get into trouble with synchronous APIs and like long-running processing (e.g., iterating over a lot of data)
I suspect besides queuing, there's something non-obvious here.
-
@kissane @fediversereport so further on this, by not using Fedify's queue option, they're also not using a queue to perform sends of Activities either.
This means delivery failures would also mess up Ghost rather good, because it'd result in one send failure cancelling others:
Sending activities | Fedify
Fedify provides a way to send activities to other actors' inboxes. This section explains how to send activities to others.
(fedify.dev)
