"In 2024, it cost $N to run a Mastodon instance with ~1000 active users for a year.

Jenniferplusplus

@mekkaokereke Actually, I should note that while I think it's important to cut the cost of running fediverse servers, there's only so much savings to be had.

I think it's more important to create ways for servers to generate revenue and be financially self-sufficient. And to do so without selling ads, because the incentive structures that advertising creates are anathema to the aspects of this place that are actually good.

I'm hoping it can be done with paid features.

Parsnicketty

@mekkaokereke @gkrnours that’s an interesting idea.

What if such a team triaged content that everyone* (*who I want to talk to) agrees is unacceptable like death threats, hate speech, CSAM, and then referred more nuanced problems to instance-specific mods?

There needs to be a way for people to learn moderating, do it for awhile, and then rotate out before they’re scarred for life.

craignicol

@dalias @mekkaokereke people accept instances with barely any moderation (well, a lot of instances defederate them, but they remain popular) and I'm sure threads will stay. It's not something I'll be comfortable with, but it's a trade-off people who don't feel threatened here might accept.

Cassandrich

@craignicol @mekkaokereke The horror of AI shit doing moderation is not false negatives but false positives. No one wants to be where they'll get randomly banned by opaque processes.

:PUA: Shlee fucked around and

@risottobias @mekkaokereke Bingo. the more instances the better for insights...

3 instances have this user suspended a user with this email.... for
* x reason
* z reason
* y reason
how do you want to proceed? etc etc

craignicol

@dalias @mekkaokereke I know people who had temporary bans from Facebook for unknown reasons but got reinstated and are still there. I don't see why that behaviour wouldn't be replicated here. If moderation isn't something you think about much, why would it matter to you if it was a faceless AI or a faceless minimum wage worker (or volunteer) that's deleted that one message in support of Harris?

Cassandrich

@craignicol @mekkaokereke On Facebook they stay because it's holding their social relationships hostage. On fedi there's no reason to stick with a shit instance that's at high risk of taking your social relationships away when there are plenty of good ones.

Nik

@esk @thisismissem @mekkaokereke @dma As a (non-Fedi) benchmark, Metafilter (/cc @jessamyn) costs about $250K a year, and most of that is moderator staff cost.

Profit & Loss - Google Drive

Google Drive (drive.google.com)

Renaud Chaput

@esk @thisismissem @mekkaokereke @dma the upgrade to Ruby 3.3 should have resulted in a 15% CPU improvement, from what I remember this was similar to what our switch to libvips did

LisPi

@dalias @mekkaokereke @craignicol To be fair, all server-centric options are at risk of taking relationships away randomly if they suffer catastrophic failure.

It's a pretty strong indictment of the model and Nomadic Identity protocols only mitigate some of the flaws.

Esk 🐌⚡💜

@thisismissem @mekkaokereke @dma so stats, based on dec 2024 exit run rate (rounded for simplicity):

#hachyderm costs about $1600/mo to run. this is up somewhat, as we've started to add some infra as part of our resilience plan announced in nov.

we currently have about:
- 55000 users
- 9700 MAU
- 3.7M toots

yielding:
- $.03/user/mo
- $.16/active user/mo
- $.0004/toot

from a raw compute & storage perspective.

again, this is based on 100% volunteer work. today, our mods and infra folk graciously donate their time to keep this thing going.

hypothetically, if we paid them, say, $120k USD/yr (chose this to make the math cleaner), that would add $10k/person/mo to the cost.

if we go with a staff of eight (mix of mod & infra), that adds $80k/mo to the run rate, for a total of $81,600/mo, yielding:

- $1.48/user/mo
- $8.41/active user/mo
- the toot figure is silly, so i'm not calculating it again

orders of magnitude of difference.

we could argue about the staff size - i went with roughly what we have today and assumed we made everyone full time so they could hachy for 32/hr/wk vs. calculating the number of hours we actually work. e.g. maybe we could it out at ~$4.50/user/mo, but still a multiple orders of magnitude bump from the raw infra cost.

Emelia 👸🏻

@esk @mekkaokereke @dma we're definitely not all working 32/hr/wk on hachyderm.. but that number rises and falls as we work on things, it's not continuous

Esk 🐌⚡💜

@thisismissem @mekkaokereke @dma yup - since this was esk's hypothetical dream staffing model where we can do many great things for the hachyderm, i modeled it after what i'd consider a minimally viable team running a globally available production service. imo, that's eight people so there's sufficient degrees of freedom to take on 1-3 epics, have a reasonable oncall rotation, and provide plenty of space for holidays, contribute to upstream mastodon/other projects, etc.

could we do it with one or two people, sure. but that isn't a team.

dominic :verified_paw:

@esk @thisismissem @mekkaokereke so if every active user gave us $5 a month we'd be pretty close to staffing a team.

that's surprisingly achievable. time to run some "for the price of a coffee month..." banners!

in case it's not clear I'm not suggesting this for realsies, but I am pleasantly surprised by how little, relatively, it would take to start paying folks for their time.

Esk 🐌⚡💜

@thisismissem @puppygirlhornypost2 @risottobias @mekkaokereke @dma

from an overall outbound bandwidth perspective, we generally use about 22-23TB. our "custom CDN" is hosted in linode, and is a handful of VMs running nginx. with the linodes (VMs) we're running, we get an allotment of 20 TB/mo egress for "free" (as in we paid for the VMs). the 2-3 TB overage usually costs us an extra $10-15/mo.

in digital ocean, as of this morning we have 19.1 TB in media. we do **not** have a retention/sweep policy in place for deleting old media; this is something for us to consider in the future as a cost containment mechanism.

we also store 7 TB of database backups. we hold 14 days of database backups (weekly full, daily differential based on that full). this is probably a very conservative posture and could be optimized, but we reeeeealllly don't wanna lose the database.

you're spot on that, if you're hosting your own edge, in-memory caching is critical for both responsiveness back to your consumer and happiness of your servers to avoid over-saturation.

Raphael Lullis

@esk @thisismissem @mekkaokereke @dma

https://communick.com offers Mastodon + Lemmy + Matrix+ Funkwhale for $29 PER YEAR, yet it struggles to get enough people interested to cover its hosting costs, let alone my labor.

Is it really bad marketing on my part?

Jenniferplusplus

@lmorchard @mekkaokereke
That might be a nice thing to do, but I'm less bothered by this particular problem. For a couple reasons:

1. It's a ridiculous problem to have in the first place. If every instance in the entire fediverse requests that page in the span of 5 minutes, that's still only ~100 requests per second.
2. The correct solution is to attach verifiable signatures to og data, so that it can be forwarded with confidence. Not to make the whole rest of the web use a big shared cache.

dominic :verified_paw:

@raphael @esk @thisismissem @mekkaokereke not necessarily. I think folks have a hard time understanding the value of services. especially when they're open source. people are terrible at value judgements generally.

Raccoon🏳️‍🌈

@lispi314 @craignicol @dalias @mekkaokereke
I want to note something, as the person in charge of moderation on a major server: there is a lot that can be done and hasn't with non-AI based automated moderation. Why is there not a feature built into Mastodon to...

- automatically report posts with racial slurs when they show up?

- automatically report when a user posts the same link several times in a row?

- automatically report when a user from your instance has been mentioned or linked on a hashtag like FediBlock?

- remind users of the report button when they get PMs that might be potential harassment?

We don't need AI. It would be wasteful, very costly, and likely have serious trust and reliability issues. We just need better systems to be built in by default to help us volunteers catch things quickly.

Rupert

@dma @raphael @esk @thisismissem @mekkaokereke If as a society we better understood the cost of "free" we wouldn't be in this exact mess.