Possible Membership Tiers for Fediverse Servers

Scott M. Stolz

@Raphael Lullis It is a known incompatibility issue. It has to do with how each platform handles aliases (or doesn't).

Hubzilla uses something called the Zot protocol, which is basically our version of ActivityPub. It supports certain things that ActivityPub does not support, which is why we still use it.

In Hubzilla and Zot Protocol, I can have clones (duplicates/aliases) of my channel. So [email protected] and [email protected] reside on different servers and are constantly synced. If one server goes down, I can log into the other. If I want to move servers, I can. Since Hubzilla and Zot allow cloning, they also recognize cloned accounts. They know these two accounts are the same account.

But, Mastodon and ActivityPub do not support this feature. ActivityPub thinks that [email protected] and [email protected] are two different accounts. So if you follow both of them, you get the message twice in Mastodon.

For ActivityPub followers, we have to send out copies of posts from all of the clones, because some people may follow one clone, and other people may follow another. This ensures that someone following an old but active clone can still get our posts. We are assuming that most people would follow only one clone and perhaps don't even know about the other clones.

Hubzilla does announce that these are cloned channels (aliases) via Webfinger, but to my knowledge, no ActivityPub platform checks this and none use this information to suppress duplicate posts.

It really isn't about blame. It is just that Mastodon does not support nomadic identity, so you get weird things like this.

Scott M. Stolz

@Raphael Lullis It is a known incompatibility issue. It has to do with how each platform handles aliases (or doesn't).

Hubzilla uses something called the Zot protocol, which is basically our version of ActivityPub. It supports certain things that ActivityPub does not support, which is why we still use it.

In Hubzilla and Zot Protocol, I can have clones (duplicates/aliases) of my channel. So [email protected] and [email protected] reside on different servers and are constantly synced. If one server goes down, I can log into the other. If I want to move servers, I can. Since Hubzilla and Zot allow cloning, they also recognize cloned accounts. They know these two accounts are the same account.

But, Mastodon and ActivityPub do not support this feature. ActivityPub thinks that [email protected] and [email protected] are two different accounts. So if you follow both of them, you get the message twice in Mastodon.

For ActivityPub followers, we have to send out copies of posts from all of the clones, because some people may follow one clone, and other people may follow another. This ensures that someone following an old but active clone can still get our posts. We are assuming that most people would follow only one clone and perhaps don't even know about the other clones.

Hubzilla does announce that these are cloned channels (aliases) via Webfinger, but to my knowledge, no ActivityPub platform checks this and none use this information to suppress duplicate posts.

It really isn't about blame. It is just that Mastodon does not support nomadic identity, so you get weird things like this.

Raphael Lullis

@scott but shouldn't it be possible for Hubzilla to track who-follows-what, and only send the messages accordingly?

I surely don't follow you from your other identities. Maybe Hubzilla could have a system that sends messages only with as:Public by default, and only addresses the responder with the "in reply to" field by the follower's known aliases?

Scott M. Stolz

@Raphael Lullis Hubzilla does track who follows what. If you follow [email protected], only loves.tech will send you the message. The other clones do not send a message to you.

But Mastodon has a shared inbox. And there are other people on your server who may be following the other clones.

So what might be happening is that you follow [email protected] so loves.tech sends you a copy of the post. But someone else is following [email protected] so authorship.studio is sending them a copy of the post. They both go into Mastodon's shared inbox on your server, one addressed to you, one addressed to them.

Mastodon does not realize that this is the same post and same user, so it adds both copies of the post to the conversation.

At least, that is my theory on what is happening.

Scott M. Stolz

@Raphael Lullis Hubzilla does track who follows what. If you follow [email protected], only loves.tech will send you the message. The other clones do not send a message to you.

But Mastodon has a shared inbox. And there are other people on your server who may be following the other clones.

So what might be happening is that you follow [email protected] so loves.tech sends you a copy of the post. But someone else is following [email protected] so authorship.studio is sending them a copy of the post. They both go into Mastodon's shared inbox on your server, one addressed to you, one addressed to them.

Mastodon does not realize that this is the same post and same user, so it adds both copies of the post to the conversation.

At least, that is my theory on what is happening.

Raphael Lullis

@scott

I'm getting really closed to get nerd-sniped by this and set up a database viewer to see who-is-following-who on the server.

Scott M. Stolz

@Raphael Lullis One of the reasons why they set it up this way is because of how Mastodon notifications work. If you follow [email protected] but [email protected] sends you the message, Mastodon will not notify you of the message. It has to be sent from the account and domain being followed.

So if two people follow different clones, the only way Mastodon users will be notified is if we send it from both clones.

But, apparently, this means in conversation view, there would be two copies of the post. Or more than two in this case.

Basically, that is the issue. We send on copy, Mastodon notifications don't work. We send multiple copies, Mastodon notifications work, but you may get duplicate posts.

Scott M. Stolz

@Raphael Lullis One of the reasons why they set it up this way is because of how Mastodon notifications work. If you follow [email protected] but [email protected] sends you the message, Mastodon will not notify you of the message. It has to be sent from the account and domain being followed.

So if two people follow different clones, the only way Mastodon users will be notified is if we send it from both clones.

But, apparently, this means in conversation view, there would be two copies of the post. Or more than two in this case.

Basically, that is the issue. We send on copy, Mastodon notifications don't work. We send multiple copies, Mastodon notifications work, but you may get duplicate posts.

Marshall Sutherland

If Alice and Bob are on different servers, obviously any messages between them would be on both servers, but if they are on the same server, do they share the messages in the database or do they each have their own copy? If the messages are shared, it seems like this could make fewer, larger servers more space-efficient than more, smaller servers.

Another scalability question comes to mind... Can you have multiple web servers for the same Hubzilla database (not a single database server with multiple Hubzilla databases)?

Scott M. Stolz

@Marshall Sutherland A single large server would be more efficient, in the terms of resources, however, there are other considerations.

First, we have the goal of decentralizing social media, so multiple smaller servers run by different administrators achieves that goal.

This encourages the creation of unique communities with their own cultures and community standards. This avoids the problem of moderators unfamiliar with another culture moderating content unfairly due to a cultural or context misunderstanding. It also allows for complete independence when someone runs their own server.

Secondly, it is also a matter of delegating the workload. Each server administrator is responsible for moderation on their own server.

Thirdly, there are different rules for social media and web hosting.

For users on our servers, they fall under social media rules, which requires moderation of content and compliance of various privacy and social media and age restriction laws.

But for web hosting, none of those rules apply to us, the web host. If someone complains, we'll investigate, but the server administrator is responsible for moderating content, not us.
Putting people on their own servers with their own domain name reduces our workload because it delegates most moderation and compliance work to the server administrator.

Another consideration is that costs for smaller servers are more predictable than costs for giant servers.

Although it would be more efficient to have a very large server with millions of people, for legal, technical, and philosophical reasons, we want to pursue decentralization.

Scott M. Stolz

@Marshall Sutherland A single large server would be more efficient, in the terms of resources, however, there are other considerations.

First, we have the goal of decentralizing social media, so multiple smaller servers run by different administrators achieves that goal.

This encourages the creation of unique communities with their own cultures and community standards. This avoids the problem of moderators unfamiliar with another culture moderating content unfairly due to a cultural or context misunderstanding. It also allows for complete independence when someone runs their own server.

Secondly, it is also a matter of delegating the workload. Each server administrator is responsible for moderation on their own server.

Thirdly, there are different rules for social media and web hosting.

For users on our servers, they fall under social media rules, which requires moderation of content and compliance of various privacy and social media and age restriction laws.

But for web hosting, none of those rules apply to us, the web host. If someone complains, we'll investigate, but the server administrator is responsible for moderating content, not us.
Putting people on their own servers with their own domain name reduces our workload because it delegates most moderation and compliance work to the server administrator.

Another consideration is that costs for smaller servers are more predictable than costs for giant servers.

Although it would be more efficient to have a very large server with millions of people, for legal, technical, and philosophical reasons, we want to pursue decentralization.

Marshall Sutherland

Although it would be more efficient to have a very large server with millions of people, for legal, technical, and philosophical reasons, we want to pursue decentralization.

I certainly wasn't suggesting massive centralization, merely wondering about the trade-offs since you will be in the position to actually have choices that most of us will never need to consider.

I wish you luck and look forward to hearing more about how this project goes.

Scott M. Stolz

@Marshall Sutherland It is more efficient to have one server, especially from a maintenance standpoint. Instead of having thousands of servers each with their own copy of the code and database, you have one server's codebase. The downside of that is if you break something, it affects a lot more people.

Scott M. Stolz

@Marshall Sutherland It is more efficient to have one server, especially from a maintenance standpoint. Instead of having thousands of servers each with their own copy of the code and database, you have one server's codebase. The downside of that is if you break something, it affects a lot more people.

Marshall Sutherland

The downside of that is if you break something, it affects a lot more people.

Now you've got me thinking about redundancy both in the web server and the database server. Not that I would ever have reason to go that far with my (for practical purposes) single user hub.

Scott M. Stolz

@Marshall Sutherland Since Hubzilla has the ability the clone channels, running a second server provides redundancy for the cloned channels. Combine that with server backups, and that is sufficient for most people.

But if you become so big that tens or hundreds of thousands of people depend on you, running a primary and secondary database server would be a good idea. The secondary is instantly synced with the first and can serve as a fall over in case of failure of the primary. Since it is immediately synced, you shouldn't lose data if you have to restore from a periodic backup.

If implemented at scale, this would increase server costs considerably for very large servers.