Ozone, Bluesky's stackable moderation system is up and open-sourced. https://bsky.social/about/blog/03-12-2024-stackable-moderationI think it's interesting in obvious ways and risky in some less obvious ones (that have less to do with "O NO BILLIONAIRES" ...

Tim Chambers

@kissane Yes, I did know they centralize base moderation is halved by bluesky staff — but not sure at which part of the bluesky / AT ecosystem…can’t be at PDS servers as those swap out, as do the crawler indexers, so where does centralized non-optional moderation occur? Maybe at the app layer?

Erin Kissane

@tchambers For just Bluesky-the-app, as opposed to other potential future ATP apps?

Looking at the whitepaper plus Paul's most recent comments, I *think* it's labelers hardwired into the official client app(s) and actioned at the App View layer plus takedowns at the Bluesky-run relay and Bluesky-run PDSes, right?

Caveat that my understanding is highly fragmented. (As are their docs and explanations, bless them.)

Tim Chambers

@kissane I *THINK* you and I see it the same way, but I personally am still feeling like I have a 20 percent or more chance of being wrong. Wish they would clearly state that.

Side question related to that: So if Gab.com built it's own full AT Protocol coalition of services (It's own app, own labler, own PDS's etc) could the BlueSky service "defederate" entirely from GabSky?

If so, would that also be done at the BlueSky service's App?

jonny (good kind)

@tchambers
@kissane
"Can you opt in to labeling" is the whole tension of labeling for content moderation - the answer necessarily has to be "no" at some level or else it wouldnt work, ppl posting harmful shit dont want it to be labeled as harmful, but then it becomes a vector for abuse as eg. The gab system eg. labels all queer people for targeting.

They have a view on abuse that "if you dont see it, then its not abuse" - so eg. here you dont see labels from labelers you dont subscribe to: https://github.com/bluesky-social/atproto/blob/a0625ba40f3d66528488a18f6d32854d9165e191/packages/api/src/moderation/decision.ts#L262

But it does look like any labeling service can label any event - there isnt a good way of making that opt in on the protocol. Im going to set up a test instance later and try and label myself and see what happens in my PDS.

The risk is that identity is so cheap that there doesnt need to be a stable "gab" labeling service - if it gets blocked at an infra level by atproto, cool, signal through a side channel to your followers youre minting a new DID and block successfully evaded. So it is effectively impossible to stop labels as designed from being used as abuse vectors.

I raised this last June and Paul did respond once and from a first look it doesnt seem like any changes were made https://github.com/bluesky-social/proposals/issues/19

Jon

Right. I don't think they've fully thought through the implications of the underlying design. I asked @bnewbold over there if they had done threat modeling but didn't get a response, oh well.

@tchambers @kissane that's also how I see it with Bluesky-run relays and PDS's, but they've also said that it's only illegal content and spam. Masnick's paper talked about *not* removing Alex Jones at this level. So it's not clear that Gab would have to have their own PDSs or relay. (1/N)

@jonny

Erin Kissane

@jonny Yep, shared moderation lists are absolutely a potential attack vector. It's one of the most bullet-biting hard tradeoffs of locked-open + decentralized shared mods layer.

Blocking isn't a strong way to prevent regular user lists from being used adversarially either—I see your issue suggesting otherwise, but I have near-zero faith in blocking as a mitigation for the kind of brigading you describe.

(con't)

Erin Kissane

@jonny I guess for me, the question becomes whether other parts of their model can mitigate the harms concentrated by adversarial use of lists and shared mods.

(My current suspicion is that without strong place boundaries, it's always going to be an arms race, but we also haven't really seen "arms race" on this exact configuration of semi-decentralized services yet and there are a lot of variables.)

jonny (good kind)

@kissane
Totally agree. And the labels are a different vector than lists alone too since they are applied to the post/account itself, rather than the post/account being indexed and etc. Also agree that blocking is at best a reactive measure, even if identity had more friction. I think youre right on diagnosing lack of place as the core of it, and its a really nasty downside of "frictionless all-to-all platform" as design goal. Fedi fiefdoms are not great, but having no sense of place doesnt feel like an alternative either.

Erin Kissane

@jdp23 @jonny My sense is that they've thought it through a lot and just come to different conclusions than most fedi people have, fwiw.

(Relatedly, I hope we'll see more docs and explanations now that this round of huge releases is out the door, because I always want that.)

Erin Kissane

@jdp23 @jonny

On the other note, I think the "illegal content and network abuse only" refers to the moderation that extends beyond Bluesky-the-reference-app/platform, in a larger future system.

Bluesky as a platform—which is what I *think* Tim and I were discussing—does takedowns and deletions for lots of things that don't rise to that level, and the team talks about that in their moderation report and other places. (I know you know this, I just want to try to keep the thread clear-ish.)

Erin Kissane

@jonny Let us not even begin to speak of Nostr ‍

Emelia 👸🏻

@kissane @jonny I think on labelling it won't actually make a possible "list of targets" since it's never "filter in this stuff I don't follow" but "filter out this stuff I might see"

So because it's subtractive, you don't know the content you don't know, as an end user. Yeah, the label operator would have a list of accounts / hashtags / etc to monitor, but that'd be internal information to them.

Jenniferplusplus

@thisismissem @kissane @jonny "Feed generators" get those labels and can opt posts in based on that.

Jon

@kissane They've certainly thought it through a lot more than ActivityPub and Mastodon did at the equivalent stage! Bryan said they've done red-teaming, perhaps that included threat modeling as well. If so it'd be a first, no social network that I know of has ever done this early in their lifecycle (or for that matter later). Time will tell.

@jonny

Caspar C. Mierau

@kissane @joshwayne @mergesort BlueSky - as a commercial company that is going to earn money from what they build - makes it obvious that they consider a central moderation instance as bad because it is like a "Supreme Court". Which is by itself already a strange way of criticm. What they don't say here is: moderation costs money. Yes, it does. And social network platform hate paying people for this hard job - which is necessary and it is just fair that they do their job on a platform where they also earn money from users generating content. The result is an ecosystem where it is ok to be harassed as you are free to move to another instance. This is just making a bad system worse and selling it as a new technical feature. If BlueSky would finally agree on it's repsonsibility, building a well paid moderation team and then introduce "composable" moderation, yes, that would be fine. As it would be an addon. But this is cost reduction by technical implementation.

When Jack initially announced BlueSky the first (!) point he made was the following:

»First, we’re facing entirely new challenges centralized solutions are struggling to meet. For instance, centralized enforcement of global policy to address abuse and misleading information is unlikely to scale over the long-term without placing far too much burden on people.«

So he argues that being a responsible company that is obliged to international laws - and it's users - is a "burden on people". Well: the people here is the stakeholders of billion dollar platforms. And that is what BlueSky is the solution, too.

I would have loved to see a blue sky in BlueSky but besides looking nice I mainly see a platform that aims towards deregulation.

https://bsky.social/about/blog/4-13-2023-moderation
https://twitter.com/jack/status/1204766082206011393

Erin Kissane

@leitmedium @joshwayne @mergesort So, Bluesky has a large and active moderation team: They do platform-style moderation transparency reporting. Paid humans review all reports. That’s what’s actually happening. (Also there are no “instances” to move between.)

I have zero problem with critique of their model, but a lot of the discussion is remarkably decoupled from actual events.

Erin Kissane

@leitmedium @joshwayne @mergesort The usual next step is to move the goalposts and say “Ah but they won’t moderate in the future and no one can prove they will, because Jack!”

(Which, sure! Maybe they kill off all their central moderation, maybe it’s all a ruse, we can make things up forever. But I have low faith about my ability to parse out futures from inferred intent and even less about most other people’s, so it’s not a mode I find fruitful.)

Vesipeto Vetehinen

@[email protected] @[email protected] @[email protected] @[email protected] it's not just Jack though. Their documentation reflects this philosophy in parts too. Maybe they should come out and say it isn't their goal anymore if that is the case?

Erin Kissane

@vetehinen @leitmedium @joshwayne @mergesort What I’m saying kinda always is that I think it’s more useful to look at *what is actually happening* than to read philosophical statements and try to work out what systems they would have resulted in if they were building on a frictionless plane.

So I really value “What is the actual system and what does it *do*” as the soundest basis for trying to understand the (very) near future.

Caspar C. Mierau

@kissane @joshwayne @mergesort Well, I quoted official statements and documentation, which is what I was studying for quite a while now - in order to understand what BlueSky wants to achieve in the future. If this is not the right type of discussion I am sorry for interrupting. I did not want to be alerting here or shout "Jack!!". All fine.