Ozone, Bluesky's stackable moderation system is up and open-sourced. https://bsky.social/about/blog/03-12-2024-stackable-moderationI think it's interesting in obvious ways and risky in some less obvious ones (that have less to do with "O NO BILLIONAIRES" ...
-
Erin Kissanereplied to Caspar C. Mierau on last edited by
@leitmedium @joshwayne @mergesort Nah, that second post was me trying to get ahead of the thread's direction, not aimed at you specifically.
I think it's great to look at stated philosophy, just not in isolation, because…
>If BlueSky would finally agree on it's repsonsibility, building a well paid moderation team and then introduce "composable" moderation, yes, that would be fine.
This is actually what they've done:
Bluesky 2023 Moderation Report - Bluesky
We have hired and trained a full-time team of moderators, launched and iterated on several community and individual moderation features, developed and refined policies both public and internal, designed and redesigned product features to reduce abuse, and built several infrastructure components from scratch to support our Trust and Safety work.
Bluesky (bsky.social)
-
@leitmedium @joshwayne @mergesort
They've also committed to doing kill-switch moderation for illegal content and network abuse (beyond-Bluesky-the-App View + official clients) across everything their relays and PDSes touch on the future ATP network. (This is a lot more central modding than happens on fedi, but still upsets a lot of people because it's less than they want, which is interesting to me.)
-
Vesipeto Vetehinenreplied to Erin Kissane on last edited by
@[email protected] @[email protected] @[email protected] @[email protected] I appreciate the distinction but I also feel like there are so many examples of getting burned when a company starts with something good and switches it up later that we should probably give at least some weight to what they are saying their intentions are too.
-
Erin Kissanereplied to Vesipeto Vetehinen on last edited by
@vetehinen @leitmedium @joshwayne @mergesort Totally agreement. I spoke too strongly—I think it’s good and appropriate to look at the cloud of philosophical stuff, but also to maintain a healthy skepticism about how that translates into the actual, and especially to put what actually happens at the center. (Because I am a The Purpose of a System Is What It Does person.)
-
Hmm, here's an example that their red-teaming doesn't appear to have considered. There doesn't seem to be any way to prevent an account from following a labeler and reporting posts -- I just tested and even if the labeler's blocked the account it can still subscribe and report. So what's to prevent bad actors from bombarding a labloer with (valid) reports of traumatizing images that Bluesky has hidden by default?
-
-
jonny (good kind)replied to Erin Kissane on last edited by
-
@[email protected] as @[email protected] mentions, it seems threadiverse type apps are going in that direction.
NodeBB (working on AP integration) is also built around being an actual community, with separated local and federated posts. They can mix, of course, but there's an explicit sense of locality that is intentionally missing in Mastodon.
-
The user experience is to submit directly to the labelers, not sure how it works behind the scenes. Therre's some discussion from a Bluesky dev at https://bsky.app/profile/jacob.gold/post/3knqwjlvhu22q But blocks are public on bluesky so no matter what they layering is they *could* be checked.
And yeah, getting a new DID is also an attack. If they threat modeled this, they either missed some very obvious stuff or skipped the all-important "implement mitigations" step
-
And @kissane it also seems to me it would be a good issue to file. I tagged a couple of their devs at https://bsky.app/profile/jdp23.bsky.social/post/3knugvlg37k2r so it'll be interesting to see if they agree.
-
yeah in my earlier reply to Bryan I suggested that whatever they could publish about it would be great. A good red team would do threat modeling and involve the whole engineering team and (even more importantly) the likely targets of abuse. But that doesn't always happen.
-
@jdp23 @jonny
@tchambers To come back to this node in the thread, interesting notes from Bryan on a piece of the discussion. https://staging.bsky.app/profile/bnewbold.net/post/3knyw7ydofu26My sense is there is a lot of iceberg under the water that hasn’t been documented or fully implemented yet, which is one of the reasons I’m in wait-and-see mode about a lot of this stuff.
-
@jdp23 @jonny This is kind of a meta comment, but I would myself be hesitant to publish or even discuss threat-modeling findings outside a core team.
(If the gaps and rough edges in the labeler system don’t get filled/filed in the coming weeks, that would suggest to me that they missed things or triaged them down, or decided to accept a new risk surface and mitigate elsewhere—but from the outside, seems like it won’t be especially clear which.)
-
-
Thanks for the link, useful info. Agreed that it's a work in progress so wait-and-see mode makes sense. @tchambers @kissane @jonny
-
@jdp23 @tchambers @kissane
i'm maybe a little more cynical, but the emerging picture seems pretty clear from the decisions that have been made and fixed in the protocol so far. i'm sorry i'm always doomposting at a million characters when i talk about this but this whole thing is fucking frustrating to meso the answer to "does bsky retain top-level control over content on the whole network" is a resounding "yes." that's necessarily so - the default relay is not optional for a number of reasons, and to avoid the baseline liability problems the protocol was designed to defray risk around they need to at a minimum be responsive to DMCA/CSAM/etc. I'll set that aside for tonight, there are obvious harms to retaining centralized power at the top, that's the whole root of the problem, blah blah blah we all know.
that means they will be forced to take a "minimal risk" stance - we take down at an "infrastructure level" the very minimum we have to, and then the layered moderation is strictly additive on that. sounds good, right? the problem is that the key safety features are infrastructure-level only - for everything below "we will get sued into bankruptcy for this content," you're on your own. the misuse of protocol features for abuse is something that you cannot prevent without control of the relay. identity is too cheap and the protocol is too public.
think about how the lions share of the day to day moderation is to be managed - sure sure ~ a l g o r i t h m s ~, but when it comes to actually keeping specific groups of people safe, these will be real human moderators like everywhere else. On the fedi moderation is already tenuous and undercompensated, but at least there is a reciprocal sense of "i watch out for you because you're on the same instance as me and we have shared goals. if i fuck up, you leave, and that has material consequences for the longevity of the instance." What does that look like when moderation comes from some exogenous service that you subscribe to? Is the entirety of keeping racists and transmisia at bay going to fall to an ever shifting group of labeling services that have no connection to the people who rely on them? Who in thee everloving fuck would volunteer to moderate all nazi shit on a protocol where all moderation activities are globally public for everyone. Who cares if they talk about how to keep yourself safe while moderating in the docs, have they ever seen how doxxing works? At the point when those labelers can't even take direct action against content, but instead can only make it invisible to the person that it's affecting, multiply the lack of motivation with lack of power and you have near-guaranteed capacity for harm.
If the goal was to "make federation simple," they've instead exploded the complexity, where instead of needing to choose an instance (insurmountably difficult!) where a few people keep watch on fediblock, now everyone needs to always keep up to date on the combinatoric complexity of what relays i'm on, what feeds they serve, which labeling services are good, and so on. It's a hostage situation that presents itself as being approachable by being "sign up on this single website" but then to survive on it you can't help but engage with the entire stack.
I disagree wholeheartedly that this is "deep in wonky tech/policy intersection" - this is the core of the protocol. this is the key determinant of power distribution on the network. everything is fun and cool when you are in beta and can learn lessons and say sorry every time there's a problem, but when the consequences of the protocol design drive you to consolidate power in the relay out of necessity and abandon vulnerable people with a "glhf, find a labeling service," we are very much in "everyone should care" territory. the instinct that it should be slow rolled and carefully messaged rather than developed in public with lots of input speaks volumes.
i'll stop. i don't like what i see. i want this protocol to work, bc as far as harm reduction goes of getting ppl off of X it's definitely doing better because it replicates the vibes of twitter better (for good or for ill). but this is going to explode over and over again, and who knows there might be some miraculous hack that makes it all work again, but it's going to get people hurt and it's on net a huge step back for a healthy information ecosystem - people get burned by traditional platforms, fine it's exhausting to switch platforms if we do it at all but at least we can sorta grasp where the harm comes from. people get burned by something that calls itself "federated," that's supposed to be "the good one," and you're going to burn through all the goodwill and curiosity of everyone that could be committing themselves to something that isn't a time bomb - and i'm not saying that's fedi, which to me has always been a transitional medium, but it could be something with half a chance of survival.
-
Erin Kissanereplied to jonny (good kind) on last edited by [email protected]
@jonny @jdp23 @tchambers I won't be able to catch up with this until probably tomorrow, so I'll just note for clarity that my own "wait" stance isn't a recommendation, just a measure of my own sense of certainty, which remains too low. (I think the pace of my work is unusual here, but it is what it is.)
