Part of my frustration with #ActivityPub and one of the things I find baffling giving everything else in it: the lack of tools for backpressure.

Hrefna (DHC)

Backpressure is fundamental in building reliable distributed systems (c.f., Notes on Distributed Systems for Young Bloods). From a C2S perspective I get why it wouldn't need to be specified, but from a S2S _federated_ protocol perspective its absence is frustrating.

All that it says is to take care not to overwhelm _others_ and a bit on rate limits

julian

@[email protected] What needs to be specified at the protocol level?

If I received a 429 I'd naturally assume that I should put the request back in the queue for retry, but I can see how other implementations might not support that.

Hrefna (DHC)

What could it say instead?

It could give handling considerations with 429s.

It could specify needing to have an abandonment time after which point you assume an actor is "dead."

RFC 6120 (XMPP), when talking about clients, has the following requirement

julian

@[email protected] There are two obvious interests at odds here:

The desire to have content propagate as quickly as possible (which necessitates sending out activities as quickly as possible)
The desire to not overwhelm a server (which would mean some form of a retry queue, potentially with exponential backoff)

I personally think guidance would be appreciated (i.e. abandonment time), but at the end of the day how an implementor handles it is their own business.

Hrefna (DHC)

@julian It Depends, but even just saying something like:

"A receiving server MAY return a 429 on a request. If this happens or if the server cannot be reached the sending server SHOULD retry with exponential backoff. They SHOULD have a period of time after which they will stop trying and SHOULD have a period of time after which they will consider the actor 'dead' and stop sending them messages."

Instead you get things like these:

Jenniferplusplus

@hrefna yes.

But I also want state convergence. No matter what, different hosts are going to wind up with different views of the state of the graph. Why are there no mechanisms to detect that and enable convergence?

Hrefna (DHC)

@jenniferplusplus Yes. Yes. A million times yes.

There's no _sync_ capability despite that we're basically building a distributed database where we are trying to have a sync'd state.

So if you fall off for a few hours what is expected to happen?

Everyone's favorite thing from C++: Unspecified.

julian

@[email protected] @[email protected] well, that's because we're all building apps around a protocol whose largest implementor doesn't specify context, ever.

So you end up receiving disjoint objects whose only relation to other objects is inReplyTo, and you have to hope you can find a match.

That automatically precludes the ability to develop sync/backfill.

Jenniferplusplus

@julian @hrefna That might help, but it wouldn't solve anything on its own. I want a mechanism to determine if two party's view of a collection are equivalent. I want to be able to query if an object is contained in a collection without enumerating the whole collection. I want a mechanism to request re-delivery or maybe failed delivery notifications.

There's so much stuff that's just not even considered. Not even at the level of "you should do authentication, somehow"

Genders: ♾️, 🟪⬛🟩; Soni L.

@hrefna @jenniferplusplus the more y'all talk about AP the more we want to stay away from it heh.

if it's worth anything: thanks for that.

Hrefna (DHC)

@julian But that _could_ be specified at the protocol level. It isn't, but it is kind of weird

There are entire sections on it for Dynamo (first image), Tiara discusses it extensively in the posted snippet (second image) and in multiple other sections, and it is covered as something to address in Requirements for Signaling Protocols (RFC 3726).

These all have multiple sections addressing different parts of this problem and solve it with different levels of flexibility, but they all address it.

Jenniferplusplus

@SoniEx2 @hrefna These things are all fixable problems, it "just" takes coordination between a few dozen part time volunteer implementers

I tell myself I'm going to just do it unilaterally and then help everyone else catch up. I'm not sure if I'm lying to myself or not.

Hrefna (DHC)

@jenniferplusplus

That's a mood.

@SoniEx2

julian

@[email protected]:

I want a mechanism to determine if two party's view of a collection are equivalent. I want to be able to query if an object is contained in a collection without enumerating the whole collection. I want a mechanism to request re-delivery or maybe failed delivery notifications.

For what it's worth, these are all goals that would align well with the interests of a broad set of implementors. I make a little noise now and again about the SWICG forum task force, but directly addressing some of these concerns would definitely be on our radar.

Current focus is on aligning on a common object type for higher level collections (the context), but I'm thinking that Collection (ordered, ideally) would be the best fit and also allow for future use cases like you mentioned earlier.

You have the ideas, we might be able to supply the coordination...

cc @[email protected] @[email protected]

Jeffrey Hulten

@julian @hrefna

There are protocol things that can be done as well:

S(a): I have something for you.
S(b): Not now. Try in x Minutes.

Or

S(a): I have something for you.
S(b): Not now.
S(a) waits a term of their own choosing.
S(a): I have something for you.
S(b): Not now.
S(a): Let me know when you have a sec.
S(b) waiting until load subsides.
S(b): I am ready for you
S(a): I have something for you.
S(b): Thanks.

Jenniferplusplus

@julian
Yeah, I've been trying to keep up with it. I've generally stepped back from the swicg lately, although you seem to have the most functional wing of that org. I've imagined reengaging in the future, when I can lead with an implementation. Otherwise it's just more theorycrafting, and there's too much of that already.

If maintainers of real implementations in the forasphere are soliciting input, that might be another story.

@hrefna @SoniEx2

Ben Pate 🤘🏻

@jenniferplusplus @hrefna I love this thread and the fact that you’re all thinking about these things. Not to muddy the waters, but...

This sounds a bit like FEP-5624 in that the original poster may want to to choose which parts of a thread to sync with others.

I think the original poster could be considered the primary source of truth, which might also minimize the amount of “gossip protocol” required to sync this distributed db.

Jenniferplusplus

@benpate @hrefna They're overlapping concerns, yes

Mike Macgirvin 🖥️

Yeah I've had a number of times where I wanted to query a collection - but hopefully something a bit more lightweight than a full blown query language.

And in the Nomad protocol, we've got delivery notifications. It's critical to finding out what happened to something in a decentralised communication system. It's 2024. Vanishing into space without any trace is not an option.

julian

@[email protected] do you know any other implementors who expose a collection (even if not defined by context)?