@phnt
-
@feld
> I spent so many hours fighting with IPFS I think it's a dead end.
Yeah, that's my conclusion, too. It should be great but is not even good. FSE had the start of a patch but I didn't get too far actually integrating it. (
https://git.freespeechextremist.com/gitweb/?p=fse;a=commit;h=2dec0a138612eebe79f757b5bc04810b81d65951 )
> Nobody working on Kubo has any interest in fixing the storage scalability problems it seems.
I think it's a little past them, if I'm being honest. They could probably get away with just integrating tkrzw.
> To use it for fedi we'd really want the garbage collection to work and not spin 100% CPU and IO for hours but that's what it does when you get enough data in there 🤬
Yeah, you wanna see a real horror, look at the directory where it stores the data. I don't know if it's still doing that but it was storing different permutations of the same hash as different files last I checked. So the Qm$whatever or the bafy$whatever, those were just stored as separate files instead of normalizing the hash.
8MB is too large for you to get fortuitous collisions, it's not a very good chunk size for transfer, just the entire thing is like a parody. -
@p Unfamiliar with tkrzw; I was thinking they should build on LMDB.
-
find you on :butterfedy1: fediversereplied to feld last edited by
@feld @p @silverpill @jeffcliff ok, very interesting to read these thoughts on the degraded state of IPFS and how unfit it currently is for use with fedi (in browser as an add-on and as a server).
it fits with my thesis that foss on m$github is getting slowly compromised, by the shell accounts that the corporate state have on that weaponized platform.... a controlled platform, at best, and a platform that deanonymizes devs, which is a way to control them at the very least.
Id consider going back to 2021 and looking at their codebase then, and comparing it to today, there were some very prominent independent media folk talking about IPFS back then and so i suspect that THAT is when the controlled demolition may have started. My understanding is once the corporates have their sights on a codebase they force the programmers to accept bad tradeoffs. Trade-offs exist everywhere, and permutations of tradeoffs can be chosen ad-infinitum. my learned experience is that foss on mshithub tends to not improve in ways that would allow it to compete with corposhit, especially in the #ux department.
to accelerate :butterfedyC: fediverse centric adaptations, just over a week ago i wrote about having #ipfs on another repo and i now, having read both your comments i think it may need to indeed be a fork. a week ago i suggested the acronym DCN (decentralised content network) a tongue-in-cheek reference/reversal of CDN....
so yeah if anyone wants to run with that idea, u have my blessing
-
silverpillreplied to find you on :butterfedy1: fediverse last edited by
@frogzone @jeffcliff @p @feld Performance issues with IPFS were there from the beginning. Many hoped that Protocol Labs will fix them but that didn't happen, perhaps due to the inherent limitations of the protocol, or due to incompetence.
In any case, this is not a "controlled demolition". There was a shift to NFTs and then Filecoin around 2020, but I don't think that affected IPFS in a negative way -
pistoleroreplied to find you on :butterfedy1: fediverse last edited by@frogzone @silverpill @jeffcliff @feld
> foss on mshithub tends to not improve in ways that would allow it to compete with corposhit,
The mass "Rewrite it in Rust" bit tends to have the side effect of converting (A)GPL software to MIT/BSD.
> so yeah if anyone wants to run with that idea,
:ocelot: I may be a bit ahead of you on this. :revolvertan: -
find you on :butterfedy1: fediversereplied to pistolero last edited by
@p @silverpill @jeffcliff @feld >re rust and (A)GPL to BSD
I'll need to keep an eye on that>ocelot/revolver
sounds like i may need to look closer
the reason i thought we could do it with ipfs was that it already was being used in i2p, but its only being used as a proof of concept at this stage.
-
pistoleroreplied to find you on :butterfedy1: fediverse last edited by@frogzone @silverpill @jeffcliff @feld
> sounds like i may need to look closer
I explained the block splitting to nyanide's final boss a few days ago: https://fsebugoutzone.org/notice/AnGc6MOGfUsuVOsblQ . Really, the better thing to read, though, would be the Kademlia paper (attached) and the venti paper, which is at http://doc.cat-v.org/plan_9/4th_edition/papers/venti/ .
> the reason i thought we could do it with ipfs was that it already was being used in i2p, but its only being used as a proof of concept at this stage.
I hesitate to say too much about numbers but I will say that I think it would be hard to make something as slow as IPFS. You'd have to try really hard.
kpos.pdf -
silverpillreplied to pistolero last edited by [email protected]
@p @frogzone @jeffcliff @feld What is your current stance on FEP-ef61? Does it conflict with what you're doing in Revolver, or complement that?
Significant progress has been made since we last discussed it. Today there are two implementations, Mitra and Streams, and they are partially interoperating.
-
@silverpill @frogzone @jeffcliff @feld I haven't kept up with it. I have some notes but you can feel free to ignore them; I don't want to get in the way of some people doing a thing. Wherever it lands, I'll try to make sure that we can interoperate without breaking the FEP semantics if possible, but some places, it's not going to be possible. So, here are the notes (and quick read, so I may have missed things or I may have the wrong idea, because the stupid DDoS yesterday cost me a lot of time and I'm trying to catch up today):
> ap://did:example:123456/path/to/object?name=value#key-id
If it's a new protocol, I don't know why we're still doing a fragment for the key ID. I don't love JSON (anti-web) but we've got to live with it; on the other hand, I don't want to abuse JSON or URLs.
> Ed25519
I'm using RSA keys so that it's easy to port myself from Pleroma. It is entirely possible that there is something I am missing and that we shouldn't be using RSA keys; maybe key algo is overspecifying.
> /.well-known/apgateway
I like this, but I wonder if there's a way around it without hard-coding another URL.
> The client MUST specify an Accept header with the application/ld+json; profile="https://www.w3.org/ns/activitystreams" media type.
This is actually objectively wrong. I don't know why this should be done rather than using the HTTP standard interpretation of "Accept:" headers. If the client doesn't include "application/ld+json" or "*/*", then you just return a 406. The way Pleroma/Masto do these things kind of shits on RFC 2616 and I really love RFC 2616. (This is why I'm not doing the terrible sniffing algorithm to differentiate between a browser and another server on the /objects/$id URLs.) If there is a good reason, maybe, but if we're tossing out the well-defined HTTP semantics that HTTP clients rely on, we can't call it HTTP.
> Portable actors
This looks roughly compatible with what I am doing; there's an ID for an actor, the network protocol is used to find the actor, etc., and then to use an AP gateway, you sign a request to be canonically located on one, the owner of that domain approves or ignores, and that's the PoT for your account as far as any other AP server knows. So your /inbox and /outbox, the server with the responsibility for delivering your posts when it sees new ones, etc.
So this section looks like it's roughly compatible with that. Instead of ordered gateways, Revolver can just send one gateway.
> Portable objects
This part, I don't know how I'm going to manage this. The "attributedTo" being tied to the gateway URLs, and then those potentially changing, that's going to be a mess with content-addressed storage. If you create a new activity by signing the head of a tree that contains pointers to objects in a stream, then those objects changing (e.g., because a gateway changed) is going to make hopping computationally infeasible once you have a long enough history (which is going to be almost all of the initial users, since they will mostly be people on FSE/bae.st that follow the move to Revolver). -
>I'll try to make sure that we can interoperate without breaking the FEP semantics if possible, but some places, it's not going to be possible.
If you want to see how it all works in practice, there is @nomad
It is a Mastodon-compatible FEP-ef61 actor that is managed from a client.
>If it's a new protocol, I don't know why we're still doing a fragment for the key ID.
'ap' URLs resolve to ActivityStreams documents, and fragments are useful for pointing to specific sections within those documents. The URL you're citing is just an example, and implementers are not required to use fragments for key IDs (although that might be a good idea).
I shall change fragment ID in this example to something neutral.
>>Ed25519
>I'm using RSA keys so that it's easy to port myself from Pleroma. It is entirely possible that there is something I am missing and that we shouldn't be using RSA keys; maybe key algo is overspecifying.Nowadays security specialists advise against using RSA (1, 2), and some modern standards do not support it (for example, there is no RSA cryptosuite for Data Integity, which we use for signing objects).
Fediverse needs to migrate to something else and EdDSA seems to be the best option (according to the rough consensus among fedi devs). Also, Ed25519 key size is very small, that's pretty cool.
>>/.well-known/apgateway
>I like this, but I wonder if there's a way around it without hard-coding another URL.Yes, follow-your-nose approach is mentioned in "Discussion - Discovering locations". Existing implementations rely on well-known locations, but I think we can support follow-your-nose if needed.
>> The client MUST specify an Accept header with the application/ld+json; profile="https://www.w3.org/ns/activitystreams" media type.
>This is actually objectively wrong. I don't know why this should be done rather than using the HTTP standard interpretation of "Accept:" headers. If the client doesn't include "application/ld+json" or "*/*", then you just return a 406. The way Pleroma/Masto do these things kind of shits on RFC 2616 and I really love RFC 2616.But the FEP should say something about Accept header and media type, I don't see any way around that. The exact same requirement exists in ActivityPub spec.
How do you prefer it to be written?
>> Portable objects
>This part, I don't know how I'm going to manage this. The "attributedTo" being tied to the gateway URLs, and then those potentially changingIDs are not supposed to change. You include this special query parameter ("location hint") when you create an object, and then it stays there even if you start using a different gateway.
Maybe there's a better way to provide location hints, I don't know. Existing implementations use compatible IDs (regular 'http' URLs with 'ap' URL appended), so not much thought has been put into pure 'ap' IDs.
>that's going to be a mess with content-addressed storage
How do you process
Updateactivities then? -
@silverpill @frogzone @jeffcliff @feld
> Fediverse needs to migrate to something else and EdDSA seems to be the best option (according to the rough consensus among fedi devs). Also, Ed25519 key size is very small, that's pretty cool.
Ah, okay. Reasonable. I'll put it on the list; I had seen support for it in Honk but because Pleroma didn't have it, I just put a note in a comment and ignored it.
> But the FEP should say something about Accept header and media type, I don't see any way around that.
The Accept header mechanics don't need redefining; you just say it's served with this MIME-type, right, and then servers that handle Accept headers correctly will handle it correctly. If you're worried an informal reading will make sloppy implementations create a de facto bad standard, then you could put a note in that the client and server are expected to implement HTTP-compliant semantics.
> The exact same requirement exists in ActivityPub spec.
It's broken in the ActivityPub spec. It is not just broken in the ActivityPub spec, but it is irreparably brain-damaged. Feel free to skip these next two paragraphs; I explain why what the ActivityPub spec says can/should be ignored, but I also cannot type this without ranting.
Why call it the "Accept" header if the server isn't going to treat it that way? If the ActivityPub spec says that a byte is nine bits, it's exceeded its authority. cwebber wants to treat it like a secret handshake because cwebber is a greasy illiterate dingus and this is network software: fuckups take years to resolve.
"Accept:" is intended for content negotiation, not like a secret handshake like cwebber seems to think: the idea was initially that a server can represent content in several possible forms, clients say which forms they prefer and indicates preference, the server tries to give the client one of its preferred forms. Support for parameters *besides* preference was explicitly dropped, so the ActivityPub spec if not just broken, it fails to comply with an obsolete version. You sure as hell don't put some stupid fucking XML-style wank in there where you cram a *namespace* into the Accept header. Dear god, I haven't seen a worse idea than ` The client MUST specify an Accept header with the application/ld+json; profile="https://www.w3.org/ns/activitystreams" media type in order to retrieve the activity.` in forever.
Ahem.
> How do you prefer it to be written?
If I were writing it, I'd leave it out: you take it as read that if they are doing web server software, they should be following HTTP's semantics. Maybe a "See also RFC 9110 sections 8.3, 12, and 15.5.7" or something. You don't specify that the client MUST send some value for the Accept header, you take a well-formed HTTP message from the client, the semantics are in the spec for what you've got to do when the message is well-formed but semantically wrong, and you've not broken anyone's clients as long as you stick to the spec. So you say what the server should do in case the message is well-formed but the semantics specify something the server can't do, and you refer to the other specs for semantics and behavior of lower layers instead of re-specifying it.
Again, you know, you're the one putting in the work so take anything I say about as seriously as is warranted: here are my thoughts.
> Maybe there's a better way to provide location hints,
Well, I think, like, you embed the location hints in the user's profile but you expect that if someone has the object, then they can figure out how to get the user. You don't get an object out of context, it's part of someone's stream.
I think maybe "how the URI is constructed and how it should be interpreted semantically" should probably be a different document.
> How do you process Update activities then?
Updates update the Actor, and those are trivial, because the actor's metadata is a slot, not a stream. Other stuff, longer answer, and food is ready. -
@p @silverpill @frogzone @jeffcliff We'll do ed25519 if it's gonna actually stick. I don't think it's been revisited since the original proposal was seen and I didn't even know Honk was supporting it.
Anyone else? Mastodon? -
@p @frogzone @jeffcliff @feld Thank you, this is a useful feedback. I'll adjust the FEP
-
@feld @frogzone @jeffcliff @p Projects that implement FEP-8b32 are already using Ed25519 keys (most notable of them is probably Hubzilla).
Some other developers are also considering Ed25519, either for embedded (FEP-8b32) or HTTP signatures (RFC 9421).
>Mastodon
-
@silverpill @p @jeffcliff @frogzone looking at that mastodon pull request: they're really trying to overcomplicate everything aren't they
-
@feld @frogzone @jeffcliff @p The way of Mastodon
