So when you share something on mastodon, even if no one clicks a link, it is going to get Ο(n) connections from n servers fetching for previews.
-
@hrefna aaaaargh. If the problem is “we need a consensus on how to behave with each other esp because this makes everything more efficient and better”, I worry for flexibility but am on board as much as possible
-
@hrefna Sounds like the classic centralisation v/s decentralisation dilemma. What's the way forward though?
-
-
@hrefna That's one of the reasons I've been interested in bringing P2P protocols into the mix. Instead of everyone fetching from one server it could be spread over a swarm of nodes with peers resharing to others that are interested instead of one server getting overloaded.
-
Mark Shane Haydenreplied to Hrefna (DHC) last edited by
@hrefna In my experience running small fedi instances from the garage in my back yard for the past 7 years I would say the problem is real if a bit overstated. A consumer grade gigabit fibre service handles this use case surprisingly well.
That said, Mastodon especially but fedi apps in general are quite inefficiently designed, and the single biggest challenge probably is the preview card issue. There are straight forward ways to mitigate the "fedi-hug" problem (which is actually not anything to do with ActivityPub) at the application level. Preview cards could be fetched "on demand" (so could post threads from origin servers, which would be nice and servers could just, you know, only fetch preview cards only for local posts then federate the data amongst themselves instead of flattening the poor potato servers serving the links we share.
Nonetheless I figure that at comparable scales and levels of distribution that ActivityPub is potentially quite a bit more efficient on resources than AT protocol.
-
Jenniferplusplusreplied to Hrefna (DHC) last edited by
@hrefna to be entirely fair to mastodon and the AP designers, the mastodon opengraph stampede is a pretty ridiculous problem to have.
For one thing, loading html for publicly viewable web pages should be easy. Don't build systems where that's not true.
For another, parsing opengraph and other linked data out of full on html pages intended for humans to view in web browsers is so wasteful. Where's the text/ld+html content type? Or some other mechanism to request just the html header?
-
@MxVerda Really we could do it with a change in the protocol, not even a difficult set of changes in the protocol, but under the current AP system…
Yeah.
-
Jenniferplusplusreplied to Jenniferplusplus last edited by
@hrefna Not that it changes your larger point, which is still correct.
-
@hrefna For folks who might be interested, there is some ongoing conversation about this with proposed solutions:
Mastodon link previews draft.md
GitHub Gist: instantly share code, notes, and snippets.
Gist (gist.github.com)
-
Hrefna (DHC)replied to unexpectedteapot last edited by
Yeah, more centralized relays, particularly "folding up" relays (a la atproto), are one strategy.
Another is attenuation, batching, and liveliness checking. Reducing the amount of requests.
Also there's various strategies for passing additional information (e.g., a preview) around with various tradeoffs.
Emphasizing server-to-server communication instead of client-to-client communication. Combine with batching and preloading and you're 80% of the way there.