Periodically I see the absolute monster that is the Akkoma timeline query
-
Jenniferplusplusreplied to Erin 💽✨ last edited by
@erincandescent the tsdb that powers timelines isn't the canonical source of truth for the app. It only handles timelines and notifications, and everything gets denormalized at write time. Single origin scenarios like populating an outbox use the canonical db.
-
Jenniferplusplusreplied to Jenniferplusplus last edited by
@erincandescent There are a few reasons I don't want to precompute explicit individual timelines for each user. A big one is that I want inactive users to cost as close to zero as possible. So with the exception of cases where the inactive user is the only follower of some remote actor, zero compute is spent on their timelines.
-
λ Natty :butterfly_:​:neofox_lesbian:replied to Erin 💽✨ last edited by
@[email protected] scared to ask where's the part that populates it with all the data the frontend needs
-
Jenniferplusplusreplied to Jenniferplusplus last edited by
@erincandescent You're right that following more people makes for larger gathers. But it's still just a single inner join at query time, because of the denormalization.
-
Erin 💽✨replied to λ Natty :butterfly_:​:neofox_lesbian: last edited by
@natty the entire thing is
-
@jenniferplusplus I think you're falling pray to corpo-think where the desired outcome for your service is an eternally increasing number of users.
In my opinion that's a wrong way to design services for communities of the fediverse.
If you limit the number then you need to care less about the inactive users because you don't need to worry about eternally increasing compute time.
-
@mariusor @jenniferplusplus even on small instances you need to think about scaling because social media is just an enormous mount of traffic; people way underestimate this
-
Jenniferplusplusreplied to Erin 💽✨ last edited by
@erincandescent @mariusor
This.But even if that didn't have an effect, supporting large instances is an explicit goal for the project. There's like 5 billion people who use social media, and approximately all of them use "corpo" services, which I think we generally agree is bad. So if that is ever to change, even community services will need to operate at internet scale. The vast majority of people cannot and will not operate their own communication infrastructure.
-
@jenniferplusplus good luck with that (not in a dismissive way) but building a billion users software stack as one person is pure hubris. (I'll be glad to eat my words though)
-
I think that network traffic should not equate 1-to-1 with the compute power to build a timeline for a user because the two tasks should be somewhat independent: ingestion of activities (the traffic that you mentioned) -> server, building timeline (based on normalized data) -> the client with which the user interacts with the server.
-
@mariusor I hope and intend that this will not be a 1 woman operation forever. But I also don't expect that the social web would become a monoculture. I really just mean that some very large nodes must necessarily exist, and I plan for letterbook to support that.