Ik im preaching to the choir on here, but I do genuinely get the pushback from cloudbrained ppl that "we don't need #p2p data infra because Amazon is giving it to us for free rn, and if they stop the NIH would step in because they dont want to lose all...
-
Ik im preaching to the choir on here, but I do genuinely get the pushback from cloudbrained ppl that "we don't need #p2p data infra because Amazon is giving it to us for free rn, and if they stop the NIH would step in because they dont want to lose all the data"
Whelp, an enormous set of 12 databases of pathogen data just went poof when the funding dried up. The only way they are able to bring back a temporary static copy is because other foundations and universities stepped in, but aside from the death of the active databases, how long will that last?
P2P data infrastructure is not some radical pie in the sky Pipedream. It is the only means for making sustainable nonprofit repositories of scientific data. It's open data that's the pipedream without P2P.
Not only can it be done in a way thats compatible with existing cloud infrastructure, it would be strictly additive to its capacity, and in the case that funding dries up, can continue to exist as long as it is useful to the people affected, who can combine whatever storage and bandwidth resources they have.
The alternative of continuously needing to recreate databases, catastrophic loss, and eventually the NIH becoming totally beholden to paying increasing fractions of its tiny budget to AWS spells a grim, grim future for open data.
I've written about this before, and am working on it, but every time this happens my Cassandra complex grows greater: https://jon-e.net/infrastructure/#peer-to-peer-as-a-backbone
The db in question: https://veupathdb.org/
#VectorBase #VEUPathDB #OpenData #OpenScience
-
jonny (good kind)replied to jonny (good kind) last edited by
An archive that runs on technology from 2001 and for literally $0/month, that's zero dollars and zero cents per month, serves as "the database" for an amount of data transferred that would have cost almost $1,000,000, or one million dollars, using S3. Sure torrents have come and gone from academictorrents, but it's tiny, niche, and bittorrent is 23 years old and very badly suited for research data. The ones that survive are the ones that people use and care about. If you want something to survive you can literally make that happen with a raspberry pi stashed in a storage closet with a flash drive attached to it.
it's such a preposterously good idea to develop p2p systems that actually meet our needs the ratio of benefit to cost is literally infinite.
-
jonny (good kind)replied to jonny (good kind) last edited by
i was using historical examples here because it is not cool to tell on a currently active private bittorrent tracker but if you know anything about the bittorrent scene, such as it is, you know that what i'm saying here is true and if anything an understatement
-
jonny (good kind)replied to jonny (good kind) last edited by
Two kinds of archives of comparable scale and complexity.
Kind A:
(a) has no resources,
(b) has no-one using their real names and thus no ego incentives or accountability,
(c) is actively being hunted by the entirety of global capital, intellectual property divisionKind B:
(a) has a ton of freely given resources
(b) is entirely driven by reputation and ostensibly has accountability mechanisms for failure to deliver
(c) is wholly legal and in fact one of the least opposed endeavors of information workWhen Kind A is finally hunted down and forced to suddenly purge all their servers without warning or has them captured by law enforcement, the site goes down but the archive is able to reconstruct itself in a matter of days.
When Kind B has its funding lapse it is a complete, catastrophic loss.
COULD IT BE that the pirates prove something about the survivability and effectiveness of p2p archives???? COULD IT ALSO BE that they reveal the basic patterns of power and control that drive people to consistently choose Kind B systems?????
-
jonny (good kind)replied to jonny (good kind) last edited by
A very important part that i elided here for effect is that it's actually not true that pirate systems have no ego or reputational incentives. They are actually vitally and sometimes suffocatingly driven by social incentives because archives are fundamentally a social process.
See @mpe 's fantastic work WAREZ that is the only academic work i know of that captures the social culture well,
or see Oliver Payne's "the Art of Warez" which is a very good historical depiction of the culture as told through .nfo art, demoscene music and keygens: https://vimeo.com/341663153 (mirror uploading now)
part of embracing p2p is embracing how it is made of people and that's a strength, not a weakness.
-
Interpeer Projectreplied to jonny (good kind) last edited by
@jonny The next question is: what if we can make something even better?
-
jonny (good kind)replied to Interpeer Project last edited by
@interpeer
Exactly -
Interpeer Projectreplied to jonny (good kind) last edited by
@jonny So... not to hijack this, but I made this video way too long ago:
https://makertube.net/videos/watch/1662e120-b86a-4040-a230-dc6ad5f658ce It's not quite on this topic, but sets the stage.
Parts 2 and 3 are ready to record. Part 2 will also look at p2p, but additionally other approaches for addressing overlapping issues.
Part 3 will be about how Interpeer is about creating a synthesis that aims to fill the blanks in all of those approaches.
Make something that hopefully is better, actually.
-
jonny (good kind)replied to Interpeer Project last edited by
@interpeer the more people dreaming about p2p the better
lmk when you have a written version, i don't usually parse factual videos well
-
Interpeer Projectreplied to jonny (good kind) last edited by
@jonny The architecture draft is at https://specs.interpeer.io/draft-jfinkhaeuser-interpeer/ - it contains much the same line of argumentation, but is probably a lot denser. And it could use some refinement and details.
I'm making these videos specifically to give a differently structured entry point into this. So I'm not sure I'll write them up as such, but I'll make the slides available!