No shit, if you train a stochastic parrot on biased data, you get biased output?

[xgranade@wandering.shop]

I pulled a random arXiv preprint¹ off a Google Scholar search for prompt engineering, and this is the size of the datasets² that they're using for all their conclusions.

The quality of evidence in that field is almost nonexistent. They're reporting on deltas of less than 1% based on a sampling procedure that can at *best* give 3% margins of error.

¹accepted at a major conference, using a preprint to avoid paywalls
²apologies for shitty alt-text, getting alt-text of tables is tricky

[aud@fire.asta.lgbt]

@[email protected] cool. this is one of those times I'm feeling salty that I let that whole "evidence" and "statistically proving or disproving the hypothesis and getting enough data to confidently say anything" get in the way of the publication needed to finish my phd.

[xgranade@wandering.shop]

@aud My whole thesis was basically on how to rigorously evaluate the quality of control in quantum systems... I hear you.

[aud@fire.asta.lgbt]

@[email protected] I had to spend six months reworking our statistical analysis pipeline because things just weren't agreeing and I still couldn't get convergence across the 900-odd simulations I was doing for the fixed set of parameters.

and these people just... they just... ... "prompt engineering". I.

I.

[aud@fire.asta.lgbt]

@[email protected] (also: I am so sorry that they keep trying to make quantum the next snake oil)

[xgranade@wandering.shop]

@aud Yuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuup.

[aud@fire.asta.lgbt]

@[email protected] (fun fact on that one: thanks to github's total fucking inability to let you do name changes, I no longer have credit for that on my github account. You'd have to know my deadname and my prior account)

[xgranade@wandering.shop]

@aud God that sucks, I'm sorry. There's too many places that my deadname appears in the academic record for me to possibly correct at this point, sadly. The fun of transitioning very late in life and *after* becoming famous in a tiny, insular field.

[aud@fire.asta.lgbt]

@[email protected] I was lucky enough that I switched to publishing under my initials because I sensed some discomfort with my actual name (even though I hated my initials) after the first or second paper, so that's helpful. But GitHub just doesn't allow account name changes, or at least didn't in 2019*.

* actually, the real problem is that you can make a new account and delete the old one and have it link, but if anyone ever registers it again all the credit will revert back. So about two days after I registered the new one and got rid of the old one, a friend helpfully registered the old one and gave it to me. So no one could squat on the account... but also now my account lost all my old contributions, sort of... defeating the purpose. I appreciated the intent though.

[xgranade@wandering.shop]

@aud What a fucking mess. The degree to which technical systems are designed with complete disregard for how names work in practice is absurd.

Cool URIs don't change, but cool names absolutely do.

[aud@fire.asta.lgbt]

@[email protected] RIGHT?!

"tech bros give a shit about anyone except themselves" challenge level: impossible

[xgranade@wandering.shop]

@aud Like, account systems have no business assigning anything other than a fucking UUID to individual accounts, then at *most* using stuff like OAuth claims as evidence of names.

[xgranade@wandering.shop]

@aud That is, everything should be more or less of the form "this technical system believes that the person who wrote commit abcdef123 is the same person who claimed the name X at time Y because of evidence Z."

[aud@fire.asta.lgbt]

@[email protected] THANK

YOU

do you bet your ass that's how I'm assigning IDs? yes. yes that is. because everything else can change ffs.

[aud@fire.asta.lgbt]

@[email protected] (obviously some extra challenges with federated stuff but nothing impossible to work around while maintaining human readability)

[xgranade@wandering.shop]

@aud Now I'm wondering if there's something like RDF / Notation3 where every triple links to evidence of that claim.

E.g.: <person with UUID X> <is> <url to external account> because <proof>, where because is a built-in part of the grammar and not a separate URN-ed verb.

[aud@fire.asta.lgbt]

@[email protected] well shit, I hadn't heard of this stuff before. This is cool!

[xgranade@wandering.shop]

@aud Notation3 is honestly very, very cool imho. It never really took off because of more general backlash against Semantic Web concepts, but it's a fun way of building an ontology for some knowledge base.

The trick, though, is how to extend that to an epistemology over the same ontology. By default, N3 recognizes the verb "to be" as primary, but that isn't great for epistemology...

[xgranade@wandering.shop]

@aud The Wikipedia article on blank nodes has some great examples of how you can use N3 ideas to represent "there exists a person such that I know the following list of facts about them" without needing any concrete ID for that person.

```
_:entity
ex:fullName "Alice Carol" ;
ex:homePage <http://example.net/alice-carol> ;
ex:hasAddress _:address .
```

[xgranade@wandering.shop]

@aud That carries the same issues with names, but if you had something like `_:entity is <https://wandering.shop/@xgranade>` and used WebFinger to look up the external URL to build up more tuples like `<https://wandering.shop/@xgranade> displayName "Xandra Granade"`, then by transitivity of `is`, you could imply the properties you want.

That's basically a very rudimentary theorem solver with external lookups.