@zebratale @carbontwelve Calculators do make mistakes. Most pocket calculators do arithmetic in binary and so propagate errors converting decimal to binary floating point, for example not being able to represent 0.1 accurately. They use floating point to approximate rationals, so collect rounding errors for things like 1/3.

The difference is that you can create a mental model of how they fail and make sure that the inaccuracies are acceptable within your problem domain. You cannot do this with LLMs. They will fail in exciting and surprising ways. And those failure modes will change significantly across minor revisions.

@carbontwelve I used machine learning in my PhD. The use case there was data prefetching. This was an ideal task for ML, because the benefits of a correct answer were high and the cost of an incorrect answer were low. In the worst case, your prefetching evicts something from cache that you need later, but a 60% accuracy in predictions is a big overall improvement.

Programming is the opposite. The benefits of being able to generate correct code faster 80% of the time are small but the costs of generating incorrect code even 1% of the time are high. The entire shift-left movement is about finding and preventing bugs earlier.

I finally turned off GitHub Copilot yesterday. I’ve been using it for about a year on the ‘free for open-source maintainers’ tier. I was skeptical but didn’t want to dismiss it without a fair trial.

It has cost me more time than it has saved. It lets me type faster, which has been useful when writing tests where I’m testing a variety of permutations of an API to check error handling for all of the conditions.

I can recall three places where it has introduced bugs that took me more time to to debug than the total time saving:

The first was something that initially impressed me. I pasted the prose description of how to communicate with an Ethernet MAC into a comment and then wrote some method prototypes. It autocompleted the bodies. All very plausible looking. Only it managed to flip a bit in the MDIO read and write register commands. MDIO is basically a multiplexing system. You have two device registers exposed, one sets the command (read or write a specific internal register) and the other is the value. It got the read and write the wrong way around, so when I thought I was writing a value, I was actually reading. When I thought I was reading, I was actually seeing the value in the last register I thought I had written. It took two of us over a day to debug this. The fix was simple, but the bug was in the middle of correct-looking code. If I’d manually transcribed the command from the data sheet, I would not have got this wrong because I’d have triple checked it.

Another case it had inverted the condition in an if statement inside an error-handling path. The error handling was a rare case and was asymmetric. Hitting the if case when you wanted the else case was okay but the converse was not. Lots of debugging. I learned from this to read the generated code more carefully, but that increased cognitive load and eliminated most of the benefit. Typing code is not the bottleneck and if I have to think about what I want and then read carefully to check it really is what I want, I am slower.

Most recently, I was writing a simple binary search and insertion-deletion operations for a sorted array. I assumed that this was something that had hundreds of examples in the training data and so would be fine. It had all sorts of corner-case bugs. I eventually gave up fixing them and rewrote the code from scratch.

Last week I did some work on a remote machine where I hadn’t set up Copilot and I felt much more productive. Autocomplete was either correct or not present, so I was spending more time thinking about what to write. I don’t entirely trust this kind of subjective judgement, but it was a data point. Around the same time I wrote some code without clangd set up and that really hurt. It turns out I really rely on AST-aware completion to explore APIs. I had to look up more things in the documentation. Copilot was never good for this because it would just bullshit APIs, so something showing up in autocomplete didn’t mean it was real. This would be improved by using a feedback system to require autocomplete outputs to type check, but then they would take much longer to create (probably at least a 10x increase in LLM compute time) and wouldn’t complete fragments, so I don’t see a good path to being able to do this without tight coupling to the LSP server and possibly not even then.

Yesterday I was writing bits of the CHERIoT Programmers’ Guide and it kept autocompleting text in a different writing style, some of which was obviously plagiarised (when I’m describing precisely how to implement a specific, and not very common, lock type with a futex and the autocomplete is a paragraph of text with a lot of detail, I’m confident you don’t have more than one or two examples of that in the training set). It was distracting and annoying. I wrote much faster after turning it off.

So, after giving it a fair try, I have concluded that it is both a net decrease in productivity and probably an increase in legal liability.

Discussions I am not interested in having:

• You are holding it wrong. Using Copilot with this magic config setting / prompt tweak makes it better. At its absolute best, it was a small productivity increase, if it needs more effort to use, that will be offset.
• This other LLM is much better. I don’t care. The costs of the bullshitting far outweighed the benefits when it worked, to be better it would have to not bullshit, and that’s not something LLMs can do.
• It’s great for boilerplate! No. APIs that require every user to write the same code are broken. Fix them, don’t fill the world with more code using them that will need fixing when the APIs change.
• Don’t use LLMs for autocomplete, use them for dialogues about the code. Tried that. It’s worse than a rubber duck, which at least knows to stay silent when it doesn’t know what it’s talking about.

The one place Copilot was vaguely useful was hinting at missing abstractions (if it can autocomplete big chunks then my APIs required too much boilerplate and needed better abstractions). The place I thought it might be useful was spotting inconsistent API names and parameter orders but it was actually very bad at this (presumably because of the way it tokenises identifiers?). With a load of examples with consistent names, it would suggest things that didn't match the convention. After using three APIs that all passed the same parameters in the same order, it would suggest flipping the order for the fourth.

#GitHubCopilot #CHERIoT

@catsalad There is no one who has ever offended me so much that I would wish JIRA on them.

@FediTips @stefano @mkj

That's all I'm trying to say with the original post, I'm trying to get across to a mostly non-technical audience why a network being on many servers is preferable to putting all your eggs in one basket.

But that's my point. To a non-technical audience, it doesn't matter. To a non-technical audience, their instance going down for a federated service, or their availability zone going down for a centralised service are indistinguishable. And 'Twitter is down' is an easier thing to explain than 'your bit of the Fediverse is down, but the rest of it is fine'.

There are a lot of benefits from federation. Being able to have a second source if one provider has terms you don't like, for example (though the fact that ActivityPub requires cooperation from the original instance to transfer your followers is a potential problem).

Federation and fault tolerance are orthogonal problems. You can build a fault-tolerant distributed system. You can build a federated system with a load of single points of failure. There are enough good reasons to prefer a federated system without claiming this one, which is a real stretch.

@mkj @FediTips

That there can still be problems (you're 100% right: there can) doesn't mean creating a situation where one affects everyone is better.

It isn’t better or worse. For most users, the thing that matters is the amount of downtime. If power goes out in each town in a country for a different week, is that better or worse than if the power goes out in the entire country for ten minutes and then comes back? Aside from the impossibility of doing a cold start of most electricity grids, I would expect most people to be happier in latter case.

The case with federation is more complex because two people can’t communicate if either of their endpoints is broken, so the failure modes are difficult to reason about individually.

Even though the aggregate network may never go down, in the same way that email is never down for everyone, that doesn’t really matter to most users. The thing that matters is how much there is downtime that affects them.

Being able to use a backup instance is not really a solution, because most Fediverse software doesn’t allow you to synchronise feeds across instances, so you can’t just fail over. Email is slightly better in that you can often send emails from an alternative server if your primary is down but you generally can’t receive them.

It is possible to build replication, transparent fail-over, and so on in a federated system, but ActivityPub doesn’t try to do any of this.

Psychologically there’s also a difference. When something is down for everyone, there’s shared commiseration, when it’s down for you that can be harder, especially when you’re the one using the unusual thing. When a thing that ‘everyone’ uses is working and the weird thing you use is down, people are less sympathetic than when the thing both of you use is down.

@FediTips If the instance that I’m using goes down, it doesn’t really matter to me that only the bit of the Fediverse that I use is down and not the rest. The surprising thing is that small instances run by volunteers seem to have less downtime than centralised systems run by teams of well-paid SREs.

@dalias @afewbugs @aku @leadegroot @Gargron

Is there a list somewhere of all of the things that you need to do to make the Mastodon web UI usable? And, ideally, a way for instance operators to make these the defaults, if upstream won't?

That said, I do want to see boosts, I just don't want to see boosts of replies to a post that's already on my feed.

@afewbugs @aku @leadegroot @Gargron I came with exactly the same question. About 80% of the things in my feed are cats and dogs, and sometimes I want to just see the cute animals, other times I want to just see the things in between them.

The other thing I'd really like for decluttering my feed is to not see replies to a post if that post is in my feed. I unfollowed someone today because a huge amount of my feed was filling up with their followers replying to their posts, which they then boost. And, because the feed is in reverse-chronological order, I see the replies first, so if I do click on them I then see the entire thread.

But my number one feature request to Mastodon would be stop doing whatever you do that breaks open in new tab. Given that both the back button and opening in new tab are broken, navigating in the web app is far harder than it should be. If I click on a post and come back, I lose my place because the feed is refreshed in the middle and I end up somewhere random. If I try to open in new tab so I don't lose my place, it doesn't work (the least discoverable bit of UI: The date in the corner actually does let me open in new tab if I click on it, but it does so on the original instance that made the post, not mine).

@jwildeboer Do you know how these numbers have changed over time? Battery densities have increased, which would naively lead to an increase in fire risk, but the safety designs (especially safely venting pressurised hydrogen that can build up when the cells deteriorate) have also improved. I wonder if there was a period when they were very unsafe, or if it was always just a marketing-driven perception.

Of all of the things I've learned in the last year, my absolute favourite is that cats establish dominance by grooming others, but rabbits establish dominance by making others groom them. This means that you can put a cat and a rabbit together and the cat will groom the rabbit and both will assume that they are now leaders of the pack as a result.

It’s not surprising that a society that values confidence more than competence would become excited by machines that generate text that sounds confident and is entirely wrong.

@Edent I only ever use Zoom with their web client. With their security track record, they’re never going to have a native app running on a system I care about. If I absolutely need to install their app ever, it will be in an ephemeral VM.

@drahardja When I was at Microsoft, I encountered the phenomenon of the -1x engineer. The people who threw a load of code at projects that then took at least one full-time engineer to fix bugs in and refactor. These people were often rapidly moved to positions where they could do less immediate damage, such as going to cross-group design meetings. Unfortunately, this raised their profile and got them promotions and transfers to other teams who did not initially know how much damage they’d done in their previous team.

Interestingly, if you look at commits, these people had the exact opposite profile to the so-called ‘ghost engineers’. The people fixing their bugs would often make a load of single-line changes, they would be the ones throwing a thousand lines of spaghetti into the repo at a time. Often, it wasn’t just that the code was bad, it was the design was fundamentally flawed and they were solving entirely the wrong problem, but they wrote a lot of code and so looked productive.

The best engineers increase functionality or performance and write a net negative volume of code.

@pvonhellermannn There are so many levels of bad planning in for climate change in the UK, but the one that really bugs me is that measures to reduce climate change do not take into account the facts of climate change.

Modern building regulations are great at keeping houses warm. A new-build house is far cheaper to heat (i.e. uses a lot less energy) than one built even 30 years ago. Yet the regulations say nothing about cooling. As a result, all of those heating savings risk being undone by people adding after-market air conditioners to keep houses cool in the summer heat waves. Simple passive cooling designs (such as putting the bottom end of staircases pointing south so that hot air has a path to rise and cause circulation) would help a lot, but they're not required.

Worse, if you buy a heat pump today, you are disqualified from the subsidy if the heat pump can do cooling as well as heating. One enterprising vendor has a heat pump that works in cooling mode only if an additional resistor is present and sells the resistor as an after-market add-on so that they qualify for the subsidy, but cooling modes should be a requirement for new heat pumps, not an undesirable extra feature.

There is simply no understanding of how to address systemic problems anywhere in government.

@futurebird

I've often felt that Christians should take a stronger position that 'intelligent design' is blasphemy. If you look at how, for example, the human eye is designed, it's entirely riddled with design flaws. The retina is not attached at the back at all and so can come loose. The optic nerve is connected right in the middle (rather than at the edges, where it wouldn't cause a blind spot). There's no mechanism for calibrating the lens (and the manufacturing quality on the lens is low enough that this is really important). The optic nerve doesn't have enough bandwidth to carry the full capacity, so there's delta compression, but there's no error correction for this and so you get a load of artefacts as a result. And that's not even getting into some of the, uh, interesting routing of nerves.

Claiming that this is the result of a god doing the best that they could is the same as claiming that the god is, quite frankly, a bit useless. You're saying that he's an incompetent deity who bodges things instead of doing good design work.

On the other hand, creating a set of initial conditions that gave rise to natural laws that drive evolution to create them is pretty impressive.

When I was a PhD student, I attended a talk by the late Robin Milner where he said two things that have stuck with me.

The first, I repeat quite often. He argued that credit for an invention did not belong to the first person to invent something but to the first person to explain it well enough that no one needed to invent it again. His first historical example was Leibniz publishing calculus and then Newton claiming he invented it first: it didn’t matter if he did or not, he failed to explain it to anyone and so the fact that Leibniz needed to independently invent it was Newton’s failure.

The second thing, which is a lot more relevant now than at the time, was that AI should stand for Augmented Intelligence not Artificial Intelligence if you want to build things that are actually useful. Striving to replace human intelligence is not a useful pursuit because there is an abundant supply of humans and you can improve the supply of intelligent humans by removing food poverty, improving access to education, and eliminating other barriers that prevent vast numbers of intelligent humans from being able to devote time to using their intelligence. The valuable tools are ones that do things humans are bad at. Pocket calculators changed the world because being able to add ten-digit numbers together orders of magnitude faster allowed humans to use their intelligence for things that were not the tedious, repetitive, tasks (and get higher accuracy for those tasks). If you want to change the world, build tools that allow humans to do more by offloading things humans are bad at and allowing them to spend more time on things humans are good at.

To put a billionaire in perspective, it's more interesting to think about the income than the capital. You can get a 5% annual return from some fairly low-risk investments (and you can further reduce risk by spreading your money across a load of these). You can often get higher, but 5% is a fairly good baseline.

If you start with $1bn and do nothing clever, you can get an income of $50 M per year by doing nothing. Even with the kind of 95% tax rate that we had for the highest income levels when The Beatles sang about it, that leaves you with $2.5M/year in income.

That's enough to buy a nice house every year, with no mortgage. It's a daily disposable income of almost $7K. It's enough to take a first-class transatlantic flight every day.

And that's just one billion.

Even with a 95% tax rate on investment income and a conservative investment strategy, someone with a billion dollars would have a daily disposable income that's more than the monthly income of anyone in the bottom 99% of earners, without having to work.

Now try these calculations again with real tax rates and Musk or Bezos' wealth.