The exact meaning of a certain setting in Calendly (app I use w/students for office hours) was really unclear in the UI.
-
Paul Cantrellreplied to Paul Cantrell last edited by [email protected]
This entire process took ~12 minutes.
Calendly had to pay both some energy-munching LLM and a live human for that time. Multiply that by their userbase, and I find it hard to imagine that this was cost-effective for them vs giving the UX and doc writing, say, 20% more breathing room from the start.
Cost-trimming is so expensive.
2/2
-
@inthehands This is exactly the kind of thing I bemoan about the dAtA-dRiVeN fad that's overtaken tech management over the past decade or so.
I've gone to managers with these concerns and been told, how can you prove e.g. these docs are inadequate, that investing more in tech writers will pay off? Come to me with the metrics and we'll talk.
I can't? I don't think it's possible, not in general? Certainly not a priori.
-
@inthehands Like, maybe I could stroke my chin and posit that you'll get a 35% reduction in support case volume and time, but I don't believe it, not really. I don't think those data are clean enough to support that analysis, or could possibly be disentangled from everything else that's changing.
You also aren't gonna see customer happiness in those metrics. You won't see the potential customers that said nah, this isn't good enough, based on the shitty docs.
-
@inthehands At a certain point, managers, leaders have to exercise judgment about what the company's going to do that's not grounded in metrics. Either that, or we're going to climbing the top of some very shallow hills, heedless of the mountains all around.
-
@donaldball
1. You can’t really manage what you can’t measure.
2. You can’t really measure what you want to manage.People always forget #2.
-
@inthehands Discount that by the fact that only a tiny percent of their user base is willing to spend even 1 minute figuring this out never mind 12. (Obviously people not understanding how that feature works has its own cost or downside or whatever, but it's pretty hard to account for)
-
@aubilenon
I mean, that’s just the thing: vague dissatisfaction’s impact materializes slowly, is hard to measure, and is hard to trace to a root cause. Crappiness always looks cost-effective at first. -
@donaldball @inthehands grimalkina at mastodon.social Cat Hicks has said something similar I recall, that without a background in scientific data collection and statistical analysis of the type of data you are trying to collect, you are unlikely to have conclusions that are really backed up by any confidence factor like a P-value, you aren't looking at limits, constraints and alternative theories, you're mostly trying to underline a preexisting conclusion with a pretty graphic
-
@raven667 @donaldball
P-values are a curse: it’s easy to produce them for any given piece of data, and •extremely• hard to make them actually mean anything. They tend to just be obfuscation for bad methodology. -
@inthehands @donaldball sure, people game P-values and the methodology of many scientific papers is not reproducible, but most tech/software metrics don't even reach that level of bullshit.
-
@raven667 @donaldball
Several of my statistician colleagues would like us all to more or less ditch p-values for both those reasons: they can be gamed, and they can be applied with so much thoughtlessness that even describing it as “gaming” is giving it too much credit. -
@inthehands: would you mind specifying what that setting is and what it does?
I'm considering using Calendly in the future.
-
@VisualPlugin
Sure. The setting is a “daily limit,” and it is per event type but not per invitee. I’d wanted a way to prevent one student from (typically in confusion) scheduling multiple appointments in the same day and squeezing others out, but the product can’t do that yet.