The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.
Built exactly this for work a few months ago. Claude has a read-only account to the dev DB and a bunch of python postgres tools, I point it at queries/views and ask it optimize, and it has to:
1) verify identical query results
2) run repeatedly to get average, worst, best, etc duration of runs
Sped up so many legacy things that none of us were ever going to bother with.
What if the shape of data in the dev db doesn't stay consistent with real life?
Is the harness for the AI set up with any sort of instructions to generally prefer broadly-applicable first-principals query performance analysis over unintuitive results that may be local maxima due to certain things specific to that env?
Even with humans I've had to unwind "optimizations" before that worked great for low-volume envs by taking advantage of 'be inefficient in the small scale with memory to save CPU and wall clock time' or such in a way that caused pain deployed.
It comes from C. Northcote Parkinson's work, where he calls it "The Law of Triviality". The idea is that people clog up decision-making by introducing a swarm of issues that have little relative impact or importance for the project at hand, leading to a massive misallocation of attention. His example is a $10 million construction project for a Nuclear Reactor where as much time is spent debating the materials to be used for a $3000 bike shed as on the actual reactor buildout. The law of triviality says "the time spent on any item of the agenda will be in inverse proportion to the sum [of money] involved."
Source: C. Northcote Parkinson, "Parkinson's Law and Other Studies in Administration" (1957)
Thanks. I know what bikeshedding is. I’m asking what cyandees specifically means. The normal definition of bikeshedding doesn’t seem to make any sense in his sentence.
I’m assuming he’s using it to mean “cruft” or “low value features” but maybe there’s some meaning I’m missing.
Funny enough, I didn't know, but your comment made me look it up, then I tracked down a PDF of the book, read the relevant section, and wrote my reply. So, even if I didn't help you out, you helped me lol.
I originally heard it in the context of picking paint color for the bike shed, which is probably more on point for software engineers, most of whom would not be involved in any discussion around building a nuclear reactor.
LLM increase the range of things bikesheddable, so some folks will try to have a LLM design a nuclear reactor and manage the coffee budget in addition to the traditional bikeshed design complications. Whether this is an improvement or more engines of what passes for civilization catching on fire, well…
I don’t understand how LLMs make more stuff “bike sheddable”. What LLMs do is allow people to contribute low value garbage to all areas, but we have a word for that and it’s not “bike shedding”. It’s “slop”.
I hope this weird redefinition of bike shedding doesn’t take off. It’s a useful and meaningful term and repurposing it to mean AI slop is negative value.
Thankfully until this thread I’d never seen anyone use the term this way.
> Can only assume english might not be your first language.
This not-subtle insult does not add anything here.
It's a low-effort drive-by comment, but I read it as tourists prompting about an issue they don't really know about.
People will recognize a glimmer of something they can attach to, prompt and get a few pages of junk about this narrow topic, and then contribute it in a way that further reduces the signal-noise ratio of the discussion or project.
Like, dude, we have the same tools. We can get the same slop from the tap at any time.
Just like humans, AI needs a "reality" to check things against. If we don't provide that in an automated way, somehow, it will always make mistakes and we'd have to point it out. Since, at that point, we're just mediating with what reality gives us and what an AI should do with it.
AI doesn't have enough senses yet. It's trapped in a box.
I’d say it’s not so much senses but defining what success means.
If you want to learn darts or perfect parallel parking for example, most of the increase in accuracy comes from simply but very deliberately
pointing out to your brain where you wanted to land versus where it did land.
We like to usually just do things and hope for the best. Defining success is not something we automatically do and naturally we don’t do it with AI either.
Very few problems really have "all the tooling to verify its hypotheses" though. even if you want to construct such an harness.
Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses'
> Also let me ask you why we need better and better and models if what we have already can produce good output with 'all the tooling to verify its hypotheses'
“Good” isn’t “perfect” and even if it was, the ability to produce perfect output with all the tooling to verify its hypotheses could still be improved, in time and token efficiency, by better models producing fewer spurious hypotheses, rejecting those it does generate faster, and taking fewer unnecessary steps in confirming its good hypotheses.
> Very few problems really have "all the tooling to verify its hypotheses"
This is such a blanket dismissal that I can’t agree or disagree.
Maybe very few of YOUR problems are this way. At least mention some problem domains.
Recent experiences: compiler-related (helpful), UI-related (agree it isn’t testable but the design iteration is quick, easy, and correct), debugging technical configuration problems (useless; I basically have to solve each problem myself before the LLM recognizes it).
We don't. You could freeze model development today and we would be able to use our current models to effectively optimize existing code for years to come.
The same tooling that you'd use. AI is not magic, it cannot know exactly how a program will run under your exact circumstances and data. Therefore, if it can use a profiler, test suite or application, or be able to read logs, just like you would, it then can make changes, evaluate them, and iterate. Personally, I've created test MCP servers that will perform actions like submit orders, have the AI capture and analyze PerfViews and Concurrency Visualizer, write out tracing data to logs... all the same stuff I was doing by hand for years.
Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD!
I’ve been working on UI component improvements and it was doing a lousy job until i specifically told it to test in a headless browser to validate it works. I think somewhere in an AGENTS.md i have an instruction to “don’t state your guesses as fact - validate findings and results”.
It bothers me that you have to explicitly state this to the agent. Makes me think what else is missing from that file which also needs to be explicitly stated, but I don't know what don't know. "Do a good job"?
This is where the harness shines. It's why products like Cursor have been growing like crazy. If you drop an extremely intelligent person into a problem they've never seen before, the initial context and knowledge-sharing just to get them up to speed is immense. They'll make a lot of mistakes until they're ready to fix your problem. The harness can leapfrog most of that. There is a booming industry in harnesses with no saturation in sight. We're not even close to maximising the utility of current models, let alone the amazing improvements which are happening week to week.
An agent doesn’t know what “job” it’s being asked to do until you tell it.
Imagine you’re blind and deaf and have temporary retrograde amnesia. You “wake up” one moment with a memory of some words in your head like “what is the bug?” …but you don’t recall the context of that question, and nor can you look/listen around to observe the context.
So you don’t know whether you’re e.g. at the office, in front of your computer, in the middle of doing some pair-programming (where, yes, you’d in investigate the bug thoroughly with tools), vs. having a conversation with a colleague over lunch (where the expectation is for you to tap into your knowledge + intuitions to either guess or say you don’t know — not to pull out your laptop.
That’s what it’s like to be one of these LLMs being prompted by some agent harness. Unless the harness injects the proper context into its “recent memory”, it just doesn’t know.
If you ask Fable or 5.6 Sol to improve performance, it will generally know to build a benchmark and create a test corpus. I'm not sure where the contrary suggestion is coming from.
Will, it depends on the AI. Anthropic used to have a lecture-length system prompt for their models to explain this stuff—part of the secret sauce for Claude Code—and famously found that the 5 series models no longer need it.
As usual, if you use anything but the best model available I’m going to state that the better ones do better. If you do use the best model available, then I’ll just mention that Fable still has limits and still needs some guidance.
One thing it does not do is deliberately build tests which test nothing at all, or which restate the code under test. I mention this because certain other models absolutely would.
This was my experience with some of the stupider models, but I haven't had Claude do this type of thing since I started using it for work about 6mo ago.
Claude will do some boneheaded things for sure, but it's pretty good about writing tests that are useful, and not removing or modifying tests just because they're in the way.
Claude is pretty bad about assuming that it couldn't have broken a test it didn't know about, as it has often told me "this is already broken on main" which is definitely NOT true.
> Tests! Unit tests, integration tests, random adhoc scripts. You know - TDD!
Tests onli validate the presence of bugs, not their abscence (Djikstra).
I'll also add that tests look at outputs and don't care how those outputs are derived. E.g. code filtering the entire db in memory will be fine in tests.
"Beware of bugs in the above code; I have only proved it correct, not tried it." - Donald Knuth
Tests prove the things you thought of worked, and it often isn't hard to find the likely edge cases such that you have reasonable confidence everything works. You will be wrong from time to time, but not that often. You can prove code correct, but if the proof is wrong (common when a human is doing it), or the spec is wrong (most people have no clue how to write a comprehensive spec) it can still be wrong.
Haven't you ever written tests that assert that certain things happen or do not happen along the way? We use mocks all the time to, say, make sure that the code doesn't load the entire db in memory. Then it's there as a regression if somebody down the line wants to do that exact thing for some reason.
If the agent can write the tests its supposed to pass and we are worried that agent produces quite a lot of slop (which is why we are doing the tests), then what is the defense against test slop? Test the tests?
1st line of defence, use something like ponytail to enforce brevity. Use property testing and behavioural testing on top of unit testing. Enforce readability standards so you will be able to understand the tests.
2nd line, code review.
Do the first pass with an agent, ask it to bounce back vacuous or tautological tests. Ask it to verify that the tests verify what they claim to. Then read them yourself.
3rd line, mutation testing. If the tests don't actually catch broken code, kill the mutants.
Doing a pass where you just ask the AI to sanity-check the existing tests (against rules like “test against the spec, not the implementation” can also help.
I'm convinced this is why there's a bifurcation in old-hand/new-hand results: the old hands include the steps they'd do and the tools they'd use, and the new-hands never learned those tools.
In this case, you need benchmarks, in addition to regular tests. You also need concise instructions (or better yet, following your language / ecosystem norms if applicable) to how to run this stuff
Yes, sort of. Not that there is anything wrong with that. Fuzzing yields tons of trash results, the occasional fleck of gold and the even more rare nugget. Then you analyze how it found the flakes and nuggets update your fuzzing harness. Every so often you get to write a paper or give a talk about a breakthrough to share meaningful improvements.
Current AI isn't super effective at making the breakthroughs, but it sure is effective at democratizing the ones you can point it at.
Until a couple of years ago you needed to be able to read research papers and code to build fuzzing harnesses and triage the findings to find vulns. Now you can point Claude, codex, Kimi, or even some smaller open models that run local and get a servable fuzzing loop in a couple of hours.
That's democratization - shifting power and skills away from those with the wealth to purchase vulns and run offensive security teams. It's still not simple, but it's one of the reasons bigger vendors want to lock down the capabilities of models that present a threat to the wealthy and powerful.
Sure. Wealth has always given early adopters and first movers wings.
As hardware specs get better, and model providers keep making different classes of mistakes that alienate users, more people will opt for or support local models. I tend to take Schneiers' "Attacks only improve" philosophy and apply it to damn near everything like this. Inference will get more expensive, but IMO cloud based inference is at or near the maximum that users can tolerate, across multiple spectra. There may be a marginal increase in cost, but as the different metrics between what is affordable for rapidly scalable, cloud based inference and the lag of local inference with open models and expensive hardware converge, prices will come down.
I am not the right person to say what the peak is going to look at, but when the companies with (practically speaking) unlimited compute resources and money are starting to take a good hard look, there is going to be a drive towards efficiency. I hear grumbling at work, and the appreciation from my leaders when I show up with a tool that shows how I reduced my token costs, or I keep asking the questions of what the token cost (real, and internal billing) are for tools my peers write are.
I also have absolutely eye-watering personal AI bills that are starting to make buying a better tier of local inference gear look more affordable, even with inflated hardware prices. The current generation of open models are very effective, and paying for multiple $200 dollar a month subs is not feasible longer term, and as one example, in one month, if I was paying enterprise token rates, I would have burned through $8000+ token budgets on one of those accounts, doing actual, practical useful work, it makes more sense to buy some much better hardware than to pay for one-time inference costs.
If you run it in a sandbox with no internet access and a read-only role login for the database, and you supervise it, how big is the risk do you reckon?
Of course there's an issue if the data in the database is contractually restricted and you don't have a zero retention endpoint for inference, but the former usually comes with the budget to fund the latter.
