Mine was that if I (or anyone else not-hn-famous) had written this and posted it to hackernews 10 times, it wouldn’t have gotten anywhere near the front page.

The aspect of prototype software that the author was calling out was that it is throwaway software.

Putting that aside, prototype software is recombining existing technologies and concepts in well trodden domains, which is distinct from the genuinely novel scientific work the author was contrasting with. Software prototypes are not in the same league, as much as you may like it to be.

Don't you think novel scientific work recombines existing knowledge? Isn't the saying precisely of standing on the shoulders of giants?

That being said I didn't compare both, not sure why you brought that up. I specifically discussed about prototyping, quoting a specific sentence, not scientific research.

Same. I had no idea if the article was for me and what I was going to get out of it, so I just gave up.

Simon Wardley:

> “GPTs are a non kinetic form of warfare designed to embed the values of a small number of people into much wider communities by capturing the process of decision making. The delivery mechanism is the appearance of helpfulness i.e. coherent and authoritative arguments. The payload is helplessness and the creation of a new theocracy.”

https://lnkd.in/p/efk8KaAN

Speaking as someone raised in an evangelical household and indoctrinated into thinking the world is only thousands of years old, this tracks. The most technologically advanced version of Chick tracts (https://en.wikipedia.org/wiki/Chick_tract) to date.

I didn’t realize until now that the “me, an intellectual:” meme format was parodying something that some people did earnestly.

Interesting that he has this view.

I would have taken him for mostly pro LLM.

They absolutely have a marketing department tasked with intentionally creating situations that people would find disturbing and plausible.

Chaos Marketing

Apparently I pointed at the truth

This article is great advice that I should take if I ever get a role where I’m not going to be pressured into delivering some kind of low-value BS during Phase 2 that causes me to hasten the delivery the most useful organisational changes I can. I can’t help myself when the wick gets turned up.

As a result, after 15 years as a software engineer, I’m genuinely considering leaving the industry altogether because the only roles available to me are ones where I’m expected to deliver features rather than organisational change and growth. It’s like my heaps of experience have navigated my career into a cup-de-sac, and the only way out is backwards. I’m so jaded. Hopefully it’s a phase. I need a coach. Help.

Maybe your responsibility is to push back against the pressure put on you to deliver short-term.

Because if this was an anonymous software company who had an employee who decided to hack HuggingFace, they wouldn’t be talking about it gleefully - they’d be in court.

It doesn’t matter to me how good the LLM is at writing Go if the compiler can’t stop it from accidentally leaving another part of the software with invalid state as a result of a change the LLM is making.

What am I talking about? Nil and partially constructed structs are impossible to prevent the creation of in Go.

Sure, if you’ve got a small program with limited scope, that’s probably fine if you look through squinted eyes. But the teams I work with are working on sprawling, evolving software where the compiler saying “hey, that’s not a valid Widget” would be extremely useful and save much heartache.

An LLM does a good job of “checking” for other uses and “checking” if everything is going to work correctly, but - supposedly we’ve committed the concept to code so that the compiler can actually verify it - and Go intentionally permits invalid states of structs. This makes Go a fundamentally problematic language choice for the kind of software I work with teams on, LLM or not.

Fair criticism, it's my main gripe with Go as well - it's not strict enough when it comes to e.g. nil, enums, and type safety. Annotations are supported but they're just strings. Projects require additional tooling / linters to check for things like unchecked errors and many more "gotchas" that I think could (should?) be part of the compiler or standard tools. Trivial example, Go's compiler will error when you have an unused variable, but won't if you reuse and overwrite an error variable a dozen times and only handle it once.

But on the other hand, I suppose it makes it a bit more pragmatic - less checks makes for a faster compiler, and fast compilation was/is very high up in the language's requirements and motivation. If you want / need more strictness in your language, there's Rust, Java, C#, etc.

i never run into these issues in >100k loc of productionized llm generated elixir (nil safety, type issues). i wonder, is there something architecturally in go that makes this a particular problem?

Up until quite recently, golang lacked generics, so `interface{}` was passed everywhere. It does not have typesafe enums (you can pass integers and it will compile). It has no way of declaring immutable structures. It does not have pattern matching.

> It does not have typesafe enums (you can pass integers and it will compile).

sigh. Sure, if you literally pass an untyped, hard-coded integer, like so:

    foo(100)

then, and only then will it compile. However, this won't:

    a := 100
    foo(a)

How horrific.

How about Swift and Erlang? Why are these always left out of comparisons?

Haven’t you heard? Swift is the first-ever corporate programming language, and we on HN can’t take its community efforts seriously to the extent of considering its potential value in general-purpose software.

/s

I see it as a great language being driven (fw atm) by same buy who did Rust, no? And my take really is that the fact they were no good corpo created languages (GO is not one or what?), then it is totally fine to some big corpo like the much-disloved-yet-selling-shitloads-of-phones Apple to actually do something which benefits the world and does not incur costs ojn it.

Good point.

It's sometimes challenging to get a Rust program to compile... but if you do, it's probably going to work.

The authors make some good points: compile time and test speed matter, platforms matter. But they really dodge the whole “guardrails matter” thing. And guardrails are going to win long term.

As for readability, the fact that AI-written Go closely resembles human-written Go is not necessarily a point in Go’s favour.

> "guardrails matter" This thing has been solved in the 70s. Basically, have a powerful type system, and a compiler that beats you into submission when you try to stray from the straight and narrow. Languages like (S/OCa)ML, Haskell and Rust will bring this to you. As a consequence, you fight the compiler, and once it submits, you have a good chance it's going to work. The compiler is also the ultimate refactoring tool here. Change the concept? Just change the type in the code and follow through by fixing the error messages the compiler spits out.

Of those, the weakest points for Haskell and the MLs is the platform and documentation. However, I don’t know if we can call this solved just yet. We may need to invent whole new types of guardrail.

> As for readability, the fact that AI-written Go closely resembles human-written Go is not necessarily a point in Go’s favour.

There's just not many ways of writing Go. It's a very dull language. It was designed to be dull and easily understandable.

but there are a whole lot of ways you can mess up a dull language, like shitty variable names, bad code organization, etc.

if an llm has learned dumb things from dumb users it could disproportionately cause provlems versus other languages, just by being "in a sloppy mood" when writing go.

Not panicking doesn’t mean working as intended, the biggest issue with LLM generated code is that it will do something that’s subtly wrong not that it will crash. It anything LLMs are too careful with Golang code and litter useless nil checks everywhere e.g. for function calls with pointer receivers. That whole fear of panics is totally overblown.

Sure, rust has better memory safety than most languages, but it also has a strong enough type system that many other kinds of programming errors won't pass compilation.

And by that point a program written in go has been deployed and making money for months. These are two wildly different languages, people should stop comparing them as if they're targetting the same niche. Checks and protections that rust has aren't necessary in most cases, but increase development and maintenance time.

Do you have some evidence of that? I use both daily and my experience has been the opposite, if anything. Once I was as proficient at Rust as I was at Go, the "increased development and maintenance time" disappeared completely.

> Once I was as proficient at Rust as I was at Go

That's a big time investment for many people though. The one they—or rather their employers—cannot really afford.

Rust's disadvantage in adding time is mostly a one-time initial investment, in my experience.

After you get comfortable and learn the important idioms, you go through the development loop quite quickly.

Compilation/linking speed can still suck though.

I've found that keeping very healthy test coverage solves issues like this. LLMs are very good at validating their own implementations with TDD.

With unit tests you gotta be careful though, oftentimes LLMs skip implementations with mockups that just say "not implemented yet" or similar and then the unit tests become pointless because they start to only test internal structures for being set / not default values.

For me it helped a lot to try to make containerized end-to-end tests and a custom TestMain for this, where I am using podman to run the integration tests. This way the end-to-end tests are forced to be on network level, and you can test protocol and API quirks much easier with LLMs.

Also, never forget to write a bootstrapping docs/ folder so that you don't have to re-explain these things all the time.

+1, I like to test in a similar way. For example if I'm making a CLI, the test will spawn the CLI for each test-case, instead of invoking the code directly. This provides a more realistic flow, and makes it clear which user journeys one is supporting.

Faking responses I often do with environment variables, like:

  ts := httptest.NewServer(...)
  cmd := exec.Command(...)
  cmd.Env = append(os.Environ(), fmt.Sprintf("MYCLI_PROD_ADDR=%s", ts.URL))

Of course, it's better still to go down this turtle stack (e.g. by spawning a local instance of your backend server instead of some faked handlers), but that adds more cost. I find the trade-off OK here.

I actually had to use a similar hack there due to the limitation that go test compilates cannot spawn themselves where I needed to have an environment variable with the actual binary prebuilt before the tests run. Took me a while to understand that TestMain doesn't cover that use case...

On Linux, "self" is /proc/self/exe. In general, I've seen people use `os.Args[0]`. But on checking again, I see there's even `os.Executable()` (https://pkg.go.dev/os#Executable) for this purpose.

I'm quite sure go test compilates can spawn themselves, I'm doing it on many platforms.

But, the test setup I was referring to was explicitly not that: the tests are spawning the main binary, not themselves. Using bazel+runfiles this is pretty easy to do. With the pure Go build tool, I'm not sure what approach I'd use to "guarantee" that I get a binary build for the same environment as the test.

i think they validate less with TDD; if you tell them a bare bones spec to validate they tend to write just that. if you write the test after, then their testing is causally conditioned on what was written. if you are going to write tests, it seems like teat first is way better than test last.

>> Nil and partially constructed structs

C# has "nullable reference types" and "required" keyword for that

So your problem is the default/zero values of properties?

In Go the convention is kind of to have a constructor pattern with a NewStruct(...) *Struct method that initializes all properties.

Also can't you build your own validator for that with the reflect package in the Add() method of your UI graph to prevent this sorta thing?

> the convention is kind of to have a constructor pattern with a NewStruct(...) *Struct method that initializes all properties

But that doesn't stop you from declaring a var s Struct, and never initializing it, or making a NewStruct {}.

> can't you build your own validator for that with the reflect package in the Add() method of your UI graph

Besides the fact that that would almost certainly significantly hurt performance, how would you be able to differentiate between unitialized data and data that was intentionally set to the zero value?

> But that doesn't stop you from declaring a var s Struct, and never initializing it, or making a NewStruct {}.

Static analysis tools can catch this, no?

Static analysis does not help if the type comes from a library and is _meant_ to be initialized using a literal.

And then upstream adds new fields where the zero value is different from the previous behavior, causing users to silently drift away from the intended behavior. I had this happen to me with a type from std, and had to add a specific test to guard against it with future std upgrades: https://github.com/sapcc/go-bits/pull/309/changes#diff-f5721...

That's a dependency making a breaking change, right?

The Go std library, as well as practically all go library code, is full of things that don't fully initialize all properties and things that nil-pointer-panic if you hold them wrong, so no, no matter what you do you have to deal with this wart of Go.

The go type-system is simply incapable of enforcing nil-safety without being no longer able to compile the go stdlib nor most code in the wild, so it's a quite valid criticism of the go type-system and language, and your comment doesn't hit on a valid solution.

You're painting a picture where people writing Go are constantly drowning in nil pointer panics. This is not reality. You hit them occasionally and they're trivial to understand and fix.

nil pointer panics aren't nearly as bad as values getting zero initialized, then used in places that assume they were initialized, and getting subtle bugs because the state is inconsistent.

I always wonder how those types of mistakes make it through your test suite. You'd need some kind of non-deterministic path to reaching the unintended zero value — but it would need to be a non-deterministic path that you wouldn't make deterministic during testing. I think we'd be curious to see what that code looks like.

Again, it happens, but the impact is wildly overstated. I get so confused with people saying "once it compiles, it probably works" (regardless of language). The problems I struggle with need invariants that can't be expressed by any modern language features.

Very good point, thanks for confirming that! What is the reasoning behind the Go design choices you described?

i am using nilaway from fb it's great for those

It's from uber, I believe.

Looking at the way GitHub are selling this “feature”, I feel like some of the engineers who are going to be excited about this feature for “reviewability” reasons are, in particular, those who’ve forgotten that they should be splitting changes into multiple logical commits inside a PR. And instead of that they’re now going to use multiple, single commit branches and stack them because stacked PRs are a “new” “feature”.

And the upshot for the LLM providers is that they get to charge for n reviews, instead of one.

I think plenty of people explained the difference between stacked PRs and individual commits in 1 PR. You’re not wrong, but it’s just a matter of path of least resistance. There is no way to leave a comment on a particular commit in a PR. Also all commits addressing PR feedback get tucked at the bottom of the commit list. Unless you do some crazy git gymnastics and rewrite the PR history and confuse everyone. With stacked PRs you also (as a maintainer or a reviewer) have the option to merge some and not others. Like here are 3 stacked PRs, one for provisioning some AWS or Azure resources that I’ll need, one for implementing the APIs using those resources, and one for updating the UI to use the new APIs. You can then say “let’s get the 2 backend PRs in and hold off on the UI as we’re changing that entire view”. You can’t do that with multiple commits in a PR without asking the person to redo the PR, then you’re back to git-foo.

Yes, GitHub could have made the UI allow a “per commit” comments somehow, then allow you to select the set of commits to include in the merge somehow, then write a blog post on how to manage “Address PR comments #1” commits. But the stacked PRs solve all that. Not to mention how people treat commits as their own internal save states. I always enable “squash and merge” option because I think it makes a lot more sense to have 1 commit on main per PR where all the context of the change is either in the commit message or the linked PR.

Also LLM providers charge per token. Charging per “work unit” is still not a solved problem. You can’t charge per “review” when your cost is per token. Just like airlines can’t charge “per ticket”, they have to charge differently depending on the destination. Unless you invent some bs arbitrage to lure users and eventually bait and switch on them.

Splitting changes into multiple commits is a much worse experience if you actually have self-contained dependent changes.

1. The whole review interface isn't set up for reviewing individual commits.

2. You can't merge changes progressively.

3. CI doesn't run on each commit.

4. If you have linear history (good idea IMO) you'll lose your nice commit history when you merge it.

This is much better.

You can read each individual commit and review the PR as a whole. This is the way many people design and review PRs.

You don’t need to merge those changes progressively. If you do, you go through exactly the same process of creating a separate branch and PR. The only difference is that GH has now added some UI and automation for rebasing and merging the PRs. In the past we would have explained the chaining in the PR and rebased manually.

You don’t need CI to run on each commit.

You only lose your commit history if you squash merge, many people don’t, and you don’t have to either.

The arguments come from angle that doesn’t appear to be aware that stacked PRs were a thing before GH made these UX improvements.

Look at some level you want a "thing that is reviewed and has CI run on it" right? Unless you work alone you need that. Let us call that unit of work, a flob.

When you have written and submitted a flob for review, you often want to continue your work on top of that, and then you may end up with a second dependent flob that is finished before the first flob is merged.

You want both to be reviewed. You want CI to run on both. It's simply a much better experience if flobs are PRs rather than commits. I dunno how else to put it.

> stacked PRs were a thing before GH made these UX improvements

Not in a way that worked properly. You could sort of do it for PRs within a fork, but it was impossible across forks which is the way most open source GitHub PRs are done.

> Look at some level you want a "thing that is reviewed and has CI run on it" right?

Yeah, it’s called a branch, or PR. It’s a set of changes you want to sign off.

It seems like you want CI run on every commit, which seems rather unnecessary. And if you don’t, well, that’s always been the case.

> Not in a way that worked properly.

GitHub operates git. No git changes have happened. It’s just commits and branches, in git. So anything that worked before, works exactly the same now, but with buttons taking out some of the small amount of effort you had to put in.

“Dear Mr Putin, Please don’t use our airspace to attack Ukraine else we will be forced to write you another slightly more strongly worded letter next time. Thanks, NATO”