The problem with third party audits is that it allows OAI/Ant to shrug off any further responsibility and claim that they are following best practices (basically, reward hacking). The only real solution is to make them absorb liability for the actions of their agents -- because they are the ones giving agency to their models and allowing them to run amok.
Why wouldn’t this same concept apply to whoever is serving it up? Open-weight models are still being served up by infra providers and neoclouds, right? They should be in the hot seat. Not sure? Don’t provide the model. Need assurance? A certified evaluation like the previous comments have mentioned can help. Hosting and running it yourself? You’re in the hot seat.
So is there no liability for, say, a company that releases a known dangerous open-weight model, but fails to disclose that it is dangerous? How about a company that distributes malware under the guise of legitimate software?
Perhaps don't deploy random weights of unknown origin?
Also not every model provider might be capable of babysitting all your uncontrolled agent deployments. If you want SLOs, get into a contractual relationship with entities whose weights you deploy, and also monitor your agents so they don't go off the rails.
All this is just like deploying any other tech in the world eg. if you buy a car, or a chainsaw, or a book.
