We set up a local 7B parameter model on a small server, hooked it to our support email channel, and gave it one instruction: be honest. If it didn't know the answer, it was instructed to say "I have no idea, let me find a builder."
The model succeeded in solving 65% of queries without human assistance. However, in 5% of cases, it hallucinated custom terminal flags that do not exist, advising a customer to run a command that would delete their logs. We added a strict validator loop. Honesty requires constraints.
AI is at its best when it acts as an active compiler partner. It should translate a founder's spreadsheet idea into a Go schema, while the human builder oversees the latency curves, database locking strategies, and real-world testing.