Why Custom Tool Schema Fail on New Claude Models
Armin Ronacher’s recent blog post talks about trhe strange issue that he started noticing with new Claude LLM models like Opus 4.8.
Bascially, Pi Agent has edit tool which has nested edits[] array however Claude Code’s edit tool call has flat shape like file_path, old_string, new_string.
And the new models started hallucinating random extra keys like oldText2, requireUnique, etc which doesn’t exist in the schema - even in some cases the actual context was correct.
What’s the cause?
Armin’s hypothesis is that the newer models are trained on Claude Code (post training RL) which has a flat tool call response for edit tool call and the extra key names that we’re getting is also similar.
Claude Code’s client is very instructive: it contains retry paths for malformed tool use, parameter aliases, type coercions, Unicode repairs and filtering of unknown keys. In other words, Anthropic’s own client appears to expect and accept a fair amount of slop and repairs it, mostly silently.
Aparently, Claude Code expects this and tries to heals itself to some extend.
Key Takeaway
If you’re building a harness and your tool call schema is different than Claude Code (or any other harness which dominates the training set), then you’ll ended up paying a hidden tax.