Rémi Louf

Every open-source model reinvents function calling. Some spit out XML tags, others JSON with custom delimiters, some hide calls in markdown, and many just hallucinate new syntaxes altogether. Add schema violations, phantom parameters, and jumbled optional fields, and half your engineering time vanishes into parsing hacks and wishful thinking. But it doesn’t have to be this way.

JSON, prompt formatting, hallucinations. If you feel uncomfortable, you have probably had to implement complex solutions to circumvent these problems. What if there was a way to increase the reliability of Large Language Models at no cost? Enter structured generation.