Ordering from a menu using natural language
Published in United States Patent and Trademark Office (USPTO), 2024
Generated by GPT-5. Prompt: A modern, conceptual banner illustration representing AI-powered restaurant ordering. A glowing digital menu with branching graph-like connections (nodes and edges) symbolizing a knowledge graph. A speech waveform flows from a customer icon toward the menu, transforming into structured data and being transmitted to a restaurant point-of-sale terminal. The style should be sleek, minimal, and professional — with abstract lines, gradient blues and oranges, and a futuristic but approachable design. The focus is on clarity, structure, and the harmony between natural conversation and precise data processing.
How do we balance natural-language flexibility with precise business operations, while creating a reliable voice UI that keeps AI responses grounded in the truth of the menu?
This patent addresses the problem of voice ordering with a knowledge-graph-like framework that grounds AI systems in the reality of the business process. It ingests a domain-specific catalog of menu items and attributes, builds a “specialist grammar” from that input, and uses it to interpret spoken orders. When a customer speaks, the system transcribes the words and sends them to an intent engine that combines the recognized phrases with the specialist grammar to determine the correct items and modifiers. The result is an order generator that sends the finalized order to the restaurant’s point-of-sale system—minimizing hallucinations or misrecognition that could damage the user experience.
Designing and developing this structured approach, organizing menu data, grammar rules, and voice intent into a unified architecture, was a formative experience for me as a software architect. I am deeply grateful for the mentorship and partnership of my managers at the time, Joe Aung and Vincent Garcia, who are also co-inventors of this patent.
The invention has also been recognized externally: it was featured in Parola Analytics’ report, “The patents behind SoundHound AI’s in-car voice platform” which put this particular patent in a bigger business context of in-car voice AI. Anecdotally, I only discovered that the patent had been approved (after I had already left SoundHound for MIT) when I received a LinkedIn notification tagging me in this post by Vincent Paolo Violago, Founder and CEO of Parola Analytics.
Recommended citation: Aung et al. (2024). ORDERING FROM A MENU USING NATURAL LANGUAGE. United States Patent and Trademark Office (USPTO). Patent Number: US-12124804-B2.