On Tuesday, President Donald Trump declared that the White House is launching America.gov, an AI chatbot designed to assist individuals in locating government services and other resources.

Google has verified its role as a partner in this initiative, stating that its AI model, Gemini, is part of the effort, although it remains unclear if additional AI firms played a role as well.

“Rather than requiring citizens to navigate the overwhelming labyrinth of countless government websites and regulations… you will now have a single access point for any question,” Trump mentioned in a post on X.

Easing the navigation of government services is a commendable objective, but is utilizing an AI chatbot the most effective solution? Large language models are not perfect and are still susceptible to generating hallucinations. Indeed, CNN recently revealed that the U.S. military came close to initiating a military operation against a Chinese ship it thought was transporting components for nuclear arms. The military canceled the operation at the last moment after realizing that the perceived threat was merely an AI hallucination.

If an individual receives incorrect information from America.gov, it likely won’t lead to nuclear conflict, but an error can still carry significant repercussions. People may depend on this chatbot for details regarding food stamp applications, visa renewals, or tax filings, where inaccuracies could result in missed deadlines, denied assistance, or financial penalties.