When you ask an AI model what a medication does, its answer can sound precise and convincing. But is it remembering information about that drug, or simply interpreting its name? A new study on Olmo 3 shows that, in many cases, the difference matters—especially when we’re talking about health.
The shortcut is in the medication’s name
Many drugs include endings that reveal their pharmacological class. For example, names ending in -pril, -olol, or -azoline can offer clues about the type of medication and its possible uses.
That pattern allows a model to generate a reasonable answer without having specific knowledge about each drug. It’s similar to recognizing that a word belongs to a family because of its ending, even if you don’t know exactly what it means.
The team led by Kaijie Mo, from the University of Texas at Austin, studied this behavior in Olmo 3 7B Instruct, a model developed by Ai2. The choice was not accidental: Olmo 3 has publicly available weights, training data, documentation, and intermediate versions.
In medicine, noticing that a model gets something wrong is not always enough. It’s also important to understand why it arrived at that answer.
What the researchers discovered
The researchers designed a test to separate three possible sources of information:
- The affix, such as the ending shared by several medications.
- The stem, meaning the part specific to the drug’s name.
- Actual knowledge about a specific medication.
To do this, they replaced the stem, the affix, or both with invented, meaningless strings. They then compared how the model’s answers changed when given real names.
The results were striking. Between 12% and 18% of the medications evaluated appeared to trigger responses guided mainly by their affix. In these cases, the model could infer the drug’s general class from its ending.
But there was an even larger group: between 51% and 59% of the medications showed few signs of specific knowledge. Responses to a real name barely differed from responses to an invented name.
Put simply, the model could appear informed without showing that it knew much about that particular medication.
Frequency in the training data matters
The team also searched for the drug names in the open corpus used to train Olmo. To do so, they used infini-gram, a tool capable of locating exact phrases in massive datasets.
The relationship was clear: the less often a medication appeared in the training data, the more likely the model was to rely on its affix instead of using specific information about the drug.
This helps explain why a system can respond well about familiar medications while being much less reliable with uncommon names. The model generalizes from the patterns available to it, but that generalization does not necessarily amount to medical knowledge.
Generalizing is not always a problem
Using a medication’s ending is not automatically a failure. Affixes often contain real pharmacological information and can help identify a therapeutic class.
The problem appears when that clue replaces knowledge about the dosage, contraindications, interactions, side effects, or clinical use of a specific medication. An answer that gets the general category right can still be wrong about the details that matter most.
That’s why an AI could identify that a drug belongs to a certain group and still offer an unsafe recommendation. In health-related matters, sounding convincing is not the same as being well-founded.
Why open models matter
The study also raises a broader discussion about auditing artificial intelligence. With a closed model, researchers can analyze its responses, but they usually do not have access to the training data, weights, or intermediate versions of the system.
Olmo 3 made it possible to connect three elements: observed behavior, possible internal mechanisms, and the information present in the training data. That traceability makes it possible to investigate not only what a model answers, but also where its answer might come from.
For systems used in medicine, this level of analysis could be decisive. It makes it possible to detect shortcuts, validate results, and design more demanding evaluations before a tool reaches patients or healthcare professionals.
Transparency does not guarantee that a model is correct, but it does make it easier to discover when it isn’t and understand how to improve it.
The lesson also applies to people who use AI in their daily lives. If an answer about a medication seems reliable because the model recognizes its name, it’s worth pausing. Is the model citing specific information, or is it simply completing a familiar pattern?
In medical matters, AI can help organize questions and find general explanations, but it should not replace consulting a professional or verifying information through reliable medical sources. A drug’s name can offer a clue, but a clue is not a medical history.