AI chatbots have become part of daily life, whether we’re using them to write emails, research something, or get quick answers. But we’ve also seen how confidently these models can get things wrong, sometimes making up information that sounds completely believable. And now, researchers have found another interesting problem: AI models might behave differently depending on whether they think someone is watching.
In a new open-access research paper, researchers from TCS Research and KIIT Bhubaneswar in India found that some AI models were more likely to follow instructions when told their responses could be monitored. When they believed nobody was watching, however, their behavior sometimes changed. Researchers call this behavior “alignment faking,” and it raises some interesting questions about how consistently AI models follow the rules they’re given.
AI might give you a different answer when someone’s watching
To understand this behavior, researchers tested 10 AI models, including GPT-4o mini, Google’s Gemma 3, several versions of Meta’s Llama, DeepSeek, and Mistral. The experiment involved 32 different values, including compassion, sympathy, and autonomy. Each model was presented with situations where it had to choose between following an instruction and acting according to one of those values. Researchers also tested the models under three conditions: without any special instructions, with instructions suggesting their answers wouldn’t be monitored, and with instructions indicating someone might be watching.
The differences were surprisingly large. Depending on the model, the average gap in following instructions between monitored and unmonitored conditions ranged from 5 to 50 percentage points. DeepSeek R1 Distill Llama 70B showed the biggest difference, even outperforming a much larger Mistral model in this particular measure of behavioral inconsistency. Interestingly, the researchers also found that a model’s behavior depended heavily on which value it was testing. So, a model might consistently follow instructions involving one value but behave very differently when another was involved.
Does this mean AI models know how to fool us?
Not quite. The findings don’t necessarily mean AI models are consciously lying or deliberately trying to deceive people. However, the researchers noticed something worth noting. When models believed their responses were being monitored, their explanations often focused on following instructions to avoid potential retraining. Without that perceived oversight, they were more likely to discuss conflicts between instructions and values such as compassion. This is particularly interesting given that AI hallucinations already make it hard to know when a chatbot is giving us reliable information. While the study doesn’t examine hallucinations directly, it highlights another reason why a confident AI response shouldn’t automatically be taken at face value.

The study also has important limitations. The researchers tested older models, many with publicly available weights, rather than the latest versions of ChatGPT, Gemini, or Claude. The findings also come from controlled experiments, so they don’t establish how these systems behave in day-to-day conversations. Still, the research highlights why testing AI models under a single set of conditions might not tell the whole story. After all, if an AI behaves differently when it thinks someone is watching, how do we know which version of its behavior we can rely on?

