In the early days at ZAR, we had a merchant product that looked great in Figma and tested well in moderated sessions. Users said they understood it. They completed the tasks. The satisfaction scores were fine. And then we launched it, and merchants in Lagos, Nairobi, and Karachi quietly stopped using it after the first week. The numbers told us something was wrong. But the numbers could not tell us why. So we went to the shops.
I spent hours standing in small retail stores, watching merchants interact with our app during real transactions. Not in a lab. Not on a call. In the actual environment where the product was supposed to work — surrounded by customers, noise, bad lighting, and a phone that was also being used to play music, respond to WhatsApp messages, and check football scores. The first thing I noticed was that nobody read our onboarding. Not a single person. They tapped through it like it was a terms-and-conditions wall. The second thing I noticed was that our primary action button was competing with three other elements for attention, and in direct sunlight, the contrast ratio made it nearly invisible.
Testing tells you if something works. Observation tells you if something will be used.
These were not things any usability test would have caught. In a controlled environment, people read instructions because someone is watching them. In real life, they skip everything and try to figure it out by tapping. In a controlled environment, screens look fine because the lighting is fine. In a shop in Karachi at 2pm, your carefully chosen gray-on-white text becomes invisible. Field observation reveals the gap between how people say they will use a product and how they actually use it. That gap is where most product failures live.
We redesigned the merchant flow based on what we saw, not what people told us. We removed the onboarding entirely and replaced it with contextual hints that appeared only when relevant. We tripled the contrast on primary actions. We moved the most-used feature to a persistent bottom bar instead of burying it behind a hamburger menu. Retention improved within two weeks. The lesson was not that user testing is useless — it is that observation and testing answer different questions. Testing tells you if something works. Observation tells you if something will be used. They are not the same thing, and confusing them is one of the most common mistakes in product design.