AI Products need to be transparent and explainable.
AI products must have trust at their core and one of the pillars of trust is transparency and explanation. High trust AI products have lower churn and higher retention so evaluating the trust of your product is super important. There are a number of strategies you can implement which I will talk through in this newsletter to help your users trust your product. To help communicate and demonstrate some of the strategies in this newsletter I've designed and built a demo project called Onit which is a mobile personal assistant helping you manage your personal and work life in one place.
The three strategies I'll cover build on each other: how sure the AI is, how it reached its conclusion, and what information it drew on to get there. Let’s get into it. A common problem with AI models in particular LLMs is they will produce incorrect results but present them as facts in a confident manner which has obvious negative consequences for your users such as them taking the wrong actions, believing misinformation and in mission-critical systems can lead to extremely dangerous situations including loss of life. This leads me to the first strategy to make your AI product and system transparent: Confidence Signalling, which helps you surface AI uncertainty visibly and consistently. A 60% confident output should never be presented identically to a 99% confident one. You can implement this in various ways using language, visual weight, or explicit probability scores to communicate this distinction.
In the Onit demo I’ve implemented confidence signalling to show how confident the AI assistant is of the urgency of the emails or tasks. This is a fairly harmless example but enables you to see that it is embedded directly into the product, it is not an add-on but part of the experience. The confidence is signalled with the bar icons (three bars is certain and one bar is unsure) so it is very easy for the user to see the amount of confidence Onit has with ranking the urgency at a glance.
This example also embodies a second strategy called Decision Origin which enables the users to trace the reasoning on how the AI came to its conclusion in a very easy and digestible way. It should show people why the AI reached a conclusion not just what it concluded. Ideally this should happen in real-time to give people the ability to act. Tracing the reasoning path is critical for solutions in regulated industries. This can be a very powerful design strategy to employ for two key reasons: one it gives the users an understanding of where the AI may have gone wrong so they can provide concise feedback to improve future use and secondly it greatly improves trust in the output of the AI.
The next strategy that dramatically improves trust is Source Attribution, when AI draws on source material, show it. Link to it. Where Decision Origin shows the reasoning, Source Attribution shows the sources used to make a decision or come to a conclusion.. Do not collapse the reasoning chain into a confident-sounding summary with no traceable origin. I’ve written some code to adapt my Onit designs to include the sources for tasks and emails to justify why emails and tasks are the set level of urgency. The goal being users have high trust in the system so they don’t need to leave the app.
A caveat across all three strategies is they will not work if they are dishonest, which is you need to build in lots of testing and logging into your AI Products and Systems. Trust is earnt and lost very quickly which has massive commercial consequences for AI products and systems. None of this makes the model less of a black box assuming you're using frontier cloud solutions. It just puts windows in it: one for how sure it is, one for how it got there, one for what it drew on.
I hope this helps.
See you soon,
Harry