AI ACI Worldwide communication

At ACI we make responsible use of our machine learning models by demystifying the information from traditionally black-box models and presented in simple ways that bring trust to our customers and that can be easily used to understand what is considered suspicious on each specific transaction.

With machine learning models is important to get a balance between explainability and accuracy. To achieve high accuracy on real time fraud detection and to ensure the models will maintain that high accuracy over long periods of time then complex models are needed, however, explainability requires simplification of the models to be able to point to the underlying reason behind the model’s response. Our Model Explainer approach is an innovative way of keeping the complexity of the models without any sacrifice to accuracy but still getting actionable insights on why the model consider the transaction unusual. Model explainability techniques can be very computationally expensive. To accurately identify what specific transaction detail is considered unusual in an individual transaction a sensitivity analysis is required and with complex models using large number features and analysing millions of transactions in real time, a sensitivity analysis is not only expensive but also limited to a very narrow time window.

To achieve efficient explainability our on-demand innovative approach uses approximations of local interpretation methods, based on Shapley values approximations that are able to identify the most influential features for individual transaction and report back in only a few seconds. This implementation is based on a sensitivity analysis that re-scores the transaction with different feature values using a limited features set combined with a carefully chosen sample of pre-scored transactions to get an accurate estimation of the importance of each feature value on the specific transaction in a very fast an efficient way.

Direct implementation of the Shapley method involves rescoring input data using all feature combinations that could be used by a model with all possible feature values and measuring the impact each feature has on the final score, by measuring difference in results between models with and without that feature, for all models, for every candidate feature. This method has not been previously used for commercial applications due to the complexity and high cost of the calculations. However, implementing an approximation of the method by limiting the features included in the sensitivity analysis and leveraging the power of an accurate pre-scored sample made possible to achieve highly accurate results in just seconds.

The method returns a configurable N number of features that the model considered most important to make the determination of Fraud/No-Fraud along with a level of importance of each feature. That information is highly valuable to understand “why” the model returned the particular score and provide actionable insights when doing a further on what exact details are the most important for each transaction.