Topic – Testing Challenges in AI systems

AI Adventures: Tackling Challenges and Mitigating Risks

 Overview

Many software applications require engineers to write testing scripts, demanding skills on par with those of the developers who created the original code or application. This additional overhead in quality assurance is consistent with the growing complexity of software products. This can be addressed with the use of AI.

Over the Years, organizations have invested significantly in optimizing their testing processes to ensure continuous release of high-quality software. When it comes to AI, testing is more challenging owing to complexity of AI.

Testing a platform hosting AI framework is complex. So, we need to understand the challenges/Risks and find ways to mitigate it.

• Data Sourcing
  • The dynamic nature of the data makes AI model less promising as it may not give the output as expected based on the data used for training.
  • It is necessary to verify the quality of data, this includes data correctness, completeness and appropriateness along with the format check.
  • Verifying rules and logic applied on the raw data to get the desired output format will help the AI model
• Data Conditioning
  • Understanding the data needed for testing is the key
  • Incorrect data loads and data duplicates can cause issues, so data Injection testing should be carried out
  • Test data sets should be created based on requirements to train the Algorithm/ Model
• Algorithmic Uncertainty
  • Uncertainty arises from incomplete specifications or misunderstood requirements. In AI, instability is due to the randomness in model training, making testing strategies challenging.
  • The Algorithms must be tested by feeding multiple sets of input data and check if the output is desired.
  • Algorithms should be tested to check if there are no issues in the processing input data giving wrong outputs.
  • Check the cumulative accuracy of the output is True Positive (TP), True Negatives (TN), False Positive (FP) and False Negative (FN)