Quarterly journal published in SPbPU
and edited by prof. Dmitry Zegzhda
Peter the Great St. Petersburg Polytechnic University
Institute of computer sciences and technologies
information security of computer systems
Information Security Problems. Computer Systems
Published since 1999.
ISSN 2071-8217
ENSURING THE STABILITY OF ONLINE LEARNING ARTIFICIAL INTELLIGENCE SYSTEMS BASED ON MODEL SIMILARITY ASSESSMENT
K. A. Tsibulskas1, V. M. Krundyshev1, M. O. Kalinin1
Annotation: The paper studies the problem of protecting artificial intelligence systems with online learning from poisoning attacks. To improve the stability, an approach is proposed based on assessing the similarity of the operation of two computational models: the reference (initial) and the operational (test). The following indicators of stability violation were identified: a decrease in the total accuracy (TA), total prediction value (TPV), and a decrease in the cosine similarity of model weights (cos_similarity). As a result of experimental study, it was found that the proposed solution allows for timely detection of poisoned data, maintaining high classification accuracy during targeted attacks on the computational model, which is further trained on test data.
Keywords: Poisoning attack, artificial intelligence security, online learning, model similarity assessment
Pages 168–178