Prevention of adversarial Attacks on machine Learning Models

Machine Learning (ML) is a key enabler of Smart Cities. ML allows to tackle the challenging size and complexity of Smart Cities’ systems, providing efficient ways to process sensor data and monitor in real time the state of critical infrastructure such as transport, power grids and water mains. Given the reliance of Smart Cities on ML for efficient managing and monitoring of the city’s infrastructure, their security hinges on the robustness of the ML models employed. Adversaries can exploit this reliance and target the city’s ML models to indirectly subvert the critical systems and compromise the safety of the population. Due to the dependence of ML models on sensor data and the inherent exposure of sensors to remote and physical attacks, compromised sensors are a prime threat vector to attack smart systems. By manipulating sensor data, attackers can poison the city’s model, preventing it from learning the correct correlation between data and the state of a critical system (opening the doors to stealth attacks), or they can trick the model into taking actions that are harmful for the city’s infrastructure and population.

To address these issues and enable the secure application of ML to smart cities, we propose a comprehensive framework to improve the resilience of ML methods. Our proposed framework is based on a combination of ensembles of ML models, collaborative training and secure edge deployment, which will provide flexible and resilient application of ML to the management of Smart Cities. We propose an innovative approach to ensemble models that will allow our system to (1) autonomously heal from adversarial poisoning and (2) constantly increase in performance over time. Furthermore, we plan to extend our ensemble model to the collaborative scenario, where multiple entities collaborate to train a shared model, without sharing their private dataset. This allows our framework to provide not only resilience, but also flexibility, allowing separate entities to reach a common goal (building a high quality ML model), without disclosing sensitive data. Finally, our framework will explore model distillation and compression in general, giving us the ability to deploy our models on the edge in a secure manner. This will allow our framework to provide a resilient monitoring infrastructure in situations where data cannot be transferred and needs to be processed on site due to various constraints.