February 28, 2024 at 05:16PM
JFrog’s security team detected around 100 malicious AI ML models on the Hugging Face platform, potentially giving attackers persistent backdoor access to victims’ machines. Despite Hugging Face’s security measures, the models evaded detection, indicating significant security risks. JFrog urges heightened vigilance and proactive measures to protect against such threats.
Based on the meeting notes, here are the key takeaways:
1. Hugging Face, a tech firm specializing in AI, NLP, and ML, has had at least 100 instances of malicious AI ML models hosted on its platform.
2. Despite Hugging Face’s security measures such as malware, pickle, and secrets scanning, malicious models have managed to evade detection.
3. JFrog’s security team found roughly 100 models with malicious functionality on Hugging Face, including a case of a PyTorch model with a payload that could establish a reverse shell to a specific IP address (210.117.212.93).
4. JFrog developed an advanced scanning system to identify malicious models, excluding false positives and ensuring a genuine representation of efforts towards producing malicious models for PyTorch and Tensorflow on Hugging Face.
5. Furthermore, JFrog discovered evidence that some malicious models’ operators may be AI researchers, posing a risk to the ecosystem.
6. JFrog suggests that some of the malicious uploads might be part of security research aiming to bypass Hugging Face’s security measures, but the risk posed by these models should not be underestimated.
These takeaways highlight the significant security risks posed by malicious AI ML models and the need for elevated vigilance and proactive measures to safeguard the ecosystem.