Cybersecurity researchers warn that threat actors are actively exploiting a “disputed” and unpatched vulnerability in an open-source artificial intelligence (AI) platform called Anyscale Ray to hijack computing power for illicit cryptocurrency mining.
“This vulnerability allows attackers to hijack companies’ computing power and disclose sensitive data,” Oligo Security researchers Avi Lumelsky, Guy Kaplan and Gal Elbaz said in a Tuesday disclosure.
“This flaw has been actively exploited over the past seven months, affecting industries such as education, cryptocurrencies, biopharmaceuticals and more.”
The campaign, which has been running since September 2023, has a code name ShadowRay from the Israeli Application Security Company. It also marks the first time that AI workloads have been targeted due to underlying flaws in the AI infrastructure.
Ray is a fully managed, open source computing framework that enables organizations to build, train, and scale AI and Python workloads. It consists of a core distributed runtime and a set of AI libraries to simplify the ML platform.
It is used by some of the largest companies, including OpenAI, Uber, Spotify, Netflix, LinkedIn, Niantic, and Pinterest, among others.
The security vulnerability in question is CVE-2023-48022 (CVSS Score: 9.8), a critical missing authentication bug that allows remote attackers to execute arbitrary code via the Job Submission API. It was reported by Bishop Fox along with two other defects in August 2023.
The cybersecurity firm said the lack of authentication controls in two Ray components, Dashboard and Client, could be exploited by “rogue actors to freely submit jobs, delete existing jobs, retrieve sensitive information, and achieve command execution remotely”.
This makes it possible to gain operating system access to all nodes in the Ray cluster or attempt to recover the Ray EC2 instance credentials. Anyscale, in a notice published in November 2023, said it does not plan to fix the issue at this time.
“The fact that Ray does not have built-in authentication is a long-standing design decision based on how Ray’s security boundaries are drawn and consistent with Ray’s best deployment practices, although we intend to offer authentication in a future as part of a defense-in-depth strategy plan,” the company noted.
The documentation also warns that it is the platform provider’s responsibility to ensure that Ray runs in “sufficiently controlled network environments” and that developers can access Ray Dashboard securely.
Oligo said it observed the shadow vulnerability being exploited to breach hundreds of Ray GPU clusters, potentially allowing threat actors to gain access to a variety of sensitive credentials and other information from compromised servers.
This includes production database passwords, private SSH keys, access tokens related to OpenAI, HuggingFace, Slack, and Stripe, the ability to “poison” templates, and elevated access to cloud environments from Amazon Web Services, Google Cloud, and Microsoft Azure.
In many cases, infected instances have been found to be hacked with cryptocurrency miners (e.g., XMRig, NBMiner, and Zephyr) and reverse shells for persistent remote access.
The unknown attackers behind ShadowRay also used an open source tool called Interactsh to fly under the radar.
“When attackers get their hands on a Ray manufacturing cluster, it’s a jackpot,” the researchers said. “Valuable corporate data and remote code execution make it easy to monetize attacks, all while remaining in the shadows, totally undetected (and, with static security tools, undetectable).”