Eclypsium claims AI server hardware can be tampered to alter chatbot answers
Eclypsium says attackers with network access to DPUs or BMCs could alter AI model weights in memory; restrict that access and enable memory protections.
- Published
- Exploitation
- No, Eclypsium describes a research demonstration and the source reports no use in real attacks
What happened
Security firm Eclypsium says it can tamper with the hardware that runs AI models so that chatbots give faulty and dangerous answers. The account comes from a CNN report on the research, republished on Eclypsium’s site. No independent party has confirmed that this method has been used outside of research Eclypsium.
According to the report:
- Changed model behavior: The researchers changed the “weights” a model uses by tampering with them remotely.
- In one test, this sharply raised the probability the model gave to a Chinese invasion of Taiwan.
- In another, a model running on the tampered hardware told a user with chest pain to stay home instead of going to the hospital.
- Platform tested: Eclypsium ran the demonstration on an Nvidia computing platform. The researchers say the approach is not unique to Nvidia and could apply to other hardware and the code it runs Eclypsium.
The report gives these conditions for the attack:
- The attacker generally needs network access to a data processing unit (DPU) or a baseboard management controller (BMC).
- Certain security configurations must be disabled.
- With access to the DPU, an attacker can read and change code and data in the machine’s memory.
- The researchers say the attack takes less than a minute and the AI user does not notice.
The report also says vulnerabilities in BMCs disclosed this summer by researcher HD Moore gave a way into the hardware in Eclypsium’s research. The specific vulnerabilities and any CVE IDs are not named in the source, so they are not confirmed here Eclypsium.
Responses quoted in the report:
- Nvidia: A spokesperson said the company “did not identify anything the industry wasn’t previously aware of” and that strong memory access protections can block the attack path.
- Eclypsium: The firm said the effectiveness of security controls “depends on how they are configured and used” and that it has been working with Nvidia on them.
- Government: Eclypsium executives said they briefed the NSA and CISA.
- Outside experts: Two independent hardware security experts who reviewed the findings for CNN called them credible Eclypsium.
Am I affected?
- Products and versions: Not confirmed. The source names no affected models, firmware versions or product lines beyond an Nvidia platform used for the demonstration.
- Who would be exposed: Based on Eclypsium’s described conditions, exposure would depend on two things: whether an attacker can reach your DPUs or BMCs over the network, and whether the relevant security configurations are disabled Eclypsium.
- Hosted AI services: If you only use an AI model through a hosted provider, the source does not say whether any provider is affected. CNN said it asked Anthropic and OpenAI about the research, and no answer is reported.
- Real-world exploitation: None is reported in the source.
What to do now
- Check which networks can reach your servers’ BMCs and DPUs, and restrict that access. This is the attack path Eclypsium describes Eclypsium.
- Confirm that memory access protections are enabled on AI hosts. Nvidia’s spokesperson said strong memory access protections can block the attack path Eclypsium.
- Review how your hardware security controls are configured, not just whether they exist. Eclypsium says their effectiveness depends on configuration and use Eclypsium.
- Watch for vendor advisories on the BMC vulnerabilities mentioned in the report. Their identifiers and fixed versions are not confirmed in the source.