Skip to main content
Blog

Part of our Software supply chain security: review, investigate, respond.

Threat Intelligence

LiteLLM Supply Chain Attacks: Exposure and Response

Investigate LiteLLM supply chain exposure with affected-version checks, build evidence, credential scoping and a recovery worksheet for AI teams.

Published Updated 7 min 1,412 words 8 sectionsBreachline Labs

The March 2026 LiteLLM compromise put malicious code into legitimate PyPI releases. For an AI team investigating exposure, the decisive evidence is what a build installed and executed, which credentials that environment could reach and what happened afterwards. Today's dependency version alone cannot answer those questions.

This article was originally published after the March incident. The 23 October revision adds a separate October incident notice and an investigation worksheet. The worksheet is practical response guidance, not a claim that BreachLine investigated affected organisations or measured the number of victims.

First identify which LiteLLM incident you are investigating

There are two distinct 2026 incidents to account for. Keep their release lists and artifact evidence separate.

IncidentAffected artifacts identified by the maintainersWhat to verify in your environment
24 March 2026PyPI releases litellm==1.82.7 and 1.82.8Historical dependency resolution, installed distributions, build records and Python execution
21 October 2026PyPI release 1.107.0 and affected Docker images under v1.107.0, latest, main-latest and main-stableThe downloaded wheel or actual container digest, pull time and execution history

The March advisory says the official LiteLLM Proxy Docker deployment was unaffected by that incident. That exception must not be carried forward to October: the 21 October advisory explicitly includes compromised container artifacts. Consult its complete digest list and incident-specific response instructions.

Datadog's October investigation describes the later release as a second compromise and says no connection between the incidents had been established at publication. Do not assume the March attribution explains October.

What the March package changed

LiteLLM's maintainers identified malicious code in proxy_server.py in both affected March releases, with an additional litellm_init.pth file in 1.82.8. The official update describes credential collection and outbound exfiltration, including secrets available through environment variables and configuration files.

The .pth mechanism changes the investigation. Python's site documentation explains that executable import lines in processed .pth files run at interpreter startup, even if the application never imports the associated package. A search for import litellm in application code therefore cannot establish that an affected environment was safe.

Datadog's March analysis recommends treating affected hosts and CI jobs as credential-exposure events and investigating persistence, outbound traffic and Kubernetes activity. That is a broader task than replacing one requirement entry.

Do not start a suspect Python environment to inspect it

When startup hooks are part of the incident, running the environment's own python or pip may execute malicious code again. The October maintainers' instructions explicitly warn against starting with pip install. Preserve the relevant artifacts and inspect a copy with trusted tooling outside the affected environment before choosing a cleanup method.

For a first pass, use existing build logs, package manifests, container metadata and filesystem evidence. Avoid importing the suspect package as a version check. An incident responder can determine how to collect volatile evidence or examine a running host without destroying useful state.

A dependency scanner can help locate a known bad version. A negative scan after an upgrade cannot prove that credentials were never exposed or that persistence was removed.

Build an exposure ledger from artifact to credential

Create one row per environment or build that may have consumed an affected artifact. Include developer machines, short-lived CI runners, image builds and deployed workloads. A dependency may have arrived transitively even if the application did not list LiteLLM directly.

FieldEvidence to collectQuestion it answers
Environment and ownerRunner ID, workload, host or image digestWho can contain and rebuild it?
Resolved packageHistorical lockfile, install log, wheel metadata or image contentsWhich artifact was actually present?
TimelineInstall, pull, startup, deployment and removal recordsWhen could the code have run?
Reachable secretsRole permissions, mounted files, environment configuration and secret-store accessWhat could that process read or request?
Network and identity activityDNS, egress, cloud audit, registry and identity-provider logsWas there suspicious communication or credential use?
Recovery evidenceReplacement artifact, revoked credentials and validation recordsWhat demonstrates that the exposure path was closed?

Separate observed facts from unknowns. “The lockfile referenced an affected version” establishes a dependency decision; it does not by itself establish successful exfiltration. “The runner executed Python with the affected distribution installed” supports a stronger exposure concern. “No network logs retained” is an evidence gap, not a clean result.

Container tags need particular care. A tag can point to a different image when you inspect it later. Record the digest that was pulled or executed, and compare it with the advisory's artifact list. Preserve the build identifier so that any downstream images can be traced back to the same inputs.

Contain the environment and the credentials it could reach

Use the ledger to organise response work. The following sequence is a proposed coordination checklist; the incident-specific advisory and your response team's evidence-preservation requirements should determine the exact actions.

  1. Restrict the suspected environment. Stop it from receiving new workloads or retrieving fresh secrets while preserving the evidence needed for investigation. Record which jobs and deployments depended on it.
  2. Establish the credential boundary. List the credentials present and the permissions that could retrieve more. Include build and publication tokens as well as LLM provider keys.
  3. Revoke or rotate from a trusted environment. Do not put replacement secrets back into an environment that may still execute malicious code. Check for sessions or derived credentials that require separate revocation.
  4. Rebuild from reviewed inputs. Replace affected environments and downstream artifacts using a verified release appropriate to the current advisory. Record the new dependency set and artifact digests.
  5. Review identity use. Investigate actions taken with exposed credentials, including changes to repositories, registries, cloud resources and access policies. Escalate unexplained activity rather than closing the incident when the package disappears.
  6. Validate the replacement. Confirm that the application works, the expected artifact is deployed and the old credentials no longer grant access. Keep a named owner for unresolved evidence gaps.

This avoids a common recovery mistake: changing the package version while leaving the same compromised build environment or credentials in place.

Fix the dependency process without promising perfect prevention

Pinning and integrity checks reduce unexpected changes, but neither establishes that a selected release is trustworthy. A hash verifies that an artifact matches the reviewed bytes. If the approved artifact was malicious, a matching hash faithfully installs those malicious bytes.

pip's secure-install guidance documents hash-checking mode and requires hashes for all resolved requirements, including dependencies. For a reviewed, fully pinned requirements file containing approved hashes, the installation command is:

python -m pip install --require-hashes -r requirements.txt

Run that only in the trusted replacement environment. The requirements file must include the actual approved artifact hashes; this is not a shortcut for selecting a safe LiteLLM version.

Review changes to package sources, version constraints and dependency files before merging. The requirements review tool and supply-chain review tools can support that static review. They do not inspect a historical runner's memory, prove that a credential was never stolen or replace host investigation.

Separate build, test and publishing permissions so a dependency executing in one job does not automatically inherit every release secret. Keep enough artifact and identity logs to reconstruct the next incident. Our software supply-chain whitepaper develops the trust-boundary approach; the Axios incident analysis provides the related npm case.

Keep application testing and incident recovery distinct

After containment and recovery, test the security boundaries of the restored application: authentication, authorisation, secret access and exposed service configuration. That work answers how the application behaves now. It does not retrospectively certify that an old build environment was uncompromised.

If you use Nebula for that follow-up, give it a defined environment and task and review the evidence it returns. The AI penetration testing guide explains the assessment workflow. Package review, incident response and application pentesting contribute different evidence; a sound response records what each one established.

Sources

Incident sources checked on 23 October 2026. Use the maintainers' current advisory when selecting replacement releases.

Continue reading

Related articles.