Open models create a security supply chain of their own

AI adoption expands the set of third-party artifacts

NIST identifies provenance and AI supply-chain risk as part of adversarial machine-learning security. The UK NCSC’s secure-AI development guidance calls for models, data, libraries, frameworks, modules, middleware, and external APIs to come from verified and well-documented sources.

A trusted repository name is not complete provenance

A deployment may combine a base model, community adapter, tokenizer, custom inference code, native dependencies, and a container image. Weakness or tampering at any layer can affect confidentiality, integrity, or model behavior. Some model-loading formats may also execute code.

Extend software supply-chain controls to model assets

1. Approve sources and licenses

   Define allowed registries, publishers, artifact types, and usage conditions.

2. Record provenance

   Capture origin, version, hash, model card, dataset lineage, adapters, dependencies, and evaluation results.

3. Verify before loading

   Check signatures or hashes and prefer non-executable serialization formats where possible.

4. Evaluate in isolation

   Use a sandbox without production secrets, broad network access, or write access to trusted registries.

5. Control promotion and rollback

   Require review before production use, monitor behavior after release, and retain known-good versions.

6. Continuously reassess

   Track upstream changes, vulnerabilities, ownership changes, and newly identified behavioral risks.