A new attack surface

Conventional security testing assumes software does what its code says. A machine learning model does not work that way. Its behavior is learned from data, expressed through millions of parameters, and shaped at runtime by whatever input it receives. There is no line of code to read that tells you how the model will respond to an input no one anticipated.

That creates failure modes a code review or a penetration test will not surface. The data a model learned from can be tampered with. The model itself can be copied or reverse-engineered. The instructions it follows can be hijacked by ordinary-looking text. An AI security audit examines the system as a whole: the data, the model, its inputs, its outputs, and the actions it is permitted to take.

Where risk enters the AI lifecycle

An AI system is a pipeline, not a single program. Data is collected and prepared, a model is trained or fine-tuned, the model is deployed behind an interface, and its output flows into other systems. Risk can enter at any stage, and a weakness early in the pipeline often becomes a larger problem once the system reaches production.

Mapping that full lifecycle is the first step of an audit. It shows where untrusted data enters, where a third party is trusted, where sensitive information is exposed, and where the model's output is acted on without review.

• Training and fine-tuning data, where poisoning and backdoors are introduced
• The model, which can be copied, inverted, or made to leak its training data
• Inputs at runtime, the vector for prompt injection and evasion
• Outputs, which become dangerous when passed unchecked into other systems
• Integrations and agents, where the model is allowed to take real actions
• Third-party models and data, which carry risk you did not create

Poisoned data and untrusted models

Training data poisoning plants malicious examples in the data a model learns from, so the model behaves normally almost everywhere and fails only on the inputs an attacker chooses. Published research has planted a working backdoor by altering a few hundred images out of three million, and has shown that a model trained to misbehave on a specific trigger can pass standard safety testing without revealing the flaw.

For a bank or an exchange, this matters most in the models that decide what is normal. A poisoned fraud, anti-money-laundering, or market-surveillance model can be shaped to overlook the precise activity an attacker intends to run. Because most institutions build on third-party base models, datasets, and libraries, provenance becomes part of the audit: what went into a model, where it came from, and whether it can be trusted.

Evasion: a change too small to see

An adversarial example is an input altered just enough to change a model's decision while looking unchanged to a person. The change is computed against the model itself, so the result is deliberate rather than random.

This is the attack that matters most for detection. A fraud, anti-money-laundering, or surveillance model exists to separate normal activity from abnormal, and an evasion attack is engineered to push the abnormal back across that line. The same technique has fooled image classifiers, malware detectors, and intrusion-detection systems in published research.

How a Modulus audit works

An engagement begins by inventorying your AI systems and the data, models, and integrations behind them, then building a threat model specific to how each system is used. From there our team tests the system the way an adversary would, combining manual review with adversarial techniques drawn from published research and the MITRE ATLAS knowledge base.

Every finding is mapped to a recognized framework, rated by impact and likelihood, and paired with a concrete remediation step your engineers can act on. The result is not a checklist. It is a clear account of how your AI can be misused and what to change first.

• Inventory and threat modeling of each AI system in scope
• Review of training data, fine-tuning, and model provenance
• Adversarial testing for prompt injection, evasion, and data leakage
• Assessment of agent permissions, tool access, and output handling
• Review of third-party model and data dependencies
• We measure against OWASP, NIST AI RMF, and ISO/IEC 42001

The risks an audit covers

Modulus tests AI systems against the recognized catalog of large language model and machine learning risks, including the OWASP Top 10 for LLM Applications. Each item below is a documented failure mode, not a hypothetical.