Artificial intelligence is becoming a source of value and a new concentration of risk. Models can expose sensitive information, amplify unsafe instructions, create convincing but incorrect output, or be manipulated through their data, tools and interfaces. Securing AI therefore means protecting more than an algorithm: it means controlling the complete system around it.
This post in a series on security at all layers focuses on AI security. I’ll look at protection across three stages: Foundations, Development, and Deployment, and in three ways: Governance, Protection, and Resilience.
As with all solutions, the starting point is to understand the AI use case and the business value, in addition to the decisions it will influence and the harm that would be caused if it is wrong, unavailable or misused. A low-risk assistant that drafts internal text should not carry the same controls as a model that handles personal data, changes production systems or affects a person’s access to a service. Classification of the solution keeps controls proportionate while preventing convenience from becoming the default risk decision.
I like to think of an AI system as an apprentice operating inside a workshop. It may be highly motivated and text book knowledge, but it still needs suitable materials, clear instructions, restricted tools, supervision and a safe way to stop. The greater the consequence of a mistake, the stronger and more independent those controls must be.
Foundations
As with any solution, an AI system inherits the strengths and weaknesses of its foundations. For AI this is the source data, prompts, model, retrieval content, dependencies and hosting platform. If any of these inputs are untrusted, poorly understood or over-privileged, later safeguards can only limit rather than remove the risk. When looking at AI and security it is even more critical to get the other layers right.
Foundations Governance
Systems should record the intended purpose, accountable owner, affected users, data classifications and prohibited uses before begining to design implementation or selecting a model. Through the SDLC it is critical to maintain an inventory of models, datasets, vector stores, prompts, agents and external services, including their origin, licence, approval status and retirement date. In addition there should be defined, measurable, quality and safety requirements and these should require independent review when the system can materially affect people, money, regulated data or critical operations.
Foundations Protection
Systems should treat training data, model weights, system prompts and embeddings as sensitive assets, but also an asset prone to attack or manipulation. Solutions should seek to verify the provenance, scan inputs, remove unnecessary personal or secret data, and isolate untrusted sources. As with data security, it is important to encrypt assets, control access through workload identities, and keep model and data permissions separate. In addition there is a need to validate third-party models and packages, pin approved versions, and protect the supply chain so any unreviewed components cannot silently enter production.
Foundations Resilience
Ensure that you preserve approved datasets, prompts, model versions and configurations, so a known-good system can be reconstructed. In addition seek to avoid dependencies on single model endpoints, or inferance infrastructure where availability is critical, and define degraded modes that do not bypass safety rules so systems can continue to operate. It is also critical to test restoration including both provider failure and model withdrawal, including whether the system can revert without losing audit evidence or exposing data to an unapproved alternative.
Development
Development is where general capability becomes a particular system. Fine-tuning, retrieval, prompt templates, tools and application logic shape what the model can see and do. Security must therefore be part of experimentation, testing and release rather than a gate added after the behaviour has already been designed.
Development Governance
Use documented acceptance criteria for accuracy, privacy, security, fairness, explainability and human oversight. Record experiments, model and prompt versions, test evidence, known limitations and approval decisions. Changes to data, tools, retrieval sources or safety controls should trigger proportionate reassessment. Keep development, validation and approval responsibilities separated for higher-risk systems.
Development Protection
Keep secrets and production data out of notebooks, source code, prompts and test logs. Isolate build environments, scan dependencies and artefacts, sign approved releases, and restrict who can alter system prompts, guardrails and evaluation sets. Test for prompt injection, data leakage, insecure tool use, excessive agency, model extraction and denial-of-service conditions. Treat model output as untrusted input before it reaches code, users or downstream systems.
Development Resilience
Build repeatable pipelines so an approved release can be reproduced and rolled back. Test with malformed, adversarial and out-of-distribution inputs, not only expected examples. Set token, time, cost and concurrency limits to contain runaway behaviour. Preserve evaluation results and release artefacts so investigators can distinguish a model defect, poisoned input, configuration error and platform failure.
Deployment
Deployment is where AI behaviour meets real identities, data and business processes. The system may receive hostile instructions, retrieve sensitive records, call tools or produce output that people trust too readily. Controls should limit the model’s authority, make its actions attributable and ensure that consequential decisions remain appropriately supervised.
Deployment Governance
Tell users when they are interacting with AI, what the system is intended to do and where its limitations require verification. Assign operational ownership, define human approval points, and provide routes to challenge important outcomes. Monitor whether actual use matches the approved purpose, review incidents and complaints, and withdraw or redesign the service when risk exceeds the accepted boundary.
Deployment Protection
Authenticate users and services, authorise every data retrieval and tool call, and give agents only the minimum permissions needed for the current task. Separate instructions from untrusted content, validate inputs and outputs, and require confirmation before destructive, financial or externally visible actions. Filter sensitive data from prompts and logs, apply rate limits and abuse detection, and record the model, prompt, identity, retrieved context, tool calls and policy decisions needed for investigation.
Deployment Resilience
Monitor quality, safety, latency, cost, access patterns and changes in input or output behaviour. Provide kill switches, circuit breakers and safe fallback paths that do not silently grant broader access. Exercise incidents involving prompt injection, model compromise, provider outage, data leakage and harmful output. Recovery should restore a known-good model and configuration, preserve evidence, notify affected owners and verify that queued or repeated actions cannot cause further harm.
AI security is not a single product or a final test. It is the discipline of keeping purpose, data, models, identities, tools and operations inside a defined boundary throughout the system’s lifecycle. Strong foundations reduce uncertainty, secure development exposes weaknesses early, and controlled deployment ensures capability never becomes unchecked authority.
In line with Moore’s law, both AI capabilities and risks are evolving at an ever increasing rate. As such it is even more critical to ensure that security and governance of all elements of the SDLC. AI has the capability to perform complicated task, but as seen recently equally has no ethical compass.
If you want to know more about AI Security on AWS there is a great course on AWS Skill Builder. Security Compliance and Governance for AI Solutions covers security, compliance, and governance issues related to AI solutions and includes learning about AWS services for governance, compliance, and privacy to help secure AI workloads.
comments powered by Disqus