This is the fifth article in our six-part series on the core pillars of Enterprise Agentic AI. The journey so far has taken us through Safety and Trust (responsible agent design), Control (centralised oversight), Quality (performance monitoring and continuous improvement) and High Performing Hybrid Teams (Agentic Org Design). This fifth article, exploring what is required in operational terms in regard to both technology infrastructure and security, is written collaboratively by Rob Price, co-founder of Futuria, together with Adrian James, FBCS Chief Architect and author of #designed4devops, and Andrew Dixon, software and architecture engineer, both also at Futuria.
In an era where enterprises are increasingly exploring and adopting agent-enabled operating models, the underlying infrastructure is not merely a supporting element but a critical pillar. This whitepaper outlines our thinking that strategic enterprise AI adoption is inextricably linked to a holistic and proactive approach to infrastructure, which must support flexibility, control, and ongoing adaptability.
Alignment to Technology Strategy and Architecture
Every agent, agent team, and component within your agentic stack must align with the organisation’s broader technology strategy. Whether your strategy is Secure by Design, Cloud-First, or Hybrid, each layer of the agentic stack should be evaluated and validated for consistency with that architecture. This involves more than just checking boxes; it requires deep scrutiny of sourcing standards for third-party components and clear expectations around integration, resilience, and data handling. Without this rigorous evaluation, organisations risk introducing significant technical debt, creating security gaps, and fostering operational friction that can derail agentic AI initiatives before they gain traction. Successful deployment necessitates a foundational understanding of how new agentic capabilities fit within, and enhance, existing architectural patterns such as microservices or event-driven architectures, ensuring seamless integration and scalability.
Multi-Tenancy and Workload Isolation
Enterprise agentic systems require sophisticated isolation strategies to ensure agents operate securely and collaboratively, without interfering with each other. This becomes critical as organisations deploy multiple agent teams with varying security classifications and resource requirements.
Agent teams will handle sensitive workloads under delegated authority, processing PII or PCI data for example. They must support the constraints of the environments they work in such as isolated compute, segmented networks and federated identities.
Agents need to operate under the same constraints as humans and procedural systems do, using audited authentication and authorisation, limited network access and defined interfaces as required by security policies. Inter-agent communications must be treated in the same way as other systems using micro-segmentation of networks or encrypted communications across open networks.
Modern agentic infrastructure must break down traditional operational silos, enabling agents to work seamlessly across NetOps, SecOps, and application domains. This cross-domain capability is essential as agent teams increasingly need to correlate data and take actions that span multiple IT disciplines without the friction of tool switching or manual handoffs.
Our agentic teams must fit into the monitoring and auditing frameworks we use, including AAA (authentication, authorisation, and accounting) controls, network and data access and behavioural heuristics. As agentic features are included in ever-more products, we need to be able to identify unauthorised and rogue shadow agents operating within our organisations.
Keeping the Stack Current: AgenticOps and ModelOps
Enterprises must actively maintain their agentic infrastructure, understanding that this is not just about one-off launches, it’s about long term operational excellence. This means ensuring every component is patched, updated and tested regularly — not just for security but also to guarantee continued performance as environments and dependencies evolve.
Enterprise agentic systems increasingly demand real-time responsiveness, with AI “agents expecting sub-50 millisecond response times”. This creates new operational pressures beyond traditional batch processing, requiring infrastructure that can support continuous, always-on operations with machine-speed decision making.
A robust AgenticOps approach is crucial here. Whilst ModelOps practices govern model versioning, performance monitoring, and safe deployment of new agent capabilities (particularly with LLMs or SLMs), AgenticOps extends this to address the dynamic and potentially unpredictable nature of autonomous agents. This necessitates:
- Automated testing and performance benchmarking, applied both both pre-deployment and in live operation. The proactive maintenance is vital for mitigating risks associated with evolving dependencies and agent behaviours
- Continuous monitoring for latency, output quality, and degradation over time, which is especially important as environments generate increasingly overwhelming alert volumes. The unique challenge here is managing both the potential for emergent behaviours in autonomous systems and the critical need to maintain signal clarity amidst exponential alert growth, requiring sophisticated filtering and correlation tooling to ensure consistency and reliability.
Infrastructure Cost Optimisation
Managing the economics of agentic infrastructure requires sophisticated resource allocation strategies. Agent workloads can vary dramatically based on reasoning complexity and introduce new requirements into our hosting platforms including self-hosted models.
GPU vs CPU allocation strategies must align with agent types — analytical agents performing complex reasoning may require GPU acceleration, while simple transactional agents can operate efficiently on CPU-only infrastructure. Auto-scaling policies should account for the unpredictable nature of agent workload patterns, potentially scaling based on queue depth, reasoning complexity, or business priority rather than simple resource utilisation.
Storage tiering becomes crucial as agent systems generate vast amounts of training data, conversation logs, and intermediate reasoning states. Hot storage for active agent memory and reasoning, warm storage for historical context, and cold storage for compliance and audit trails must be balanced against cost and performance requirements.
Security by Design: Zero Trust and ABAC for Agents
As agent capabilities grow, so does the enterprise’s risk surface. Therefore, security must be embedded by design, not retrofitted. A Zero Trust approach is foundational: agents must not inherit open access to systems or data. Instead, Attribute-Based Access Control (ABAC) ensures that each agent is explicitly granted the minimum access and authority required to perform its task. This granular control, based on dynamic attributes and context, is critical for managing the complex interactions and permissions of an expanding agent ecosystem.
Security also extends to:
- Preventing prompt injection attacks, a unique vulnerability for language model based agents
- Tracking and validating all agent interactions
- Detecting and responding to anomalous behaviours across the environment.
These measures are vital for mitigating the new attack vectors that autonomous agents can introduce through their interactions and autonomy.
Agent Runtime Security
Beyond traditional access controls, agent runtime environments require specialised security measures to contain potential threats and limit blast radius. Sandboxing agent execution environments isolates agents from underlying infrastructure and from each other, preventing malicious or compromised agents from accessing unauthorised resources.
Runtime monitoring must track agent system calls, resource access patterns, and API interactions in real-time. This includes monitoring for unusual file system access, network connections to unauthorised endpoints, or attempts to execute system-level commands. Infrastructure-level prevention mechanisms should block agents from accessing unauthorised APIs or data stores, even if an agent somehow obtains valid credentials.
Container and VM security hardening for LLM/agent workloads requires specific considerations beyond standard practices. This includes restricting network egress to only approved endpoints, implementing strict resource limits to prevent resource exhaustion attacks, and configuring logging to capture all agent interactions for forensic analysis.
Specialised threats like prompt injection attacks require infrastructure-level defences, including input validation proxies, output sanitisation layers, and anomaly detection systems that can identify when an agent’s behaviour deviates from expected patterns.
Secrets and Credential Management for Agents
Agent systems require sophisticated credential management infrastructure to securely access the systems they need while maintaining the principle of least privilege. This goes beyond traditional service account management due to the dynamic and autonomous nature of agent operations. The agents are operating under the delegated permissions of the administrators that configure them and their users, though it would be fair to ask – as we did in Pillar 4 – to what degree these administrative rights could also be passed to an agent fulfilling that role.
Agents can be ephemeral in nature and require the use of vault systems supporting short-lived tokens with automatic rotation, ensuring agents never hold long-term credentials. Certificate management infrastructure should provision and rotate TLS certificates for agent-to-service communication, with automated revocation capabilities when agents are decommissioned or compromised.
Secure credential injection into agent runtime environments must occur without exposing secrets in container images, environment variables, or configuration files. This typically requires integration with enterprise secret management systems and runtime secret mounting.
Comprehensive audit trails must track credential usage across agent deployments, including which agents accessed which credentials when, and correlating this with agent actions and outcomes. Integration with enterprise identity systems (LDAP/Active Directory) enables centralised identity governance while supporting the distributed nature of agent deployments.
Hosting, Data Locality, and Performance
Agent workloads may span multiple platforms—public cloud, private cloud, and even on-premise infrastructure. Strategic hosting decisions must balance several critical factors:
- Security requirements such as data residency or classification
- Cost efficiency, especially with smaller, less compute-intensive SLMs. These models not only reduce operational costs but also enable new deployment models such as edge computing, which can be critical for applications requiring extremely low latency
- Performance and latency, particularly where agents rely on fast access to local data sources or APIs.
A flexible, modular agent architecture allows organisations to deploy agents in proximity to the systems or data they serve, reducing friction and overhead. This proximity directly impacts real-time decision making capabilities and can provide a significant competitive advantage.
Modern agentic infrastructure should support generative user interfaces that adapt dynamically to operational contexts, creating collaborative environments where human operators and AI agents can work together seamlessly. This includes supporting real-time interface generation based on current operational state, incident context, and team collaboration needs.
Edge and Hybrid Deployment Architectures
Many enterprises require agent deployments across multiple locations to minimise latency, comply with data residency requirements, or maintain operations during network disruptions. Edge computing infrastructure for agent workloads must balance local processing capabilities with centralised management and oversight.
Hybrid cloud strategies become complex when agents span on-premise and cloud environments. Some agents may need to run on-premise due to security requirements or data gravity, while others benefit from cloud scalability and managed services. The infrastructure must support seamless agent mobility between environments as requirements change.
Data synchronisation between edge agent deployments and central systems requires careful architecture to avoid conflicts while maintaining consistency. This includes synchronising agent knowledge bases, policy updates, and learned behaviours across distributed deployments.
Network architecture for agent-to-backend communication across locations must account for variable connectivity, bandwidth constraints, and security requirements. This may require local caching of critical agent resources, offline operation capabilities, and intelligent routing of agent requests to optimise for performance and cost.
Data Management, Lineage, and Classification
As agents ingest, process, and generate data autonomously, it’s essential to maintain a clear data lineage. This includes meticulously tracking:
- Where data originated
- How it was labelled, merged (particularly bridging of domain boundaries), or transformed
- How it was stored, classified, and accessed over time.
This transparency is not only critical for compliance, but also foundational for building trust and auditability in agent decisions. Clear data lineage directly facilitates tuning agents against the right data sets, ensuring their performance, accuracy and ethical alignment. Robust data management is the bedrock upon which reliable and trustworthy agentic systems are built.
Compliance and Lifecycle
Infrastructure must support the full lifecycle of agents—from deployment to decommissioning – and accommodate industry specific compliance requirements. For agentic AI, this means audit trails, policy enforcement, and forensic readiness must be embedded as part of the infrastructure itself, not retrofitted. Proactive embedding ensures that organisations can meet the evolving regulatory landscape surrounding AI, mitigating potential consequences of non-compliance. This integrated approach allows for continuous monitoring and rapid response to any compliance deviations throughout an agent’s operational lifespan.
Generative AI adds a new facet to supply chain security too. We need to be cognisant of the supply of models, the data they were trained with and the jurisdiction it was done under. As the market matures and commercial consolidation occurs, we need to manage this changing set of constraints and maintain our flexibility to be able to react and change model suppliers. This is one of the advantages of an agentic approach where there is flexibility in the selection of, or transition between, model suppliers.
Sustainability as a Strategic Imperative
We have looked at optimisation strategy already in the context of quality and speed – decisions around the models we select, and the team dynamics of the agents we aggregate. We will look at cost optimisation in the final of our six pillars in the next article. But before we leave this pillar on infrastructure, we also recognise the desire of some to optimise for sustainability.
Enterprises increasingly track energy usage and environmental footprint, looking to optimise workloads and avoid unnecessary computation. This is especially relevant as smaller-scale language models (SLMs) offer pathways to more efficient operations. By strategically optimising infrastructure and leveraging efficient models, organisations can achieve significant operational savings, reduce their environmental impact, and enhance their competitive differentiation.
Looking Ahead
With the infrastructure in place, the final consideration is commercial viability. How do organisations make these investments pay off? How do we balance innovation with control, or agility with financial discipline? Our final pillar—Commercial Governance and Value Realisation—explores the economic structures that ensure agentic AI delivers tangible, accountable outcomes at enterprise scale.
Note: This article was written with the assistance of ChatGPT and Claude for some content generation and refinement.
