Five cyber security imperatives from an AI-first organisation

By Vikas Tatwani

In an age where AI capabilities are evolving at unprecedented speeds, the challenges of securing these advanced technologies are equally sophisticated.

As a leader in this AI-first approach, we at Infosys have spent nearly a decade integrating automation into our operations becoming pioneers in the automation of cyber security operations. Along this journey we have learned some key lessons on how to tackle cyber-security in an AI-driven environment, below are five key imperatives that Australian business need to know when on their path to becoming a secure, AI-First organisation.

1. Manage The Exposure to Specific Threats

AI systems are susceptible to unique threats such as jailbreaks, prompt injections, and adversarial attacks. That’s why we developed robust methodologies to manage these risks. For instance, adversarial attacks, which subtly manipulate input data to deceive AI models, are countered through adversarial training. This involves exposing AI models to a wide range of manipulative scenarios during the training phase, thereby building resilience against such attacks.

We also employ network distillation during the model training phase. This technique combines outputs from multiple deep neural networks to reduce the susceptibility of AI models to minor disturbances, enhancing overall robustness.

2. Deploying a ‘Secure AI by Design’ Approach

 For cybersecurity to be effective in an AI-First environment a ‘Secure AI by Design’ approach needs to be deployed. A ‘Secure AI by Design’ methodology is akin to traditional cybersecurity practices like role-based access management and homomorphic encryption. This approach integrates security measures directly into the AI development process. By embedding security protocols from the ground up, it ensures that AI models are not just functional but also secure.

Our secure-by-design approach includes techniques such as differential privacy and anonymisation, which protect sensitive data while enabling AI systems to learn and evolve. It’s important to emphasise responsible AI hygiene measures, including fairness, transparency, and explainability, to ensure ethical and accountable AI deployment.

3. Red Teaming and AI Defense Platforms

Red teaming is a proactive measure involving simulated attacks to identify vulnerabilities in AI systems. We have institutionalised this practice, conducting continuous, real-world attack simulations to uncover potential security gaps. This ongoing exercise allows Infosys to stay ahead of emerging threats and refine our defense strategies.

In addition to red teaming, we have has built specialised AI defense platforms. These platforms consist of three layers:

1. Detection Layer: Identifies any form of attack on the AI system.
2. Response Layer: Provides real-time responses to detected attacks.
3. Threat Management Layer: Enables audit, observation, and telemetry, and accesses a threat database to understand and mitigate attacks.

These platforms are crucial in defending against various AI-specific threats, such as evasion, inference, poisoning, and backdoor attacks.

4. Manage Security Risks in Mission-Critical Systems

AI’s integration into mission-critical systems, such as autonomous driving, electric utilities management, and healthcare, presents unique security challenges. Its imperative organisations address these by employing stringent risk management protocols.

At Infosys for example, in the context of autonomous vehicles, we ensure that AI models undergo rigorous testing to handle edge cases and corner scenarios that could pose safety risks.

Similarly, in healthcare, we prioritise the accuracy and reliability of AI systems used for diagnostics and treatment.  We do this by implementing a robust validation process to ensure that AI models can accurately classify medical conditions, thereby reducing the risk of misdiagnosis or mistreatment.

5. Meeting compliance standards of AI

As per a Forrester study, 4 out of 10 of enterprise AI decision makers report that regulatory and compliance as one of the barriers to adoption of generative AI. Generative AI, capable of creating realistic text, images and videos, pose significant privacy and security risks resulting in enterprises paying large fines to regulatory authorities for non-compliance.

Splunk’s State of Security 2024 report highlights concern about Australia’s cybersecurity landscape, amid a high rate of AI adoption. On the positive side, Australia leads in GenAI technology adoption and policy formulation, with 69% using public GenAI tools and 73% having established security policies. However, 65% of business leaders lack a full understanding of GenAI’s implications, indicating a significant knowledge gap. Despite GenAI adoption, 34% of organisations lack a GenAI policy, and there is no clear consensus on whether GenAI favors cybersecurity defenders or threat actors.

We have developed guardrails for generative AI models to combat this threat. These guardrails screen input prompts for various prompt injections and jailbreak attacks, disabling malicious attempts without human intervention.

A Strategic approach to an AI-first security

AI is both a defensive and offensive tool, capable of detecting threats, automating responses, and enhancing human potential in cybersecurity roles. However, it also cautions against challenges AI poses, like enabling sophisticated cyber-attacks and spreading misinformation through generative AI. Therefore, when using AI to boost enterprise cybersecurity, a strategic approach is needed. Thus, strategy should be based on three principles: Secure by Design, Secure by Scale, and Secure the Future, this will ensure AI solutions are comprehensive, scalable, and future-ready.