Artificial intelligence has become a defining force in the digital economy, enabling new efficiencies, innovation, and growth. Yet alongside these opportunities, AI has fundamentally reshaped the cyber threat landscape. As digital connections multiply and systems become more autonomous, organisations and governments face a rapidly expanding attack surface that demands new approaches to cybersecurity.

At our flagship AIMX Singapore conference in 2025, a panel of experts discussed at length what hyper-accelerated cybersecurity threats - enabled by AI - mean for organisations in terms of preparedness and resilience.

The panel discussion was moderated by ⁠Rajesh Sandhu⁠ from Mastercard, ⁠Veronica Tan⁠ from Singapore's Cybersecurity Agency, ⁠Christopher Chew⁠ from Cisco, ⁠Dr Jonthan Pan⁠ from Singapore's Home Team Science & Technology Agency and ⁠Dr Lim Woo Lip⁠ from ST Engineering.

Catch the conversation in the latest episode of The AIMX Podcast. Or, you can read the rest of this AI-generated summary.

An Expanding Attack Surface

One enduring reality of cybersecurity is that digital exposure continues to grow. Over the past decade, increasing connectivity has widened the opportunities for cyber exploitation, a trend that AI has significantly accelerated.

With the emergence of large language models, social engineering attacks have surged dramatically. AI has lowered the barrier for crafting convincing, targeted messages at scale. Previously common red flags—such as spelling errors or awkward phrasing—have largely disappeared, replaced by polished, context-aware communications that are far harder to detect.

This trend signals not just an increase in volume, but a qualitative shift in threat sophistication.

AI-Enabled Impersonation and Automation

AI’s ability to generate multimodal outputs—text, voice, video, and images—has made impersonation one of the most prominent emerging threats. Attackers can now simulate realistic identities, conversations, and workflows that were not feasible even a few years ago.

Looking forward, automation through agentic AI introduces an even more concerning dimension. AI agents can continuously scan for vulnerabilities, pivot when defenses are encountered, and operate without fatigue or interruption. Unlike human attackers, these systems can execute campaigns in parallel, drastically increasing speed, scale, and persistence.

While fully autonomous cyberattacks may not yet be mainstream, the transcript suggests that semi-autonomous attacks—combining AI execution with human oversight—are already within reach and are likely to dominate in the near term.

Deepfakes and the Erosion of Trust

One of the most destabilizing consequences of generative AI is its impact on trust. Advances in deepfake technology now make it possible to produce convincing videos, voices, and documents that are explicitly designed to deceive.

This has serious implications for sectors such as finance, where identity verification and know-your-customer (KYC) processes rely heavily on biometrics. Cases have already emerged in which deepfakes compromised biometric authentication during financial onboarding outside Singapore, prompting regulatory attention.

The threat is not limited to formal systems. AI-generated documents—such as receipts, medical certificates, and identity records—can appear remarkably authentic, down to visual imperfections like folds or wrinkles. As a result, everyday operational trust within organisations is increasingly under strain.

In live communication contexts, such as video calls, deepfakes challenge long-standing assumptions about authenticity. The transcript highlights that verifying identity can no longer rely on visual realism alone, reinforcing the need for secondary verification and direct source confirmation.

Policy, Speed, and Technological Neutrality

A persistent tension exists between the pace of cybercriminal innovation and the speed at which governments can respond through policy and legislation. Threat actors adopt new technologies rapidly, while formal legislative processes may take years.

To address this mismatch, policy frameworks are increasingly designed to be technologically neutral—focused on outcomes rather than prescribing specific technologies. This approach aims to preserve regulatory relevance even as underlying tools evolve.

While policies may lag innovations, neutral frameworks help ensure flexibility and durability in governance.

Foundations Before Innovation

Despite the novelty of AI-driven threats, many core cybersecurity principles remain unchanged. Systems—AI-enabled or otherwise—must still be hardened, secured, and risk-assessed using established cybersecurity fundamentals.

AI-specific risks must then be layered on top of this foundation. Treating cybersecurity and AI security as intertwined but distinct disciplines allows organisations to break a complex challenge into manageable components.

This layered approach helps organisations avoid being overwhelmed by the convergence of two already complex fields.

Human-in-the-Loop and Explainability

While AI can dramatically enhance detection, response, and simulation capabilities, keeping humans in the loop is still important. Autonomous systems can recommend actions, but accountability and judgment must remain human responsibilities.

Explainability is critical in this context. Human operators must understand why an AI system makes a particular recommendation, enabling learning, oversight, and informed decision-making.

Overreliance on AI—where humans disengage and assume systems will “handle everything”—is a significant risk in itself.

Workforce Implications and Skills Convergence

The rise of AI reshapes not only threats but also the cybersecurity workforce. Traditional specialization is giving way to multidisciplinary skill sets, where cybersecurity professionals must understand AI, and AI practitioners must grasp cyber risk.

Beyond specialists, general employees will increasingly require basic AI and cyber hygiene—understanding issues such as hallucination, verification, and appropriate reliance on AI outputs.

This mirrors earlier cybersecurity awareness efforts, where responsibility gradually expanded beyond dedicated teams to the entire organisation.

AI as Both Threat and Defense

AI is fundamentally a double-edged sword. The same technologies that enable sophisticated attacks also provide powerful defensive capabilities. AI-driven systems can simulate adversarial behavior, identify vulnerabilities, and detect emerging tactics before they are used in the wild.

To remain effective, cybersecurity must shift from reactive to proactive and preemptive models, using AI to anticipate and prepare for evolving attack techniques.

However, technology alone is insufficient. Effective incident response still depends on human readiness—clear decision-making authority, practiced processes, and the ability to act decisively under pressure.

Preparing for 2030

Looking ahead, several capabilities will define cybersecurity resilience by 2030:

• Human–machine collaboration, where people work effectively alongside AI agents
• Using AI to defend against AI, matching attackers’ scale and speed
• Adaptability, as technologies, threats, and domains continue to collide
• Cross-domain skills, as cybersecurity, AI, data governance, and future technologies such as quantum computing increasingly intersect

Ultimately, cybersecurity in the AI era is not solely a technological problem. It is a socio-technical challenge that requires trust, collaboration, continual learning, and shared responsibility across individuals, organisations, and nations.