From phishing to deepfakes, the rise of GenAI-based tools signals a chilling shift in the cybersecurity landscape
| Photo Credit:
tsingha25
As cyber threats evolve, cybersecurity experts warn of a new frontier: Dark AI.
Nation-state actors and other hacking groups are now weaponising Generative AI to launch stealthier, more sophisticated attacks.
From phishing to deepfakes, the rise of GenAI-based tools signals a chilling shift in the cybersecurity landscape. Hackers can now prompt Dark AI tools to generate phishing messages, create imposter websites, write malicious code, and produce deepfakes in just seconds.
Deepfake audio and video technologies, which previously required significant resources and expertise, are now commoditized and easily available on marketplaces on the dark web.
“AI evolution and adoption have taken place at a breakneck speed in the last five years. But this has also brought with it a dark underbelly which has significantly altered the cybersecurity threat landscape – a change that no one has probably witnessed in the last two decades,” Ankit Sharma, Senior Director and Head – Solutions Engineering at Cyble, said.
A cybercriminal could craft and send, say, 10 phishing emails customised only for each individual in 10 days, and the success rate would be fractional. Dark AI enables the same criminal to launch thousands of personalised phishing emails simultaneously, each tailored to a specific individual based on their digital footprint.
“AI enables single actors to execute enterprise-level attacks with minimal resources and advanced cybercrime capabilities,” he said.
Kaspersky experts are now observing a darker trend – nation-state actors leveraging LLMs in their campaigns, according to Sergey Lozhkin of Global Research & Analysis Team (GReAT) at Kaspersky.
Dark AI refers to the local or remote deployment of unrestricted large language models (LLMs) within a comprehensive framework or chatbot system, used for malicious, unethical, or unauthorised purposes. These systems operate outside standard safety, compliance, or governance controls, often enabling capabilities such as deception, manipulation, cyberattacks, or data abuse without oversight.
“Traditional attack methods were static. They couldn’t be modified after the launch button was pressed. However, AI-powered attacks have now altered this narrative. They can adapt based on their target’s responses and change tactics mid-way to increase the chances of a successful hit,” Sharma said.
These are AI models that are intentionally built, modified, or used to perform unethical, illegal, or malicious activities such as generating malicious codes, crafting fluent and persuasive phishing emails for both mass and targeted attacks, creating voice and video deepfakes, and even supporting Red Team operations.
Apeksha Kaushik, Principal Analyst at Gartner, said that by 2027 about 80% of companies without robust AI risk mitigation strategies may face catastrophic outcomes, including litigation, reputational damage to leadership and long-term brand impairment.
“The rise of Dark AI is not a distant threat—it is already here. We are witnessing a rapid shift from theoretical misuse to AI-as-a-service models, where easily accessible text-to-speech tools enable attackers to gather information and impersonate trusted users,” Kaushik said.
Large-scale video phishing campaigns now leverage automated, real-time text and voice generation models, combined with phishing-as-a-service platforms. This commoditization of malicious AI tools is fundamentally reshaping the security landscape, making sophisticated attacks accessible to a much broader range of bad actors.
“Enterprises that fail to recognise this evolving threat and do not integrate disinformation security into their broader risk management strategies are leaving themselves exposed,” she added.
More Like This
Published on August 12, 2025
