A Fraudster’s Paradise – O’Reilly

Darkish net discussion board posts talked about the phrase “AI agent” much more within the second half of 2025 than within the first half. May this imply that fraudsters are charmed by the AI hype? Or is AI really a sport changer for cybercrime? AI-related discussions—evident each in what “the unhealthy guys” are saying and in what fraud-fighters are exploring—are all over the place. In truth, a Visa PERC evaluation evaluating knowledge from June to November 2025 to the six months earlier than that discovered an enhance of greater than 450% in darkish net agentic AI-related posts.

Documented monetary losses from deepfake-enabled fraud exceeded $200 million in Q1 2025 alone. And Visa reported a 25% enhance within the second half of the yr in malicious bot-initiated transactions. There’s AI-generated rip-off and fraud content material all over the place you look, and it doesn’t all the time give Eric Clapton six fingers to make it apparent that one thing’s off:

None of that is taking place in a vacuum. On-line fraudsters have embraced GenAI with open arms in a approach that has modified the form of the web fraud panorama. On the identical time, unusual shoppers are leaning into the form of dishonest that GenAI makes simple.

How can the monetary and digital ecosystem combat again in opposition to this new wave of AI-powered fraud? Studying from previous expertise, collaboration and information sharing between fraud-fighting professionals is vital to our collective potential to brace for impression. This quick weblog put up is a name for harnessing the facility of AI for reinforcing the know-how of digital defenders, as a option to strengthen our general protection.

Keep in mind COVID?

AI isn’t the primary time that the foundations of cybercrime have modified dramatically (and it received’t be the final). For instance, again in 2020, through the first COVID-19 lockdowns, work-from-home schemes exploded, together with first-party fraud, phishing scams, and extra. On the time, the neighborhood responded successfully. Groups of fraud-fighting specialists joined forces to fulfill just about, study the brand new terrain and ultimately write a examine information that will empower organizations to defend communities from a surge of digital fraud. Our ebook Sensible Fraud Prevention was the results of that collaboration, and it shortly grew to become a useful coaching useful resource to many groups within the ecosystem of their combat in opposition to on-line monetary crime.

At this time, a brand new wave of fraud is rising, powered by AI and particularly GenAI. Whereas some AI-powered initiatives are right here just for a short time, others have gotten really highly effective and harmful instruments in nefarious fingers. It’s solely pure that the skilled neighborhood will as soon as once more regroup to kind a playbook in opposition to these developments.

As we interview specialists in various fields for our subsequent ebook, The Fraud Fighter’s AI Playbook (with coauthor Chen Zamir, now in early launch on O’Reilly), we’re constructing an image of the ways in which GenAI is altering not simply the methods fraudsters function on-line but in addition the form of the web fraud panorama itself. We’re seeing, too, how important it’s that fraud fighters themselves put money into exploring and utilizing this know-how to spice up their success, technique, and inner repute in their very own firms.

Extra fraudsters, doing extra hurt

There are extra on-line fraudsters at the moment, finishing up extra fraud assaults, than ever earlier than. Not all of this may be blamed on GenAI. Reasonably, GenAI entered right into a fraud world that, on reflection, was poised to leverage it for crooked enlargement.

The COVID-19 pandemic drew many new fraudsters on-line, by means of three predominant tracks:

• Digital transformation + time to spare. Retailers, banks, and different organizations needed to cram years of digitisation into weeks or months. It was inevitable that this might lead to some vulnerabilities in processes, insurance policies, or programs. Individuals caught at residence with no work have been tempted to make some cash exploiting these weaknesses and discovered quick.
• COVID aid applications. Applications put in place to tide people over the difficulties of the pandemic didn’t all the time embody checks to confirm identities or claims. Those that discovered make fraudulent submissions, typically even by creating pretend or artificial identities or companies, came upon how simple and the way profitable that may be.
• Rip-off compounds. Human trafficking rings pivoted from in-person exploitation to forcing individuals on-line to hold out phishing assaults and scams of all types.

None of those developments has disappeared, and the rip-off compounds particularly have expanded massively for the reason that finish of the pandemic. This was the world that GenAI was born into.

With GenAI, much more fraud assaults are potential, at a stage of personalization that will have been unattainable at scale with out the know-how. Frequent makes use of of GenAI within the wild embody:

• Phishing campaigns, personalised by utilizing open supply details about the goal (sourced through GenAI), and the suitable language and cultural touchstones for every sufferer (made simple by placing conversations by means of GenAI). Extra typically, the impression is seen in scams of all types, from romance scams to funding scams to kidnapping, catfishing, blackmail scams and extra—equally personalised. 

• Malware and bot creation to hold out assaults, steal data, arrange pretend accounts, and extra. Fraudsters not want a lot technical potential to create malicious applications or automation, enabling extra fraudsters to amp up their attain. 
• Deepfakes. Whether or not for clickbait technology or for monetary scams, deepfakes enable attackers to move id validation checks. In stunning instances, deepfakes have been even used to pretend the trail to a job (and the wage and entry to knowledge that comes with it). Broadly, AI simplifies the method of making pretend or artificial identities utilizing open supply data, paired with stolen private credentials.
• Faux web sites, pretend apps, pretend promoting. Manipulated content material may be utilized to steer shoppers to buy nonexistent or low-quality merchandise, however it will possibly additionally goal digital promoting. Main manufacturers are dropping hundreds of thousands of {dollars} to AI schemes that generate views, impressions, or clicks for undeserved advert income. A examine of faux cell apps, led by Gilit’s group at DoubleVerify, discovered that pretend iOS grew to become thrice extra frequent (accompanied by six occasions extra pretend Android apps) in 2025 in comparison with earlier years, a development amplified by AI.
• Fast evolution in sophistication and evasion of bot schemes. Agentic AI is getting higher every single day. Because of this operators of bot networks are smirking at CAPTCHA challenges (“remedy this puzzle to show you’re not a bot”). Fraud assaults of 2023 had some rookie errors, like bot networks that attempted to move as human beings watching TV content material besides that they have been coming in with gadget settings of fridges. 2026 attackers will not fail at fundamental deception as a result of they’ve AI chatbots (FraudGPT, WormGPT, and many others.) to information them on their approach.

The excellent news—if there’s any actual excellent news—is that not less than these aren’t new varieties of assaults. They’re acquainted assaults, carried out extra convincingly, at far better scale.

GenAI is a aspect hustler’s finest buddy

It’s not simply devoted fraudsters utilizing GenAI to develop their attain. Peculiar individuals use it to stage up their dishonest sport too.

Refund fraud has turn into simple at a really convincing stage due to GenAI. Many retailers ask for photographic proof that an merchandise has arrived damaged or broken, and with GenAI, that’s one thing that may be faked in seconds. For the reason that picture is created for a goal from scratch, there’s no option to discover an unique on-line as proof that it’s a cheat.

Some individuals have gotten much more inventive, utilizing the identical form of trick as a part of an insurance coverage declare. Others use GenAI to whip up pretend receipts, which they’ll declare again from their firm.

It’s vital to notice that, as with the skilled use instances, it’s not that these cheats are new sorts of assaults. What’s new is the benefit, scale, and class with which they are often carried out.

The bottom shifting below our toes

After we wrote Sensible Fraud Prevention, we included a dialogue of issues like phishing, victim-assisted fraud, refund fraud, and so forth. The deal with the ebook, although, was on the ways in which fraudsters money out their schemes. Observe the cash, and discover the fraudster.

Now, solely slightly greater than three years after ChatGPT burst into all of our lives, that emphasis has shifted. Conventional third-party fraud, the sort you get when a fraudster makes use of your bank card on-line, isn’t even within the high three fraud considerations. TransUnion reviews that the most enterprise loss in 2025 got here from rip-off/approved fraud (24%), adopted carefully by artificial id fraud (20%) and account takeover (20%). That’s the GenAI impression.

There’s a major price ticket hooked up to numbers like that. The identical report famous that “firms worldwide misplaced 7.7% of their annual income on common on account of fraud over the previous yr.” Within the US, it was 9.8%.

There’s additionally a worrying impression on belief. When deepfakes are frequent and convincing, who can ever consider their eyes? Clients don’t know which internet sites are actual, which messages are genuine, or which advertisements or provides may be trusted. Companies don’t know which claims are professional or how finest to remain forward of the verification challenges they now face. Marketplaces battle to guard consumers from dishonest sellers, and sellers from dishonest consumers, and everybody from exploitation by malicious actors.

Ha! Fraud fighters have GenAI, too

The ray of hope in our analysis is that fraud fighters have GenAI too, and groups are already experimenting in quite a lot of methods. Some are how they’ll use brokers to develop their open supply analysis to make their selections quicker and extra correct. Others are engaged on craft prompts to assist analyze knowledge or work out developments that can be utilized to establish and cease fraud. Nonetheless others are leveraging GenAI to research documentation, to select fakes or alterations. And so forth.

It’s additionally encouraging to see how groups are utilizing GenAI to develop their attain internally inside an organization. In some methods, it’s virtually like fraud departments are getting the assistant they’d all the time wished to do the duties they’d all the time meant to get to—like pulling the info and placing it collectively for a biweekly replace to related stakeholders or creating detailed materials with useful illustrations or graphs for displays to different departments.

It’s inevitable that when a completely new know-how comes alongside, the fraudsters could have an higher hand initially. They aren’t hampered by issues like regulatory considerations or authorized necessities, and so they don’t care about issues like accountability, accountability, or shopper belief. It’s just about of their job description to disregard these issues, in truth.

The fraud-fighting trade has been sensibly cautious about figuring out establish and make use of GenAI, nevertheless it’s clear that they’re not standing nonetheless. The groups who do the most effective with this evolving problem would be the ones who work carefully and persistently with departments throughout their firm to adapt shortly to the enterprise’s wants—and meet them.