I joined David Dean Mauro on the Cyber Crime Junkies podcast to talk about how fraud works inside contact centers, and how AI is reshaping both sides of that fight. Here are some highlights, with the full transcript here.
On why insider fraud rarely trips an alarm
Todd: When you call with a customer support complaint, it's not like healthcare where there's a single EHR that the person's interacting with. They're interacting with seven or eight different systems in order to solve the case. Each of those has a different granularity of permissions and access controls. And so a lot of the issues come from not necessarily the systems being misconfigured, but people taking advantage, and sometimes it's called business logic abuse. If you look at each thing in isolation, it's fine. It's not ringing any alarm bells. It's how the person combines those together in order to have an effect that leads to a bad outcome for the business.
On how far an insider will go
Todd: The most complex that we've seen is, someone knew what the route was for the UPS or the FedEx truck in their area, and so they would send it along that route, so then they could follow the truck and steal the packages from the doorsteps of where they were sending it to.
David: So they were involved in the customer service element, then they became a porch pirate.
Todd: Exactly.
On coaching new agents into fraud
Todd: They would call in and chat in until they got a new agent that didn't know what they were doing, and they would walk them through this internal bespoke tool that this company has, through a particular gift card workflow. […] In physical retail, there's this idea of laundry detergent being a prime thing to steal, because it's dense, it's expensive, and you can offload it easily. It turns out, on delivery, laundry detergent is heavy. A good one to do is Nespresso pods. They're dense, they're expensive relative to the weight of the item. Easy to offload because they're fungible. So this attacker would find these agents, get them to go through a weird workflow to get this special case refund or gift card code, and immediately turn those into Nespresso pods that they could sell on the black market.
On socially engineering the chatbot
Todd: You have folks going into the customer support chatbot on a site and doing a certain set of instructions to trick that in order to give them a refund or give them a different price. This whole community is on, what is the right language you need to use for a certain bot to get it to give you a deal or a discount? And the legal precedents, probably not in the US, but at least in Canada and some other countries, is, if the AI chatbot is giving you a price quote, that is a legally binding price quote.
On treating AI like any other fallible employee
Todd: The main lesson is, AI is fallible. It can be tricked given enough ingenuity. And so don't give it, it's the same thing you would have in the rest of your org, least-privileged access. Don't give it access or the ability to do things that it shouldn't do.
On why smarter agents can be easier to con
Todd: The research out of Stanford showed that when you use smarter models with more reasoning tokens, or more time thinking about it, those can be tricked easier in some cases, because they rationalize the behavior. It has to complete the task. So we have an open source project, Agent Browser Shield, with 30 different policies that hide these different things. It'll hide the countdown timers, the inventory ones. When you get to the shopping cart phase, it will uncheck these pre-checked items like insurance, so that the agent has to reason about whether it wants to buy insurance versus just taking the default.
The full conversation is on YouTube. A lightly edited transcript is on this site. Agent Browser Shield is open source on GitHub.