How AI is changing the shady world of cybercrime | talkBIG podcast | Season 4 Episode 4

Katie Timms (00:00)

AI is revolutionising business efficiency, but it's also arming cybercriminals with unprecedented weapons. Cyber-attacks have become more sophisticated, and today's attackers use AI to personalise phishing, mimic trusted individuals through deep fakes, and bypass defences in ways that we've never before seen possible. So, the question that every business leader should be asking is, how do we defend when the bad guys have AI too? Hello, I'm Katie Timms. I'm a partner at RSM Australia and I've been in the industry for over 25 years.
Welcome to talkBIG. This is where we talk about business, money, and the economy to help you get ahead. In this episode of talkBIG, we'll explore how AI is giving cyber criminals new superpowers and the steps that businesses should take to stay protected. From AI-powered phishing scams to automated attacks, we'll unpack the latest threats and share practical defence measures every organisation should consider.

Now joining me today is an expert, so no pressure, but an expert in cybersecurity and artificial intelligence, Ashwin Pal. Ashwin is a partner in our Risk Advisory division at RSM Australia, specialising in privacy and security, and has over 27 years of experience in the cybersecurity industry in the Asia Pacific. At RSM Australia, Ashwin leads the team, assisting clients to identify cyber exposure and risk, developing effective risk management strategies and implementing the necessary measures to safeguard operations. And at the moment, a big part of that includes artificial intelligence. So welcome Ashwin.

Ashwin Pal (01:41)

Thank you, Katie, and happy to be here.

Katie Timms (01:43)

Now, your part of the business has always both fascinated and horrified me because I think it is one of the most rapidly evolving and high-risk areas that businesses are facing these days. So, give me a big picture. Talk us through how AI is changing the way cybersecurity is needing to be managed, the way attacks are happening, and why that shift matters.

Ashwin Pal (02:04)

Well, with AI, what you’ve got is attacks are getting faster, cheaper, and better or harder to detect. So, what we used to see in the past, and obviously I've been doing this for 27 years, you did make me sound older than I am, long enough. And what I’ve seen within that time is the attacks tend to stay the same. It's just that they’re going at a much faster rate, much easier for the adversary to then execute.

And much harder for people like us to then detect and protect our businesses.

Katie Timms (02:36)

It does seem to be moving so quickly. I mean, you know, even recently we saw, you know, the release of Fable 5 and here's my, you know, big you know, AI knowledge. But, you know, that release being pulled back by the US government because of security concerns. I mean, you know, what kind of attacks are we seeing come through, you know, or with the help of all of this kind of AI?

Ashwin Pal (02:54)

That’s a very good segue into the answer, all right? So, there are really two major categories. So, what you highlighted is the ability for software effectively sorry, for AI to look at software, to look at software vulnerabilities pretty quickly, but not just find the vulnerabilities, craft attacks on the fly to exploit the vulnerability. It's that second part, which is the scary bit.

Because before, what used to happen is hackers, you know, I'm a white hat, but people like me would go and find vulnerabilities and then if there was an existing exploit code, we could use that to then exploit the vulnerabilities. But if there wasn't one, we would then have to go away, write all of that. You can imagine how much time and effort that takes. AI does that now within minutes, and in there is the paradigm shift. The second category are deepfakes, social engineering. So, what you’re seeing now, just you know, voices being cloned, people's faces being cloned, actual videos like this, you could do this using AI, and I wouldn't even be there, just for clarity. I'm here and I'm real. But you know, you can imagine if you could use a fake video to convince a CFO to make a payment. More on that later.

Katie Timms (04:13)

Okay. So, I mean that's probably a really good example then, because you as I say, your space is horrifying because you see tangibly how this plays out, you know, in businesses. So, I mean, can you give me an example? You know, let’s hide identities and change some facts. But, you know, what's an example of what like an AI enabled attack might look like in reality?

Ashwin Pal (04:32)

So deepfake social engineering is probably a good one and there's a quite a famous one that happened in Hong Kong maybe about two years ago now. So, what ended up happening is hackers created a whole meeting, right? With executives from an organisation sitting around a table talking to a CFO, requesting payment. That meeting, everybody at that table were complete fakes, complete fakes.

Now, Katie, you can imagine if you are being requested by RSM's executives to do something and looks as real as you and I talking right now. Would you know it? Would you pick it up? Would you not follow the instructions? That's exactly what happened, and a $25m transfer then happened.

Katie Timms (05:17)

This was a couple of years ago. So, we're even talking, you know, in the scheme of AI, that's two years old now. I mean I'm assuming that what they can do these days is far, far better. You know, we've gone a long way from, you know, getting the email, you know, and being instructed to click on a link. I mean, we're well past all of those risks now.

Ashwin Pal (05:19)

Totally. Totally. And the attacks are just so much harder to detect. That's the big issue with AI. You know, if I wind it right back, you know, there was a time where you could easily pick up those scam emails because the spelling was so poor. Well guess what? You don't need to learn English anymore. That's what you have AI for. These are some very simple but very effective examples of how AI is basically you know, is turbocharging these attacks.

Katie Timms (06:00)

And you referred to yourself as a white hat. You know, are you sort of one of the frontline defenders against, you know, so a hacker still in terms, but you know, one of the good guys? Is that how you would see yourself?

Ashwin Pal (06:03)

Mm-hmm.

Absolutely.

Absolutely, Katie. And the term, you know, the line I've usually used all along is to catch a thief, you got to think like a thief. That's literally what it comes down to. the cyber side of things, it’s basically you know, cops and robbers. You just got to try and understand how the robbers are going to get in and you have to stay one step ahead. And if they get one step ahead of you, that's where the problems start. And defending is not easy. I often say, we have to get it right every single time. They only have to get it right once. And in there lies the challenge.

Katie Timms (06:44)

So I mean, with that in mind, so this now I'm back to being horrified by what you do because, you know, the risk of exposure in this, you know, and the reality of businesses having to try and plug all of these gaps that they might have, you know, I mean, what are the really core like where are the biggest gaps that we see people? What are the repeated mistakes that businesses are making? You know, where are the core vulnerabilities that we can start with to try and, you know, provide some level of protection?

Ashwin Pal (06:50)

Okay. Yeah. So, look, there are a couple of elements, right? And I’ll try and keep it simple. So, there's the people aspect, which we spoke about, and it’s no surprise that the two categories that I mentioned exist. So, the baddies are going to try and exploit the human factor because in that case it doesn't necessarily mean that there is a system vulnerability, a technical issue, right? What it basically means is I’m going to try and trick you.

Into doing something that you would normally do, that you have access to, but would then grant the privileged access. So that's one side of things, and I'm oversimplifying how that works. But the people factor is one element that most businesses arguably don't pay enough attention to. The second aspect is understanding, getting a thorough understanding of your entire environment. Now, if you don't do that, if you don't understand where the vulnerabilities exist, and this is at a technical level, then even one hole could be picked up by an adversary using AI quite quickly in this day and age, and then use that basically to get into your environment and steal data or whatever nefarious things they want to do. So, it’s about making sure that you cover the people side of things.

But also have a good understanding of your entire environment and make sure that you’re protecting everything throughout the entire environment. And Katie, if you just even think about RSM, which is obviously not the largest organisation in Australia, but the systems you use on a daily basis, you can start to understand the complexity that we are dealing with here.

Katie Timms (08:48)

Absolutely. And I mean, you talk about the human element, which I do think is really probably a big concern because how do you protect against the humans? Humans are the ones that have autonomy, humans are the ones that are making, you know, different decisions. I mean, what can business owners do for that? Is it training? Is it, you know, extra security? Is it layers? Like what, you know, are some things that you can look at to try and protect the humans?

Ashwin Pal (09:11)

So, there are three parts to it. One involves making sure that basically you’re covering the people side of things by training them, right? And I break this down into people, process and technology. People directly, training, training, and more training, right? And training is about developing muscle memory. Okay. So instead of thinking it comes naturally, but the second part is also morphing and changing the training based on the threat landscape. Now, the scary part, unfortunately, is the cybersecurity threat landscape is changing on a daily basis. Right? New tools, new vulnerabilities, whatever you can think of is coming up all the time. So that's the first part. The second part is process or policy, making sure that people understand what they need to do. It is documented in policy, and it is signed off.

Again, if you think about the training we go through, Katie, we go through both of those elements on a fairly regular basis, right? And the third aspect is technology. So, people will make mistakes. You know, if you're in the middle of something, if you're really busy, you click on something that you weren’t meant to. Those things happen. That's where you have what I call the fallback technology measures.

So that if somebody clicked on something which then started exfiltrating data to some other country, usually you weren’t meant to. You know, as an example, you’ll have like a data leakage prevention solution which will see that and stop it before the data goes out of your network. So, it is a layered approach, and you do need to take that layered approach.

Katie Timms (10:48)

Because I know RSM tests us, you know, on our cyber. You know, we’ll get the emails and then we'll get the damning report of all the people that thought they were sent a Valentine's Day e-card and going through on the link. You know, but I mean is that the kind of training? Like, you know, I mean we laugh about it at the time as to well, that was pretty sad, but you know, is that the kind of training that you would suggest for businesses? You know, is it trying to balance recognising the risks with understanding where, you know, future roadblocks might be.

Ashwin Pal (11:18)

Yeah, absolutely, Katie. You have to think about what's going on and making sure that the training is relevant to this day and age. For instance, you know, the Nigeria email scams, not saying that's not important anymore, but there are more important things now, because you know, the scammers have moved way past that. The other thing that I recommend by clients to think about is making the training role specific. So for instance, somebody sitting at reception, may require more training in physical security, being able to recognise somebody who doesn't quite look right and is trying to walk into the organisation as a tradesperson Somebody in finance being trained more on business email compromise, for instance, than you know, somebody else doing another type of job. So, it's making sure that the context is there.

Katie Timms (12:05)

You know, if we say, all right, we can train our people, we can do as much as we possibly can to mitigate the risk from the humans, then on like the software side of things, I mean, you know, what kind of gaps are we seeing in, you know, for businesses there? Is it

outdated? Is it, you know, not fit for purpose? Is it just insufficient? You know, what are the some of the risks that businesses have got on that side of things?

Ashwin Pal (12:13)

Yeah.

Largely outdated, and there are two reasons for it, to be honest. One is plain old laziness, which does happen. Not ideal, but it does happen. The other one is patching can be tricky, Katie, because if you look at a system, there are many layers within that system, and each layer has its own piece of software. You know, database is different to an operating system, which is again different to an application. I'll leave the technicalities aside, but there have been many instances where if you patch an upgrade,

Katie Timms (12:50)

Thank you.

Ashwin Pal (12:53)

an operating system, it'll break the application that's sitting on top of it. And it’s the application that users use to do what they need to do, right? And in there lies the problem. So, if you can't do that, what do you do then? Right? Do you change the application? That could take time. Do you leave the operating system vulnerable? Probably not a good idea. Do you isolate that particular system and that operating system?

Maybe. You see the complexity that’s involved with patching. It’s harder than people think it is. And then, I've been talking about IT. There's a whole thing about operating or you know, operational technology, what we call OT systems that operate machinery, that operate power grids, the dams etc. And they’re even harder to patch. Patching generally means you're going to have to take the system down. Now,

would you be happy if Sydney Water stops supplying your water for a day while they're patching the OT systems? You get the point. Right? This is not easy, which is why you need to come up with a plan to be able to pick up the vulnerabilities in the first place, but then also have a plan around how you’re going to address that. And if you can't, what mitigating controls you’re going to put in?

Katie Timms (14:14)

So, I mean, given how fast AI is evolving, and it's, you know, I look at where we were two years ago to where we are now in terms of AI, and the consistency of business owners having to try and play catch up constantly, you know, trying to plug every single hole. How do we then use AI as part of our, you know, wearing your white hat, how are you able to use AI to help, you know, use a defence?

Ashwin Pal (14:38)

Well, see AI shifts the paradigm in our favour as well. So, if I think back ten years ago, a lot of the security tools were basically they were based on fixed rules, right? So, if they saw something, they'll pick it up. But then if they didn't see it, then guess what happened? The attack went through. AI now helps us effectively do anomaly detection, do predictive analysis, right? So, it's based on context.

It's based on behaviours. For instance, if all of a sudden Katie is logging in from Russia in the middle of the night, hmm, likely not going to be Katie. Okay. Now, if somebody is coming through using your valid credentials, a traditional security system would let you in, an AI based system, would detect this doesn't quite meet the normal pattern that Katie has. Now, depending on how you’ve configured your response, it could block you or it could at least raise an alert for a human to then investigate and then subsequently block you. Again, you'd design the rules based on how critical the event is. However, that gives you an idea and an illustration of how we could use AI in a defensive capability, as opposed to obviously what the adversaries are doing, trying to use it to attack us.

Katie Timms (15:55)

Do you think that, you know, the use of AI in a defensive capacity is keeping up with the you know use of AI as attackers? Like, you know, do we have limitations in this space and we're always going to be playing catch up?

Ashwin Pal (16:08)

We do. And I like the word catch up. I was presenting to a client yesterday in the banking sector. And we were talking about the massive amounts of sort of fake documentation, interviews etc., that are happening. You would have heard about the $1bn of fake mortgages that Commonwealth Bank went through not long ago, right? That was maybe four months ago, potentially less. There were a topic of discussion and the tools that are out there in terms of detecting fake AI documents, and you know, fake interviews potentially like this, are getting better, but we are reactive in terms of the tools we make. So, there's an attack created, and then we would come up with a tool to try and blunt the attack. That window between the attack coming up and the tool being created is the one that adversaries exploit most. So short answer to your question is we are playing catch-up to your words. It is natural,

unfortunately, because people don't proactively go and spend money on something if they don't need to. And that's both on the client side as well as people that are coming up with tools to protect us. However, when an attack comes out, tools do come up fairly quickly.

However, as I mentioned earlier, there's that window in between tools coming up and the attacks coming up. The tools are getting better. There was a stage where the adversaries could get around a fake interview. But now the liveliness tests are getting pretty smart. Like hackers are saying it is pretty difficult to get around the tools, the detection tools can pick up that that person on that computer screen is not real. That he or she's not alive.

Katie Timms (17:49)

Okay, so and that's called a liveliness test, is it? So, there are things out there, you know, because what you're what you're describing sounds like a movie to me, you know and there's been a lot of movies with cyber-attacks, and they mostly feel ridiculous and far-fetched. But what you're talking about is a little too real. You know, so there is now the ability for someone to, you know, someone, but AI to look at something and say, that's not a real person, that is a deepfake. Like we're getting further down that path. Okay.

Ashwin Pal (17:52)

That's what it's called. Yeah. Getting better.

Katie Timms (18:16)

Great. Then I'm going to put aside the movie that I watched where it was all about deepfakes, and I'll worry more about now, as you said, Sydney Water and Die Hard 4 happening and you know, someone's taken over the power grid. So, I mean, for businesses who are, you know, because you've been doing this for a billion years, you know, you are very experienced in this. But for businesses who are still early in their kind of cyber learnings, you know, what is what's a realistic way that they can start using AI?

Ashwin Pal (18:20)

Okay.

Katie Timms (18:42)

You know, effectively in their security.

Ashwin Pal (18:45)

Well, look, I would couple that with two elements. as I said earlier, what you shouldn't do, or you cannot do is throw tools at your environment and think of it as a silver bullet. It just doesn't work. And a lot of people make that mistake, right? If you don't understand the problem before you solve it, guess what happens? You can't solve it. So, understand the problem, understand your environment, understand where the gaps and vulnerabilities are, and come up with a plan. Now when you start executing the plan, is when AI tools and things come into play. Fortunately, and I gave an example earlier as to how AI is now being embedded in cybersecurity tools and how they’re helping us. Every decent vendor that I know of is using AI within their tool sets. So, what you then need to do is once you've come up with a plan, understand the vendors that are out there that can help you, look at their tools or you know, if you don't have the expertise, then obviously rely on somebody like RSM to help you with that, to understand you know, who are the better players in the market. Get that advice, deploy those tools, which would then naturally use their built-in AI capabilities like I described before, to be able to defend you. But I go back to what I said earlier. Understand the problem before you solve it. If you don't understand the problem, you have no chance of resolving it.

Katie Timms (20:03)

And I think that's a really great point. I mean, you know, just it feels like a really simple way to address this is to go, great, I've paid for this product that is going to protect me, but you might be building a fence, you know, to stop animals coming in when animals aren't your problem, it's the water that's coming through underneath, you know. So, what you're saying absolutely makes sense. So, do you think that's the most important thing for business owners is to take a step back, get an understanding of the risks that are facing them before trying to just put solutions in place?

Ashwin Pal (20:19)

Absolutely, Katie. And look, I mean, the defensive mechanisms that I'm suggesting here have not changed. AI effectively has made things go faster and better. It hasn't affected you know, in essence, it hasn't changed how everything works. So, the old ways of resolving these issues are still there. It's just that you need to get a bit more methodical and a little bit quicker.

Katie Timms (20:53)

Hm. Yeah. And do you think that the use of mobile technology, so phones, you know, everything is on this device. We are so getting so used to quick responses to all of that. Do you think that that is leaning into some of the risks that, you know, do exist? You know,

people are clicking on links, people aren't taking the time to do, you know, their due diligence and things. Is this another area of exposure for business owners?

Ashwin Pal (21:11)

Yeah. Absolutely it is, Katie. And mobile devices in particular sort of have their own unique set of issues because again if a link is sent onto a mobile device, the link generally is truncated, right? So, it goes into a smaller device easier. The problem with truncation and Katie through training, you would know. You know, we are told to hover over the link, don't click it, just read the link and make sure it looks okay. Well, guess what happens when the link is truncated?

The other issue you have is on a mobile device everything is smaller. And generally, you're looking at stuff on a mobile device on a train, on a bus, you know, you're multitasking. Oh my god, yes, don't even go there. Yes. yes. And you know, while you’re busy doing other things, that's when you make that mistake. Having said that, Katie, everything I've just mentioned, as much as you can put security tools, and remember.

Katie Timms (21:47)

Mm-hmm. Crossing the road in some instances.

Ashwin Pal (22:06)

The three-step process I mentioned earlier on the mobile device to protect you, the first line of defence tends to be that policy element and the training element, right? Again, you need to build muscle memory. Everything I described about smaller devices, truncated links, clicking on it by mistake, these are all people issues. So, you need to train your people to be able to address that. I mean a common so obviously everyone's familiar with multi-factor authentication. A common attack now is called MFA bombing.

Katie Timms (22:13)

Mm-hmm. Yep.

Ashwin Pal (22:34)

So, and Katie, you'd be familiar with our systems, right? So, using authenticator, if you want to get in, pop-up comes up and you have to do whatever you need to do. Now, if an attacker has your password and if they're trying to log into a system, your system, multiple times, you'll keep getting that pop up, right? And you kind of go, I'm not trying to log in. Cancel, cancel, cancel, cancel. Say for instance, you know, your daughter is crying, you need to run. Instead of hitting cancel and the approve button is right next. Yeah. You get the idea, right? So, the adversaries know human behaviour. They’re exploiting human behaviour.

Katie Timms (23:09)

Okay, well I'm back to being horrified again and terrified of what's going to be happening on my phone. How often, you know, do businesses need to be reviewing their cyber strategies, you know, when they're when they're looking at that that you know, all those defences? How often do we need to be looking at this?

Ashwin Pal (23:14)

Yeah.

Yeah.

I would recommend Katie at least annually, right? And that annual will tie in with when you do a test of your environment. So, test and then obviously respond accordingly. Now that is what I'm calling sort of middle of the bell-shaped curve, depending on where you are, if you are a higher risk organisation, for whatever reason, then you might want to do it six monthly.

Or even quarterly and you know, quarterly on high-risk systems. You know, if you are a major defence contractor and you have got state secrets. I'm making that up, but you kind of get the idea. And there is System X that is in play here, then you want to look at it every quarter. And you want to, you know, update the system security plan, for instance, on a quarterly basis. But, you know, if you're a run-of-the-mill coffee shop as an example, then annual may be okay.

There is no right or wrong answer. Everything with cybersecurity is very risk dependent. So again, I go back to what I was saying earlier. Understand the problem, understand the risk, understand your environment, and then methodically approach it.

Katie Timms (24:34)

So, what would be your tip for business owners for right now so that they're better prepared for what is coming next?

Ashwin Pal (24:40)

Look, to be honest, I would just summarise what I’ve just said. I think the two key elements that you need to focus on is the people side of things. they can be so easily exploited. You do need to pay particular attention to the human element. And the second part is, as I said, understand the problem before you resolve it. You know, understand your environment, understand what is in there, what systems may be vulnerable.

What AI tools are you using? Shadow AI is a big thing as well. Make sure you understand what your user is doing with AI. Understand the problem, quantify the problem, and then take a methodical approach to resolve it. If you do both of these elements, it puts you in a much better position than perhaps others equivalent or parallel to you. Hackers are very lazy, right? Time for them is money. Exactly. There are so many low hanging fruit out there, if you are better than the next person, they will just leave you alone and go to that next person. An analogy that I often use within cybersecurity is how do you not get eaten by the bear? You just run faster than your friend.

Katie Timms (25:36)

Mm.

I was on a safari once and someone said to me, you just carry a little knife and make sure that they just need to be one person that's bleeding more than you. So, same kind of philosophy, right? I mean, so you know, we've talked a lot about the defence and things that businesses can do. How are mums and dads and retirees and people supposed to defend themselves against, you know, this more and more advanced AI? I mean, I still have to sit down with my mum and dad and look at their emails and say, mum, that's very clearly spam. Like, you know.

Ashwin Pal (25:54)

Yeah. It exactly. Yeah.

Katie Timms (26:21)

How are we supposed to help and protect these people who are probably some of our most vulnerable?

Ashwin Pal (26:26)

Absolutely. And it's, you know, the analogy I use is with my dad. And you know, the good thing is I talk to him and I educate him. But there are some good resources available that are provided by the Federal Government. So, the eSafety Commissioner, for example, the ASD, ACSC websites, they have some really good guidelines available. Arguably they're probably not as well publicised as they should be. But hopefully,

those that are going to listen to this podcast can spread the message. So, the eSafety Commissioner's website has some really good tools. You can just Google that, find that out. and the ASD ACAC website has some really good tools. Pretty easy to navigate because the guidelines are categorised by individuals, small business, medium, large, and government etc. So, you can go in and have a look.

Some very good basic guidelines are available there as well. That's probably you know the tip that I would provide. And, you know, not sounding silly, but you know, podcasts like this would help as well. If you can get a hold of authoritative, reliable podcasts like this, you know, while you're walking to the bus stop, listen to it.

Katie Timms (27:28)

Yeah, absolutely.

Because it is quite a confusing, you know, space, you know, for the lay person, you know, to try and keep up with. where how fast is AI going to keep developing? What’s your tips for us in this space? I mean, you know, where are we going to end up in this? Because it does feel like, you know, this is a train that is well and truly on its way.

Ashwin Pal (27:57)

Two out of two Skynet. Now that's the conspiracy theorist in me. Yeah, the day AI becomes conscious is going to be interesting. And look, I mean, jokes aside, AI development is accelerating. Just recently, over the last week or two, you know, we’ve had articles come through from founders and people high up in Anthropic.

Katie Timms (28:00)

Yes, yes, yep.

Ashwin Pal (28:20)

Who have come up with Fable 5 etc saying that we need to slow down. We need to put guardrails in place, checks and balances. So, at the moment, the investment in AI, the development in AI is going at a massive pace. Nothing like I've seen before in IT or cyber. Having said that, I think caution is being raised around where we end up. Where we could potentially end up and making sure that we put guardrails in place that it doesn't become a train wreck.

Katie Timms (28:53)

Because you're right.

Ashwin Pal (28:53)

So, the ultimate end goal could be completely, you know, conscious AI. And I think it is absolutely possible in our lifetimes. What that looks like and how we can exercise a degree of control is going to be the question that needs to be answered.

We don't want to be the not dominant species on the planet.

Katie Timms (29:12)

Yes, so true. All right. Well, now that I'm going to be scared of my computer for the rest of today, look, thank you so much, Ashwin, for coming along. I've really enjoyed, you know, I always enjoy talking to you about this stuff. As I say, I do find what you do really fascinating. I think you've made some really excellent points here. Like most things, this can't just be a slapdash approach to it. There really needs to be a proper step back. You know, businesses need to understand where their risks are and then make sure they're addressing those risks, not just thinking, hey, we've paid for a product.

All is good. You know, we need to first understand that. And also, that human element, which is the bit that we cannot control, but I think maybe is sometimes underestimated by businesses.

Ashwin Pal (29:52)

Indeed. Thank you so much Katie. I appreciate your time and I hope the listeners get something useful out of this.

Katie Timms (29:59)

Yes, thank you. And thank you to everyone for joining us on talkBIG. If you found this episode helpful, please subscribe and leave a review.

Ashwin Pal (30:09)

Thank you.