Staying one step ahead of the cyber-spies

If you want a job that rides the wave of the future, get hired by a firm that combats cyber-threats.

Criminal and malicious hackers are endlessly inventive and every day despatch novel viruses and other digital threats into cyber-space to wreak havoc.

Getting paid to tackle these is about as cutting edge as you can get.

One emerging discipline in this field of cyber-incident response tackles the most skilled and serious of these hackers - those who work for nation-states.

The UK's GCHQ now estimates that 34 separate nations have serious, well-funded cyber-espionage teams targeting friends and foes alike.

The threat from these state-sponsored digital spies has been deemed so serious that the intelligence agency has designated five firms victims can all on if they are caught out by these attackers.

"We get called when people have a big fire and we come along with our hoses and try to put it out," says James Allman-Talbot, head of incident response in the cyber-security division of BAE Systems.

That captures the fact that, more often than not, the fire brigade arrive to find a building still in flames. When it comes to cyber-fires, that means the hackers are still embedded in a victim's network and are still trying to steal data or burrow more deeply.

Unlike the fire service, the BAE team do not arrive in a blaze of lights and sirens. They have to be more stealthy.

"If the attackers have access to the victim's email servers the last thing you want to do is discuss it on there," says Robin Oldham, head of the cyber-security consulting practice at BAE, who is also part of the incident response team.

Tipping off the bad guys could prompt them to delete evidence or, if they have more malicious motives, shut down key systems and destroy data, he says.

Instead, responders first gather evidence to see how bad the incident is and how far the hackers have penetrated a network.

It's at this point that the team use the skills picked up during earlier careers. All of the team have solid technical computer skills to which they have added particular specialities.

Prior to working at BAE, Mr Allman-Talbot did digital forensics for the Metropolitan Police and Mr Oldham has significant experience running large complex networks.

The good news about most organisations is that they typically gather lots of information about their network and often it is anomalies in the logs that expose suspicious activity.

But that extensive logging has a down side, says Mr Oldham.

"It can mean we have a large amount of data to work with and analyse. In some cases that means a few hundred million lines of log files."

Once incident response teams get their hands on data from a victim they start analysing it to see what has happened.

It's at this point that the allied discipline of threat intelligence comes into play. This involves knowing the typical attack tools and techniques of different hacking groups.

Good threat intelligence can mean responders hit the ground running, says Jason Hill, a researcher at security firm CyberInt.

"If you understand how they operate and deploy these tools and use them to attack the infrastructure you know what to look and how to spot the tell-tale signs."

In the past, nation state hackers have tried to bury themselves in a target network and siphon off data slowly.

"Criminal hackers have a more smash and grab mentality. They do it once and do it big," he says.

More recently, he adds, it has got harder to separate the spies from the cyber-thieves.

One example was the attack on Bangladesh's central bank - widely believed to have been carried out by North Korea. It netted the rogue state about £58m ($81m).

Russian groups also span both sides of the divide. Some criminal groups have been seen working for the state and often they use the tools gained in spying for other jobs.

"The motivations of the groups have really become blurry of late," says Mr Hill.

Attribution - working out which group was behind a breach - can be difficult, says Mr Allman-Talbot, but spotting that one attack shares characteristics with several others can guide the investigators.

One widespread attack, dubbed Cloud Hopper, sought to compromise companies selling web-based services to large businesses. Getting access to a service provider could mean that the attackers then got at all its customers.

Thoroughly investigated by BAE and others, Cloud Hopper has been blamed on one of China's state-backed hacking groups known as APT10 and Stone Panda. Knowing how they got at a victim can help free the hackers' hold on a network and reveal all the places that need cleaning up.

Even with up-to-date intelligence on attack groups and their chosen methods, there will still be unanswered questions thrown up by an investigation, says Mr Allman-Talbot.

The joy of the job comes from during investigations as the team figures out how the bad guys got in, what they did and what data they got away with, he adds.

BBC News is looking at how technology is changing the way we work, and how it is creating new job opportunities.

He likens it to solving complex puzzles and problems using experience, good hunches, deep analysis and coding skills. It's a challenging profession that regularly bestows solid intellectual rewards.

"There are lots of eureka moments," he says.

The deep knowledge built up by the responders as they investigate and clean up a breach can also help others that might not even know they have been penetrated, says Mr Oldham.

"There are people that see the smoke alarm go off and pick up the phone and tell us that something is wrong. There's others that we go to and tell them that their house is on fire," he adds.

Mr Allman-Talbot says some of the satisfaction with the job comes from helping people and making life online safer.

"Just as with criminal cases, there's a real sense of doing good. We are investigating incidents that have badly affected these organisations."

There's little doubt that the job is only going to more important as time goes on. The cyber-spies will not stop and are only going to get better at what they do.

"It's just going to get more and more complex," says Mr Allman-Talbot. "It's the next form of warfare."

Illustration by Karen Charmaine Chanakira