Ferret Out a RAT: Don’t Just Build a Better Mousetrap

August 3, 2020
Tim Wade
Deputy Chief Technology Officer
Ferret Out a RAT: Don’t Just Build a Better Mousetrap

Persistence techniques that evade endpoint protections have existed for a long time. The recently released Boothole (CVE-2020-10713) is yet another example of this. So why do so many security talking heads continue to prescribe incremental improvements on top of the same old solutions to address this problem? It’s time to rethink Detection and Response and use data from the one source that attackers can’t detect or evade—the network.

It’s not unusual for advanced persistent threats (APT) groups to go after Linux servers in an environment as a persistence and exfiltration gathering point. Having the ability to maintain persistence on these servers, even after an OS re-install, is the icing on the cake. One such vulnerability, codenamed ‘Boothole,’ was recently discovered by researchers at Eclypsium. This vulnerability in the GRUB2 bootloader utilized by most Linux systems that can be used to gain arbitrary code execution during the boot process, even when Secure Boot is enabled. Attackers exploiting this vulnerability can install persistent and stealthy bootkits or malicious bootloaders that could give them near-total control over the victim device while surviving OS-reinstalls.

Endpoint detection tools operate above this layer, making them blind to the activity of the attacker. The only approach to detect and evict attackers exploiting such techniques is to look at the network trace they leave as they remotely control the device. No wonder, BlackBerry research dubbed this decade the ‘Decade of the Remote Access Trojan (RAT).'

So how can we ferret out the RAT using network data? The secret lies in the threat modeling of the behavior. By analyzing numerous samples of RAT, Vectra threat detection team identified differences in communication patterns for a RAT communication compared to a regular one.

Consider the following example of time-series plots for the data transferred during two TCP sessions. The first TCP session represents traffic for a RAT and the second normal internet traffic. In the RAT time series you can see spikes in bytes received followed by bytes sent. Those alternating spikes are commands being issued from the external attacker and the infected host responding. Comparing that to a normal TCP session traffic, there is a clear difference. In live networks, these differences are nearly impossible for a human to discern given the sheer volume of data, the subtlety of the difference in communication, and the fact that the communication is within a single TCP session that may be encrypted.

While humans will struggle to see the differences when they look at real traffic, AI models that have seen thousands of samples excel at making this distinction. Specifically, a deep learning architecture known as long-short term memory (LTSM) recurrent neural networks retain ‘memory’ over time were specifically designed to work with this type of data. By applying LSTM based AI models to look for the communication pattern of a RAT in network data, they can be detected in real-time with high fidelity based on the behavior observed. The beauty of the approach is in the effectiveness. It works regardless of the specific tool or implant, and it is agnostic to encryption, not requiring any decryption of the traffic.

The Vectra approach to threat detection blends human expertise with a broad set of data science and advanced machine learning techniques. This model delivers a continuous cycle of threat detections based on cutting-edge research, global and local learning models, deep learning and neural networks. To learn more about our approach to data science read our white paper or take Vectra Cognito for a test drive to see for yourself.