Most sessions on the internet today are encrypted. By any measure, more than half of all internet traffic uses TLS to encrypt client/server communication. Thanks to services like Let’s Encrypt, it is easier and more cost effective than ever to set up a web server that uses HTTPS to establish encrypted sessions for incoming traffic.
This is great for our everyday privacy as we engage with services on the Internet as part of our daily lives. It’s also great for attackers because it’s easier than ever to setup encrypted command-and-control communications channels.
The attacker perspective
Attackers don’t want you to know what they are up to and they are looking to blend in. There are a lot of options open to attackers for command and control. Just look at the command-and-control (C&C) tactic in the MITRE ATT&CK framework to get an idea of the menu of options.
Multistage channels, data encoding and multi-hop proxies come to mind as favorites. Attackers can use standard encrypted protocols like TLS or their own custom protocols. They can encrypt or encode the data in their channel before it even goes into the encrypted TLS tunnel.
The security team perspective
The security analyst wants to discover the attacker’s C&C channels and needs to do so even when the traffic is encrypted. To detect these C&C channels, we must assume encryption is in place. Therefore, we need to rely on metadata that is visible in case the channel is encrypted.
So, what metadata do you have left to look at when everything is encrypted? Data flows as a function of time, as shown in the figure below. Fortunately, the Vectra Cognito flow engine provides this data for each data flow it tracks, with sampling down to the half second. The image below shows the bytes sent and received over time, demonstrating the ebb and flow of an interaction that Cognito captures.
At Vectra, our journey to use this time-domain data effectively to detect command-and-control channels has taken various forms within supervised machine learning. We have discovered that several machine learning algorithms working together offer both a coarse and granular view into behaviors:
- Random forests, effectively a collection of decision trees, do a nice job of delivering coarse-grain telemetry. The random forest observes time-series windows and tracks over 20 features, including client/server data ratios, consistency of data, server break frequencies, and session length.
- Recurrent neural networks additively give the ability to represent temporal behaviors, where one sequence impacts the next and uniquely surfaces human-driven features. In effect, this replicates human memory.
- Long short-term memory (LSTM) deep-learning neural networks are capable of learning long-range temporal dependencies and relationships, including the ability to forget. This method is commonly applicable across a broad spectrum of use cases, including natural language processing. In security, it allows us to track relative behavioral patterns of attacker behaviors over long periods of time and correlate relevant activity over long periods of time.
The convergence of these methods, each of which has taken years to build and fine-tune, enables us to understand the behaviors of encrypted traffic and generate high- fidelity command-and-control detection alerts. The key to success with these methods is how you employ them and the training data that you use to train the model.
At Vectra, we are fortunate to have an award-winning team of data scientists and security researchers that develop algorithms, curate an understanding of available samples, and continually tune the algorithms to optimize performance.
Detecting encrypted command-and-control channels with high fidelity and low noise isn’t easy, and it’s not something that you can accomplish with a simple anomaly detector (trust me, we looked into it). It requires a mixture of the right data, the right machine learning method for the job, and the right team to build the model.