Sandfly 1.5.0 Released – Enhanced Linux Process Forensics, Rootkit and Network Sniffer Hunting

Sandfly 1.5.0 has been released with many new detection methods for Linux rootkits, malware and suspicious activity. The latest update expands on our already thorough intrusion detection and threat hunting net for Linux.

This version of Sandfly improves performance and focuses on:

  1. Enhanced process forensics and network activity reporting.
  2. Retrieving cryptographic hashes from suspicious binaries even if deleted from the disk.
  3. Open network and file descriptor sockets in use by processes.
  4. Hunting for suspicious sniffer pcap files hiding on a host.
  5. Hunting for rootkit hiding techniques involving the /proc file system.
  6. Detecting malicious network sniffers operating on a host.
  7. Searching for network tools such as netcat, socat, and other processes masquerading under different names.

Enhanced Process Forensics Network Activity Reporting

Whenever a process is flagged as suspicious we will include the full network profile of what it is doing. We did this before for network specific threats, but now we do network profiling for all processes whether we flagged it for suspicious network activity or not.

This means you can see what ports a process has open and what IP addresses that process is communicating with every time. This allows you to quickly determine if a flagged threat has any network activity associated with it you need to investigate.

Network Process Ports Open
Network process ports that are open.

Process Cryptographic Hashes

Sandfly will show you the cryptographic hash of the binary behind a malicious process even if the binary has been deleted from the disk. A lot of Linux malware removes itself from the disk once it goes memory resident to conceal itself. However, Sandfly is still able to get the hashes of these files regardless. You can use these hashes to run them through online databases to see if the threat is known or not. Sandfly supplies MD5, SHA1, SHA256 and SHA512 hashes for you to use for this purpose.

Linux Process Cryptographic Hashes
Linux process cryptographic hashes.

Process Open Sockets and File Descriptors

Sandfly will show all open sockets and file descriptors for a process. This is not only network sockets, but local Unix file descriptors on the disk along with open terminals, etc. that are attached to a process. This is useful for digging into suspected backdoors and other malicious network tools that are piping data through sockets on the disk or over the network.

Linux Network Process File Descriptors Open
Linux network process file descriptors that are open.

Flagging Binary Files

Files that are flagged by Sandfly during an automated investigation will also have a new attribute called file_is_binary. Sandfly will determine if a file is binary or not during the investigation regardless of the extension. This gives a quick indicator if a malicious file is trying to masquerade as non-binary when it really isn’t (e.g. a file with a .txt extension but really contains binary data).

Linux Binary File Masquerading as Text.
Linux binary file masquerading as text.

Hunting for PCAP Files

Sandfly now has several new investigation types that specifically hunt for pcap (packet capture) type files on the remote system in unusual or suspicious locations. The pcap format is the industry standard for network sniffing tools like tcpdump. Sandfly will flag pcap files that are in unusual or suspicious locations such as binary directories, temp directories, lib directories, etc. We also have incident response sandflies that will flag pcap files anywhere on the file system, and also any pcap files that are hidden anywhere.

Pcap files are used as part of normal system admin tasks with tcpdump. However, when pcap files are in unusual system areas or hidden they could be part of an attacker’s arsenal to grab network traffic such as passwords or other activity. Sandfly hunts for pcap files in suspicious locations because they can be an indicator of a malicious sniffer on a Linux system. Pcap files in user home directories are ignored by default to prevent false alarms, but can be searched for with incident response sandflies detailed below.

Sandfly Detects Suspicious pcap File in /tmp Directory
Sandfly detects suspicious pcap file in /tmp directory.

Process with PCAP File Open

Like hunting for pcap files above, this Sandfly will flag processes that have open pcap files on the disk in suspicious locations. This sandfly can help spot rogue sniffers that may be operating on the system but are sending their data to a pcap file for later use and retrieval. Below is a partial capture of a malicious network sniffer operating on a Linux host:

Sandfly Process Pcap File Open
Sandfly detects a suspicious process writing to a pcap file under the /tmp directory.

Processes Running from /proc

The /proc file system on Linux is special in that it normally just contains kernel information and is not writable as a normal file system would be. There are techniques available though where attackers run processes from this directory in an effort to conceal their presence on the host. Binaries located under /proc that are running are extremely suspicious and Sandfly will flag them now, along with whether they have open network sockets as well.

Sandfly spots a malicious network daemon from /proc with full process forensics for Linux.
Sandfly spots a malicious network daemon from /proc with full process forensics for Linux.

Process Masquerading Detection for tcpdump

The program tcpdump is a powerful sniffer that is loaded by default on most Linux distributions. While it’s great for debugging network issues, it is a true gift to an attacker looking to quickly grab network traffic from a compromised host. Sandfly will look for tcpdump running on a system but has been renamed to hide what it really is. This is a common tactic with some rootkits that establish persistence and then use the tcpdump program to grab more credentials or operate covert channel backdoors to allow access onto the host.

Below is a screen capture of the abbreviated view of this alert:

Sandfly spots a malicious tcpdump masquerading under a different name.
Sandfly spots a malicious tcpdump masquerading under a different name.

Process Masquerading Detection for netcat

Like tcpdumpnetcat is another powerful networking tool installed on Linux by default in many cases. While netcat has many legitimate uses, for an attacker it frequently is used for backdoors, exfiltrating data, or other malicious activity. Like tcpdump, if Sandfly sees netcat running under another name we will let you know because the system is likely compromised.

Linux process masquerading for netcat.
Linux netcat process masquerading as something else.

Process Masquerading Detection for socat

Socat is another tool like netcat for network socket operations. It has the same legitimate, and illegitimate use potential. Socat masquerading under another process name is likely malicious as well.

Linux process masquerading for socat.
Linux socat process masquerading.

Process Masquerading Detection for Shells

We have expanded the ability to detect system shells that are operating under a masqueraded name. Renamed system shells are commonly done for backdoors and other malicious activity.

Sandfly Detects a Linux Shell Masquerading As Another Process Name
Sandfly detects a Linux shell masquerading as dbus-daemon running a malicious script.

Processes Running from Suspicious Directories

We have had checks for suspicious directories on Linux a while. These help to spot weird directories like /bin/…, /tmp/. ., and so on. We now apply these criteria to running processes. If we see a process that is running from a suspicious directory, we will flag it and profile the process and file binary. Almost 100% of the time a process running from a suspicious directory is malicious and trying to hide.

Linux Process Running from Suspicious Directory.
Linux detects a process running from a suspicious directory.

Incident Response Sandflies Expanded

We’ve also expanded the incident response sandflies to hunt for variants of all of the above. For instance you now have the ability to hunt for hidden pcap files anywhere on a file system, or just pcap files in general if you want to find out where a sniffer may have left data behind anywhere on the file system. This allow incident responders to quickly search for threats that may have been using built in tcpdump on Linux to steal data from the wire.

Sandfly Helps Incident Responders Find Hidden Pcap Files
Sandfly helps incident responders find hidden pcap files that were used as part of a rogue sniffer on a host.

Upgrading is Easy

Upgrading Sandfly is easy. Please follow the instructions outlined in the documentation below:

Upgrading Sandfly

Keep on Hunting

Sandfly’s agentless threat hunting and intrusion detection for Linux is expanding constantly. We thank you for using Sandfly to help protect your Linux systems. Interested? Try it today.