Linux: Essential Commands for IR Triage

 By Charles Joseph | Cybersecurity Advocate

 Last update: November 25, 2023

Imagine you’ve just been hacked.

You’re shocked and overwhelmed, and now your boss is breathing down your neck, demanding answers.

You know that you need to act quickly — your company’s bottom line, and maybe your job, depends on it.

But there’s a problem.

As the system administrator, you knew the importance of having an incident response plan in place. But sadly, you weren’t proactive, and now you’re flying blind.

No worries; we’ll help guide you through the initial triage.

As a practical matter, if you can identify the source of the intrusion, consider unplugging it from the network and taking a snapshot of its memory. This information is volatile, so time is of the essence.

If the system is essential for business operations, you’ll have to make the difficult choice of either 1.) keeping it online while you diagnose and address the hack, or 2.) pulling the network cord.

Ultimately, this will be a decision for the stakeholders.

If you can’t identify the source, quickly getting up to speed on how your hosts are configured will help you make more informed decisions.

In this article, we’ll address *nix hosts.

The following section will list the most essential commands to help with the initial triage.

So let’s get to it.

Essential Commands for Incident Response Triage

During an incident response, you should gather as much information as possible to understand the scope and impact of the incident, determine its cause, and plan for remediation.

Here’s a list of information you may want to collect from a *nix host, along with the commands to gather the data:

1. System information:
   • Hostname: hostname
   • Operating system: uname -a
   • Distribution and version: cat /etc/os-release
2. User and process information:
   • List of users: cat /etc/passwd
   • Currently logged-in users: w
   • Running processes: ps aux or ps -ef
3. Network information:
   • Network interfaces and IP addresses: ip addr or ifconfig
   • Listening ports and associated processes: netstat -tuln or ss -tuln
   • Active connections: netstat -tupn or ss -tupn
4. File system information:
   • Mounted file systems: df -h
   • Directory listings of important locations (e.g., /var/log, /tmp): ls -l /var/log and ls -l /tmp
5. Logs and audit trails:
   • System log: cat /var/log/syslog or cat /var/log/messages
   • Authentication logs: cat /var/log/auth.log or cat /var/log/secure
   • Cron logs: cat /var/log/cron.log
   • Custom application logs (depends on your application’s log location)
6. Bash history:
   • Bash history for each user: cat /home/username/.bash_history (replace “username” with the actual usernames)
7. Security tools output:
   • Check for rootkits: Use tools like chkrootkit or rkhunter
   • Analyze system for malware: Use tools like clamscan (from ClamAV)
8. Other important information:
   • Installed packages: dpkg -l (Debian/Ubuntu) or rpm -qa (Red Hat/CentOS)
   • Scheduled tasks: crontab -l (for each user) and cat /etc/crontab
   • Startup services: systemctl list-unit-files --type=service or chkconfig --list

Remember to run these commands as a privileged user (e.g., root) to ensure you can access all necessary information.

You may need to modify the commands depending on your Linux distribution and version.

Additionally, consider collecting this information in a structured way, such as creating a script to automate the data-gathering process and output the information to a single file or a series of files for analysis.