Due to some recent activity discovered while investigating an incident with a company I collaborate with, It was time for me to dive into the CIS Benchmarks and refresh what are the best practices to secure an Ubuntu server. I created a CIS Ubuntu Benchmark Summary here so you do not have to read over the more than a thousand pages that the original CIS Benchmark for Ubuntu 24.04 has.
When it comes to Linux devices, people have historically thought of them as a good server OS. However, the truth is that Linux is a widely spread operating system, not only for servers but also for computers. Moreover, Linux is one of the most commonly used IoT or embedded board OS types. Therefore, understanding Linux security is no longer something that server administrators can do and that we should be blind to; rather, it should be something that we should get more familiar with.
How can we protect our system?
According to the CIS Benchmark, we have hundreds of options and parameters that we should ensure that we consider in order to protect our systems. In order to simplify and understand the expectation, this suggestion will be broken down into five pieces: Package managers, App Armor, firewalls, SSH, and audit and logging.
Package managers:
To ensure that we keep our software secure, we download legitimate versions. There are three essential practices that we should ensure we take care of:
1- Ensure that we have some GPG keys installed and configured. It might seem obvious to some people that we must ensure the identity of the other party when we are downloading software. But because this is a behind-the-scenes task, it is easy to overlook this simple step. As we are so used to web browsing, we look at the fact that a website has a valid certificate with the name of the web. When it comes to software, we should ensure the same thing: that we know that the identity of the software provider is who we think it is. And that the piece of software we receive and install has not been tampered with (More info: https://manpages.debian.org/stretch/apt/sources.list.5.en.html)
2- Check the package management sources. We must download software from legitimate providers. Therefore, following along the same lines as the first recommendation, we should also check that our package management source file points to the official sources. Can we trust the keys if we verify the PGP keys provided by a nefarious source? That is why both these steps are essential and encompassing.
3- Finally, when it comes to package managers, we should ensure that we use them! This ensures that we keep the software up to date, especially regarding security updates. Many package managers classify the different updates available for a particular package. But the most basic and essential of them is the security updates. We can use something like
App Armor
It is a security feature installed in the Linux Kernel, and it can help us determine what access each application can have—allowing for the compartmentalization of each application. This means that if one application gets compromised, it will not be able to compromise or access the rest of the system. App Armour is installed by default in many Linux distributions (e.g., Debian, Ubuntu, OpenSUSE).
Moreover, to ensure that we get the most out of it, we should ensure that App Armor is enabled during the system boot. This would prevent any attack from succeeding during the boot process. We need to configure the enablement in the bootloader to enable it. (More Info: https://apparmor.net/)
App Armor has two modes: enforce and complain; enforcing the program is equivalent to blocking and reporting. Ideally, this is the state we want to be in. Nevertheless, this might generate some issues with the functionality of different applications. Therefore, at the beginning of the implementation, we should enable the app Armor in complain mode, allowing us to see potential errors with the configuration of the systems without impacting the system functionality.
Firewalls
Regarding Linux firewalls, we have different options; the three most common are UFW, NFtables, and IPtables.
These three are the most common host-based firewalls on Linux machines. Even though the older IPtables are phasing out. If you have any systems currently running IP tables, we should ensure that we plan a transition to UFW or NFtables. And even though it might sound silly, we must ensure that we only have one of these Firewalls enabled at any time.
When it comes to the configurations of the firewalls, it is kind of the same on any of the three firewall options:
1- Ensure that we allow internal communications to localhost
2- If we need to allow inbound traffic to a specific port, we should generate a specific rule allowing traffic and apply a DENY ALL for the rest of the connections
3- Allow outbound connections.
Be aware that the NFtables configuration is volatile. This means if you do not save the configurations, the rules will work until the machine reboots; at this point, all the rules that were not saved would have been lost.
SSH Configuration
SSH is still the most common way for administrators and users of servers to access the machines remotely. Therefore, ensuring SSH security is paramount for any server. Nevertheless, we do not need that functionality on desktops. In that case, the SSH daemon can be removed.
1- Check on file permissions:
Ensure that the permissions on the /etc/ssh/sshd_config files are correct and that the SSH private and public host key files are correct. Only the root user should have read capabilities to those files as they contain some critical data.
2- Allow users and groups explicitly within the ssh config file.
3- Ensure that strong ciphers are used on ssh. Connections. The ssh protocol can be designed to keep our systems secure. However, using weak ciphers would be the equivalent of having a security door installed but not using a deadbolt. Even if the door is safe, we must use a deadbolt to ensure no unauthorized person enters.
4- Configure the timeout for sessions by using the ClientAliveInterval and ClientAliveCountMax
5- Unless you need graphical connections over SSH, remove X11 forwarding.
6- Ensure sshd DisableForwarding is enabled
7- Ensure GSSAPIAuthentication and HostbasedAuthentication are disabled
8- Ensure sshd IgnoreRhosts is enabled
9-Ensure sshd KexAlgorithms are configured. Key exchange configurations are essential because they determine how the sender and the receiver share encrypted messages. As previously mentioned, the ciphers allowed to exchange such messages should be refined to exclude any weak ciphers from the list.
10- Set a sshd LoginGraceTime; this parameter will define the maximum time a user can take to authenticate through SSH to establish a session. Setting this a lower number helps prevent attackers from depleting the server by leaving many unauthenticated connections. The recommended value for this setting is 60s. However, depending on the use case, it may be possible to reduce it even further.
11- Ensure the right LogLevel is configured; the SSH service must be set up to generate logs; different security engines can use these logs to analyze the activity occurring on the SSH protocol. Moreover, proper logging will allow for a more effective forensic investigation if a security incident occurs. It is recommended that this setting be at the INFO or VERBOSE level.
12- Ensure that the MaxAuthTries parameter is configured. This would be extremely helpful, especially when preventing brute-force attacks.
13- Ensure that we configure MaxSessions to allow just the number of concurrent sessions we expect on the server. This will prevent attackers from leaving a bunch of open sessions hanging there, which might overload the daemon and generate a DoS attack.
14-Configure MaxStartups; similarly, we should only allow a certain number of concurrent unauthenticated sessions. So this max number is not necessarily the number of attempts of connections or established connections but the number of connections that have started a connection but have not completed the authentication process yet.
15- Disable PermitEmpty Passwords; this is important to prevent attackers from attempting to connect to different users without a password. As we know, we should not allow any user to have an empty password, so enabling this parameter makes no sense.
16- Disable PermitRootLogin. We know that the root user should never be allowed to use SSH. Users should use their accounts to log in via ssh and, if needed, use the sudo command to perform root-level operations.
17- Enable the use of PAM. The Pluggable Authentication Module (PAM) interface allows us to use different authentication methods.
Audit and logging
Audit logging is a cornerstone to ensure the machine’s security; it ensures that we have some logs generated on the system that can later be used in investigations or through different IDS/IPS systems to detect unusual activity.
We ensure that the logging service (rsyslog – OR – journaled) is enabled to allow the capturing of logging events. We should ensure that the logs get started from the moment of the boot.
Now, enabling logging, especially on a low level, might generate an overhead of disk usage by the amount of logs being stored. We can use several methods to handle any potential issues in this regard. In the case of rsyslog, it has a log rotation capability that can be set up from: /etc/logrotate.d/rsyslog.
After we know that the logs are being generated, we must ensure that the auditing services(audited) are set as expected; this includes ensuring that the auditing for processes that start before the audit is
enabled in the GRUB. This would ensure that the logging capability begins early during the boot process to maximize the visibility of the actions taken in the system.
Regarding the logging sizes, we should configure things like the audit_backlog_limit. This would allow us to ensure that the queue of logs to manage does not grow out of proportion. In this regard, we should also set the max_log_file in MB. This parameter can be found inside the /etc/audit/audit.conf.
When the max log file reaches the limit, the file will be compressed, and a new file will be started. Ensuring that the log files remain manageable. Nevertheless, we must also ensure that the logs are not deleted automatically when we need more space. This could be detrimental to any investigation efforts. But rather
We have several parameters that we can take advantage of to ensure that we do not run out of system space but keep logging capabilities. Some of these actions are: space_left_action and admin_space_left_action. Both of these can be configured within audited.
Nevertheless, what should we log?
In the benchmark, there is a clear distinction between two different audit rules: file system rules, which are the ones generated when accessing a specific file or directory, and system call rules, which track system calls made by the system.
Logging things such as modifying files that control systems administration privileges like changes to the sudoers file, recording actions taken by users taking another identity, like when users use the su command. Monitoring modifications to the sudo log file, tracking changes to the system date and time information, alterations to the system network environments, and using any privileged command.
When it comes to file systems, we should monitor:
We should add logs for any mounted filesystem. In that case, it might generate many logs, so if we focus on auditing relevant filesystems, we can be more strategic and avoid overflowing logs. This can be accomplished by monitoring the mounted file systems that use the noexec and nosuid mounting options.
We should track system events such as unsuccessful attempts to access files, events that modify user or group account information, and recording changes to the DAC permissions, like using the chmod or the chown commands. For the successful mounts of file systems, we should also monitor session initiation information, logins, and log-out events, including failed login attempts. Something that might seem obvious but can be overlooked is the instances of users deleting files.
On that same line, we should audit for modification of MAC policies, with changes on the app armour profiles or using commands like chcon, chancel, and usermod.
The loading/unloading and modification of kernel modules are interesting to monitor, as they should remain relatively constant.
One of the last things to check is whether the audit configuration has some overshared permissions. We should ensure that regular users can’t modify or access the audit configuration file, which the root user should own. Moreover, we should ensure that the audit configuration running is the same as the one stored on disk
Conclusion
To sum up, different use cases can lead to other recommendations, and that means that if you are using Ubuntu as a desktop system, it might not require the same level of hardening as a server. For instance, some of the services mentioned in this article, like the SSH server or the SSH daemon, should be turned off for desktop users. We should only have a client version of ssh, not the server on a desktop.
Remember that security should be based on layers, so do not rely only on server hardening to ensure your application is secure. We must also look at the network layer and the application to provide some redundancy in the security measures. If you do not know where to start, check some DevOps concepts to understand how you can better integrate security in your lifecycle.