How To Monitor Linux Network Usage - Top Network Monitoring Tools

Features of NetHogs

1. Process-level monitoring: NetHogs monitors network traffic at the process level, linking bandwidth usage with PIDs.
2. Real-time updates: It provides real-time info about network activity, updating the display every second by default.
3. Refresh rate: You can adjust the refresh rate using the -d option followed by the interval in seconds.
4. Sorting options: NetHogs lets you sort the output by sent or received traffic using the -s option followed by sent or recv.
5. Selecting interfaces: By default, NetHogs monitors all network interfaces, but you can choose an interface using the -i option followed by the interface name.

Examples

Example 1: Finding a Process Using a Lot of Bandwidth

If you notice a sudden increase in network traffic on your Linux server, to find the process responsible for the increased bandwidth usage:

1. Run NetHogs in a terminal: sudo nethogs
2. Look at the output and find processes with high sent or received traffic.
3. Once you find the problem process, you can look into it more and take action, such as stopping the process or changing its settings.

Example 2: Monitoring Traffic on an Interface

If you want to monitor network traffic on an interface, such as eth0:

1. Run NetHogs with the -i option: sudo nethogs -i eth0
2. NetHogs will show bandwidth usage for processes using the interface, making it easier to focus on relevant traffic.

NetHogs Output

| PID | USER     | PROGRAM                  | DEV  | SENT      | RECEIVED  |
|-----|----------|--------------------------|------|-----------|-----------|
| 1234| john     | /usr/bin/firefox         | eth0 | 500 KB/s  | 1.2 MB/s  |
| 5678| www-data | /usr/sbin/apache2        | eth0 | 250 KB/s  | 800 KB/s  |
| 9012| root     | /usr/sbin/sshd           | eth0 | 10 KB/s   | 5 KB/s    |

In this output, NetHogs shows the PID, username, program name, network device, and bandwidth usage (sent and received) for each process. This info can help you quickly find which processes are using the most bandwidth and take action.

Features of nload

1. Real-time monitoring: nload updates network usage information in real-time, providing up-to-date insights into network activity.
2. Traffic visualization: The tool shows graphs for incoming and outgoing traffic directly in the console, making it easy to spot trends and anomalies.
3. Bandwidth usage statistics: nload shows the current, average, minimum, and maximum network usage, as well as the total data transferred.
4. Customizable display: You can customize the display by changing the refresh interval and selecting the units for bandwidth (e.g., bits, bytes, kilobits, megabits).
5. Multi-device monitoring: nload can monitor multiple network interfaces at the same time, allowing you to keep track of network activity across different devices.

Example Usage

To start monitoring network traffic with nload, run the command:

nload

By default, nload will show network usage information for all available network interfaces. To monitor a specific interface, use the -d option followed by the interface name:

nload -d eth0

To change the refresh interval, use the -t option followed by the desired interval in milliseconds:

nload -t 1000

This command will set the refresh interval to 1000 milliseconds (1 second).

Real-life Scenarios

1. Troubleshooting network issues: When experiencing slow network performance or connectivity problems, nload can help identify if the issue is related to high bandwidth usage or network congestion. By monitoring the real-time traffic, you can determine if there are any unusual spikes or sustained high usage periods that may be causing the problem.

2. Capacity planning: nload can be used to gather network usage data over time, which can help in capacity planning. By analyzing the average and peak usage, you can determine if your current network infrastructure is enough to handle the load or if upgrades are necessary to accommodate future growth.

3. Detecting anomalies: By regularly monitoring network traffic with nload, you can establish a baseline for normal network behavior. If you notice any sudden changes or unusual patterns in the traffic, it may indicate a potential security breach, such as a denial-of-service attack or unauthorized access attempts.

Limitations

While nload is a useful tool for monitoring overall network usage, it does not provide detailed information about network bandwidth consumption by individual processes or process IDs (PIDs). If you need to analyze network usage at the process level, tools like NetHogs or iftop may be more suitable.

Example Commands

To display all active TCP connections and the process using each connection:

netstat -atp

This might output:

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 localhost:mysql         0.0.0.0:*               LISTEN      1234/mysqld         
tcp        0      0 0.0.0.0:http            0.0.0.0:*               LISTEN      5678/apache2        
tcp        0    316 192.168.1.5:ssh         203.0.113.10:12345      ESTABLISHED 9012/sshd: user

To show routing table information:

netstat -r

Example output:

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
default         192.168.1.1     0.0.0.0         UG        0 0          0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U         0 0          0 eth0

To display network interface statistics:

netstat -i

Sample output:

Kernel Interface table
Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0      1500 12345678      0      1      2  8765432      0      0      0 BMRU
lo       65536  1234567      0      0      0  1234567      0      0      0 LRU

By combining these options, you can use netstat to gather data about your network usage and diagnose issues.

Real-world Applications

Consider a web server that has intermittent timeouts when clients connect. You could use netstat to:

1. Check if the web server process is listening on the right port:

   netstat -anlp | grep :80
   

   If you see a line like tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1234/apache2, the web server is listening on port 80.

2. Look for many connections in the TIME_WAIT state, which could mean a misconfigured load balancer or clients disconnecting improperly:

   netstat -an | grep TIME_WAIT | wc -l
   

   If this number is very high (e.g., thousands), it suggests connections aren't closing cleanly.

3. Monitor the rate of new connections to see if the server is overwhelmed:

   netstat -an | grep SYN_RECV | wc -l
   

   Many connections in the SYN_RECV state could mean the server is getting connection requests faster than it can process them.

While netstat provides detailed data, it only shows a snapshot of the current network state. For continuous monitoring, you would need to run netstat periodically or use a tool that builds on top of netstat, such as netdata.

Also, while netstat can show process names and PIDs using the -p option, tools like nethogs and iftop are better for monitoring bandwidth usage by process or connection over time.

netstat is an important tool for any Linux system administrator or developer who needs to understand and debug network behavior. Its ability to show detailed connection data, routing tables, and interface statistics make it valuable for troubleshooting. By knowing how to use netstat, you can quickly find sources of network issues and keep your systems running smoothly.

iftop Features

1. Real-time monitoring: iftop updates the information every 2, 10, or 40 seconds, giving you a near real-time view of network traffic.
2. Bandwidth usage by connection: iftop shows the bandwidth usage of each connection, helping you find which connections use the most bandwidth.
3. Host resolution: By default, iftop resolves IP addresses to host names, making it easier to identify the hosts in each connection.
4. Display options: You can change the iftop display by sorting connections by bandwidth usage, changing the display mode, and setting the screen refresh interval.
5. Filtering: iftop lets you filter the connections by source or destination host, network, or port.

Examples

Example 1: Monitoring an Interface

To monitor bandwidth usage on an interface, such as eth0, run:

sudo iftop -i eth0

This command will show real-time bandwidth usage for each connection on the eth0 interface.

Example 2: Finding High Bandwidth Connections

To find which connections use the most bandwidth:

1. Run iftop: sudo iftop
2. Press s to sort the connections by bandwidth usage, with the highest usage at the top.
3. Look at the connections at the top of the list to see which hosts are involved and how much bandwidth they use.

For example, if you see a connection between your web server and a client using a lot of bandwidth, it could mean the client is downloading a large file or there might be an issue with the application causing too much data transfer.

Example 3: Using iftop with netstat

To find the processes causing high bandwidth usage:

1. Run iftop and find the high bandwidth connections as shown in Example 2.
2. Write down the source and destination IP addresses and ports for the connections you want to check.
3. In another terminal, run sudo netstat -anp | grep <ip>:<port>, replacing <ip> and <port> with the values from step 2.
4. The output will show the process ID (PID) and name responsible for the connection.

For example, if you find that a connection between your server and a remote host is using a lot of bandwidth, you can use netstat to find which process on your server is responsible. This can help you see if the bandwidth usage is normal (e.g., a backup process or a large file transfer) or if it points to a problem (e.g., malware or a misconfigured application).

iftop Output

                   1.2Mb            2.4Mb            3.6Mb            4.8Mb       6.0Mb
└─────────────────┴───────────────┴───────────────┴───────────────┴───────────────
localhost                                        => example.com                      1.08Mb  1.08Mb  1.08Mb
                                                 <=                                  2.55Mb  2.55Mb  2.55Mb
localhost                                        => example.org                        0Kb   676Kb   442Kb
                                                 <=                                   68Kb   102Kb    68Kb

In this output, iftop shows the bandwidth usage for each connection, with the source host on the left and the destination host on the right. The arrows show the direction of the traffic, and the three columns on the right show the average bandwidth usage over the last 2, 10, and 40 seconds.

Understanding iftop Output

The table above shows how to understand the iftop output. For each connection, you can see the source and destination hosts, as well as the average outbound and inbound bandwidth usage over the last 2, 10, and 40 seconds. This information can help you find which connections use the most bandwidth and in which direction the traffic is going.

Limitations

While iftop is useful for monitoring network bandwidth usage by connection, it has some limits:

1. Needs root privileges: iftop requires root privileges to listen to network traffic, which may not be ideal in all environments.
2. Doesn't show process-level information: Unlike tools such as NetHogs, iftop doesn't directly show which processes are responsible for the network traffic. However, you can use iftop with other tools like netstat to get this information, as shown in Example 3.
3. Limited filtering options: While iftop allows filtering by host, network, and port, it doesn't offer more advanced filtering options, such as filtering by protocol or packet size.

Installing speedometer

To install speedometer on Debian-based distributions, use apt:

sudo apt install speedometer

On Red Hat-based distributions, speedometer is available in the EPEL repository:

sudo yum install epel-release
sudo yum install speedometer

Using speedometer

To start monitoring network traffic with speedometer, use the command:

speedometer -r eth0 -t eth0

This command will display two graphs: one for incoming (-r) traffic and one for outgoing (-t) traffic on the eth0 interface.

Customizing the Display

speedometer allows you to customize the display in various ways:

For example, to display the incoming and outgoing traffic on eth0 with a 2-second interval and a fixed scale of 10 Mbps:

speedometer -r eth0 -t eth0 -i 2 -s 10M

Displaying Multiple Interfaces

One of speedometer's advantages over nload is the ability to display multiple graphs side by side for different interfaces. To monitor traffic on multiple interfaces simultaneously, specify each interface with its own -r or -t option:

speedometer -r eth0 -t eth0 -r wlan0 -t wlan0

This command will display four graphs: incoming and outgoing traffic for both eth0 and wlan0.

Real-life Examples

Monitoring a Web Server

Suppose you have a web server with two network interfaces: eth0 for incoming traffic and eth1 for outgoing traffic. To monitor the bandwidth usage on both interfaces, you can use the following command:

speedometer -r eth0 -t eth1 -i 5 -s 100M

This will display two graphs, one for incoming traffic on eth0 and one for outgoing traffic on eth1, updated every 5 seconds with a fixed scale of 100 Mbps. This setup allows you to quickly see if the server is reaching its bandwidth limits and take action if necessary.

Comparing Wired and Wireless Interfaces

If you have a laptop with both wired (eth0) and wireless (wlan0) interfaces, you can use speedometer to compare their performance:

speedometer -r eth0 -t eth0 -r wlan0 -t wlan0 -i 2 -s auto

This command will display four graphs, showing the incoming and outgoing traffic for both interfaces, updated every 2 seconds with an automatic scale. By comparing the graphs, you can see which interface is faster and more stable, helping you decide which one to use for bandwidth-intensive tasks.

Use Cases

1. Monitoring server bandwidth: When running speedometer on a Linux server, you can quickly see the current network usage and identify if any interfaces are approaching their limits. This can help you make decisions about upgrading network infrastructure or optimizing applications.

2. Troubleshooting network issues: If users are reporting slow network performance, running speedometer can help you see if any interfaces are saturated with traffic. You can then investigate further to find the cause of the high usage (e.g., a backup job, a misbehaving application, or a security breach).

3. Comparing interface performance: By displaying graphs for multiple network interfaces side by side, speedometer makes it easy to compare their relative performance. This can be useful when load balancing traffic across multiple links or when diagnosing issues with a specific interface.

While speedometer provides a more customizable display than nload, it still lacks some features found in other tools. Like nload, speedometer doesn't break down traffic by connection or process, so it's less useful for finding specific sources of network activity. Tools like iftop and nethogs are better suited for those use cases.

Real-life Scenarios

1. Identifying top talkers: By using NetFlow to monitor traffic flows, administrators can identify the top talkers on the network - the hosts, applications, or users using the most bandwidth. This information can help in capacity planning, fixing performance issues, and enforcing network usage policies.

   For example, an administrator at a university notices that the network is slow during peak hours. By analyzing NetFlow data, they find that a small group of students are using a large amount of bandwidth by streaming high-definition video. The administrator can then work with the students to find other solutions, such as using a local media server or limiting video quality during peak hours.

2. Detecting DDoS attacks: NetFlow can help detect distributed denial-of-service (DDoS) attacks by identifying unusual spikes in traffic from multiple sources. By setting thresholds and alerts in a NetFlow analyzer, administrators can quickly respond to attacks and take measures to lessen their impact.

   For instance, a company has a sudden surge in traffic that overwhelms their web servers. By analyzing NetFlow data, the security team identifies a large number of incoming flows from a range of IP addresses, indicating a DDoS attack. They quickly implement traffic filtering rules to block the malicious flows and restore normal operation.

3. Optimizing application performance: NetFlow can provide insights into application traffic patterns, helping administrators understand how bandwidth is being used by different applications. This information can be used to prioritize critical applications, adjust Quality of Service (QoS) settings, and ensure that applications have enough bandwidth to perform well.

   For example, a call center relies on a Voice over IP (VoIP) system to handle customer calls. By monitoring NetFlow data, the IT team notices that VoIP traffic is competing with less critical traffic, resulting in poor call quality. They use this information to adjust QoS settings, prioritizing VoIP traffic over other types of traffic to ensure a better user experience.

4. Capacity planning: By analyzing NetFlow data over time, administrators can identify trends in network usage and plan for future capacity needs. For example, if traffic to a particular application or server is consistently growing, additional bandwidth or hardware resources may be needed to avoid performance bottlenecks.

   For instance, an e-commerce company monitors their network traffic using NetFlow and notices a steady increase in traffic to their web servers over several months. By extrapolating this trend, they can estimate when they will need to add more server capacity to handle the increased load and budget accordingly.