Deep Packet Inspection (DPI) is an application-layer traffic detection and control technology frequently used by enterprises and Internet Service Providers (ISPs) to identify and block network attacks, track user behavior, block malware, and monitor network traffic.
DPI technology is hailed by technical experts and network managers as a crucial tool for addressing the number and complexity of internet-related dangers. DPI systems use the OSI modelâs application layer to extract statistical information, capable of locating, identifying, classifying, rerouting, or blocking packets containing specific data or code payloads.
DPI examines both the data and metadata associated with individual packets, whereas stateful packet inspection only evaluates header information, such as source IP address, destination IP address, and port number. When a packet approaches a checkpoint, DPI intercepts any protocol violations, viruses, spam, and other anomalies, preventing the packet from passing through the checkpoint.
What is the history of DPI?
DPI technology has a long history spanning over 30 years, dating back to the 1990s.
ARPANET was the first place where deep packet inspection appeared. The TCP/IP data transmission protocol was initially used by ARPANET, and engineers learned how to address UNIX security issues by managing raw packets using header and metadata.
In 1990, ARPANET was shut down, but as the modern internet became more widespread, TCP/IP issues became more prevalent. Network engineers created the OSI concept in the 1980s to standardize the metadata collected by the mid-1990s. The OSI model enables extensive statistical analysis by formalizing packet metadata levels. For example, auxiliary headers (also referred to as stateful data or shallow data) can reduce bandwidth while ensuring correct information routing.
Packet metadata allows ISPs to more easily distinguish different data categories, and deep packet capabilities may inspire new business models. Additionally, network neutrality has been a controversial issue for over two decades, and DPI technology could transform pipeline owners into data owners.
What is the difference between DPI and traditional packet filtering?
Every packet in a network has a header that contains basic information about its sender, recipient, and transmission time. Traditional packet filtering reads only this information. Older firewalls often operate this way as they cannot efficiently process other forms of data quickly.
Now, firewalls can address these issues through DPI for more thorough, continuous packet scanning. DPI extracts or filters data beyond the packet header to enable more comprehensive and sophisticated network monitoring and defense.
| Difference between DPI and traditional packet filtering (Source: okta)
DPI is an essential part of the network security ecosystem. Rather than just looking at incoming packets to detect protocol anomalies, DPI can analyze, find, and block packets as needed. This contrasts with standard network packet filtering, which sorts packets based on source and destination.
How does DPI work?
Traditional firewalls often lack the processing power for comprehensive real-time inspection of large traffic volumes. With technological advancements, DPI can perform more complex inspections, checking both headers and data. Typically, firewalls with intrusion detection systems frequently use DPI.
In a world where digital information reigns supreme, every bit of digital information travels across the internet in small data packets. This includes emails, messages sent via applications, accessed websites, video conversations, etc. In addition to the actual data, these packets also contain metadata identifying the traffic source, content, destination, and other important details. Using packet filtering technology, data can be continuously monitored and managed to ensure proper routing. However, traditional packet filtering falls short in ensuring network security.
Listed below are some key methods of deep packet inspection in network management:
- Pattern/Signature Matching
Firewalls with intrusion detection system (IDS) capabilities match each packet against a database of known network attacks. IDS searches for known malicious specific patterns and disables traffic upon detection. One downside of the signature matching strategy is that it only works with frequently updated signatures. Additionally, this technique can only combat known threats or attacks.
| Pattern matching in DPI (Source: OpenPR 2018)
- Protocol Anomalies
Since protocol anomaly technology does not simply allow all data that does not match the signature database, the protocol anomaly technology used by IDS firewalls does not have the inherent drawbacks of pattern/signature matching methods. Instead, it uses a default deny strategy. Based on protocol definitions, the firewall decides which traffic should be allowed, protecting the network from unknown threats.
- Intrusion Prevention Systems (IPS)
IPS solutions can block the transmission of harmful packets based on content, thereby stopping suspicious attacks in real-time. This means that if a packet represents a known security threat, IPS will proactively block network traffic according to defined rule sets. One downside of IPS is the need for frequent updates to the network threat database for new threats and the potential for false positives. However, these risks can be mitigated by creating conservative policies and custom thresholds, establishing baseline behavior for network components, and regularly evaluating alerts and reported incidents to enhance monitoring and alerting.
What is the architecture of DPI deployment in networks?
DPI engines are typically deployed inline with firewalls in routers, SDNs, and packet gateways. Non-critical analysis can also be performed via offline packet analysis.
DPI is a standard option in 4G LTE and 5G packet gateways (P-GW). For example, an ISPâs backbone network can be a 40-Gb/s system with four 10-Gb/s DPI modules.
In the dynamic service environment implied by cloud/SDN, due to the high demand for CPU resources, DPI may coexist with network devices (as software running in virtual switches) or in the control layer (in controllers between applications and switches). Real-time analysis using DPI is fed into big data analysis packages, which helps service providers understand what end users are doing and adjust service offerings accordingly.
| Inline-enabled DPI (Source: Accolade 2020)
Will real-time analysis by DPI slow down network data transmission speeds?
Currently, many DPI methods are resource-intensive and costly, especially for high-bandwidth applications. Since DPI is done in real-time, it is not suitable for general processors or switches.
In recent years, due to advances in computer engineering and pattern matching algorithms, DPI has been gradually made possible. Now, dedicated routers can perform DPI. Routers with program dictionaries help identify the intentions behind the LAN and Internet traffic they route, eliminating vulnerabilities from repeated attacks by known viruses.
| Network Traffic Statistics (Source: Mikov 2013)
What are the advantages of DPI?
When it comes to network performance for enterprises or any organization, DPI offers several significant advantages.
1) DPI is an essential tool for ensuring network security. DPI detects risks or blocks potential threats hidden in data by scanning packets beyond just the packet header. This makes it easier for enterprises to detect malware, prevent data leaks, and block other security threats to the network and its users.
2) DPI provides more options for controlling network traffic. DPI enables rule programming to search for specific data types and distinguish high/low priority packets. In this way, DPI can prioritize higher-priority or mission-critical packets throughout the data stream, sending these packets across the network before lower-priority communications.
3) DPI can be used to inspect outgoing traffic attempting to leave the network. With DPI, enterprises can discern the transmission location of packets, allowing them to develop filters for preventing data leaks.
4) The real-time processing of packets by DPI is subject to predetermined rules. According to preprogrammed rules implemented by the team, all data of packets, from headers to contents, is examined and automatically processed. The system automatically sorts, filters, and prioritizes each packet, preventing network slowdown.
5) DPI can react to traffic that matches profile configurations. For example, alerting users to discard packets or reducing the bandwidth accessible to such traffic.
The limitations of DPI
Though DPI is highly beneficial for network monitoring and security, there are several considerations when using DPI. While DPI provides certain security, it also introduces new vulnerabilities into the network.
1) DPI is excellent at blocking buffer overflows, DoS attacks, and certain malware attacks, but it can also be used to develop similar attacks.
2) DPI complicates existing firewalls and other security-related technologies, making them more cumbersome. To keep DPI rules effective, frequent updates and modifications must be ensured.
3) DPI can reduce network performance and speed because it introduces network bottlenecks and places more demands on firewall processors for online inspection and data decryption.
4) Some privacy advocates and net neutrality opponents might not support DPI because it can access specific information about the source and destination of information.
Do firewalls use DPI technology?
To defend networks, firewalls perform deep packet inspection, not just identifying threats and notifying teams. DPI is a technology adopted by next-generation firewalls (NGFWs) to ensure network security with features like content detection and intrusion detection. Specifically, standalone IDS aimed at identifying attacks and protecting networks, as well as firewalls with intrusion detection system functions, widely employ DPI.
Due to the integrated support for protocol and application classification with DPI, firewalls can classify network traffic in real-time down to the application level. With application visibility, firewalls can manage access rights, prioritize traffic, and optimize service quality for mission-critical applications. Most importantly, uninterrupted access to cloud services is always provided, and business networks are shielded from malware and cyberattacks.
How do DLP and DPI work together?
DLP (Data Loss Prevention) solutions often cover hundreds of different file types, with advanced content and context scanning tools already available. These tools have predefined rules for data protection standards and regulations (like GDPR, HIPAA, or PCI DSS) as well as intellectual property. DPI enables endpoint access to network functions, enhancing the flexibility and accuracy of DLP policy implementation.
By combining DLP solutions with DPI, enterprises can more easily whitelist or restrict specific websites to determine the exact location of file transmissions. It allows enterprises to decide which websites should be permitted for transmission and which should be restricted based on the situation.
Organizations can also whitelist domain names of email clients, limit sensitive data transmission to appropriate departments such as finance and HR, and block access from all other addresses. DPI is a great complement to DLP solutions because it enhances the accuracy of applying DLP policies. By automatically removing unwanted sensitive data transmission destinations while allowing use of valid channels, it can proactively reduce the impact of DLP on employee productivity.
What privacy concerns does DPI pose?
DPI is sometimes referred to as âfull packet inspection.â Given the large volume of traffic on most networks, DPI is usually automatically performed by software according to predefined criteria by the network operator. DPI can identify the content of all unencrypted network traffic, allowing ISPs to intercept nearly all network activities of customers, including web browsing data and emails, as most network traffic is unencrypted. After inspecting a userâs packet content, ISPs can use DPI to take actions based on filtering criteria. DPI has been used to attempt to:
- Enforce copyright laws
- Prioritize certain packets for transmission
- Identify computer viruses and spam
DPI also allows non-ISP service providers (e.g., search engines and webmail providers) to build user profiles based on internet usage. ISPs analyze packet headers for various purposes, including packet routing optimization, network abuse detection, and statistical analysis. This type of inspection is often referred to as âshallow packet inspection,â providing ISPs with basic information about internet traffic without disclosing the content of consumersâ emails or web browsing. However, DPI can enable ISPs to access the content of all unencrypted internet traffic transmitted or received by their customers. Therefore, DPI is controversial, and some privacy and net neutrality organizations oppose the use of DPI.
What are the use cases of DPI?
Network security relies on deep packet inspection capabilities that assess whether specific packets reach their intended destination through network traffic. DPI is not only about simply looking at incoming packets, but also analyzing, discovering, and blocking communications as needed. This contrasts with standard network packet filtering, which sorts packets based on source and target.
Another feature of DPI systems is packet-level analysis to find the root of application or network performance issues. It is considered one of the most precise methods for tracking and analyzing application behavior, network usage problems, data leaks, and other difficulties. Deep packet analysis also helps perform tasks such as:
- Measuring excessive network latency for mission-critical applications
- Enhancing application accessibility and fulfilling SLAs
- Creating historical data reports and conducting forensics
Additionally, DPI can help content owners prevent unauthorized content downloads. DPI can also be used for policy enforcement, delivering tailored advertisements to users, and conducting lawful interceptions.
What are some DPI software and tools?
Deep packet analysis is particularly useful in next-generation firewalls. As it is used as a component of intrusion detection systems (IDS) and intrusion prevention systems (IPS), the application of DPI has become increasingly widespread in recent years. DPI is often included as a feature within security devices or configured as virtual DPI on servers. While dedicated security/DPI devices are the optimal installation choice, users can also opt for software or services to implement DPI.
Below are some popular DPI tools:
1) Wireshark: A popular free open-source packet analyzer that can be configured for intrusion detection (ID). This utility allows filtering file contents from the command line using tshark to study network activity.
2) Netfilter in Linux: Classifies packets as HTTP, Jabber, Citrix, Bittorrent, FTP, etc., independent of ports.
3) Netflow from Cisco: Introduced on its routers to collect IP network traffic information when traffic enters/leaves an interface and build access control lists. It consists of a traffic collector and analyzer.
4) SolarWinds Netflow: A network bandwidth monitoring (collection and analysis) tool available in both free and paid versions.
5) Scrutinizer from Plixer: A tool capable of network traffic analysis for Cisco and other vendorsâ network devices.
How to choose DPI and analysis software?
With DPI software, organizations can deploy sensors, configure security metrics, and more. When selecting DPI and analysis tools, consider the following criteria:
1) There should be a packet scanner capable of reading headers and offloading SSL on proprietary networks to read payloads.
2) It should be a system observing network devices continuously.
3) It should offer the ability to switch from DPI to SPI (Stateful Packet Inspection).
4) To evaluate a program without payment, there should be a free trial or demo service available.