Search Images Maps Play YouTube News Gmail Drive More »
Sign in
Screen reader users: click this link for accessible mode. Accessible mode has the same essential features but works better with your reader.

Patents

  1. Advanced Patent Search
Publication numberUS6772334 B1
Publication typeGrant
Application numberUS 09/655,459
Publication dateAug 3, 2004
Filing dateAug 31, 2000
Priority dateAug 31, 2000
Fee statusPaid
Publication number09655459, 655459, US 6772334 B1, US 6772334B1, US-B1-6772334, US6772334 B1, US6772334B1
InventorsGregor A. Glawitsch
Original AssigneeNetworks Associates, Inc.
Export CitationBiBTeX, EndNote, RefMan
External Links: USPTO, USPTO Assignment, Espacenet
System and method for preventing a spoofed denial of service attack in a networked computing environment
US 6772334 B1
Abstract
A system and a method for preventing a spoofed denial of service attack in a networked computing environment is described. A hierarchical protocol stack is defined. The hierarchical protocol stack includes a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer. A packet requesting a session with the session-oriented protocol layer is received from the networked computing environment. The request packet includes headers containing a source address of uncertain trustworthiness. The request packet is acknowledged by performing the following operations. First, a checksum is calculated from information included in the request packet headers. A request acknowledgement packet is generated. The request acknowledgement packet includes headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address. Finally, the request acknowledgement packet is sent into the networked computing environment. An acknowledgement packet is received from the networked computing environment. The acknowledgement packet includes headers containing an acknowledgement number. The acknowledgement packet is validated by performing the following operations. First, a validation checksum is calculated from information included in the acknowledgement packet headers. Then, the validation checksum is compared to the acknowledgement number of the acknowledgement packet. No state is maintained by the authenticating system until the comparison has succeeded.
Images(10)
Previous page
Next page
Claims(20)
What is claimed is:
1. A system for preventing a spoofed denial of service attack in a networked computing environment, comprising:
a hierarchical protocol stack comprising a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer and receiving a packet from the networked computing environment requesting a session with the session-oriented protocol layer, the request packet comprising headers containing a source address of uncertain trustworthiness, the hierarchical protocol stack receiving an acknowledgement packet from the networked computing environment comprising headers containing an acknowledgement number; and
an authentication module acknowledging the request packet and validating the acknowledgement packet, comprising:
a checksumming module calculating a checksum from information included in the request packet headers and calculating a validation checksum from information included in the acknowledgement packet headers;
a packet module generating request acknowledgement packet comprising headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address and sending the request acknowledgement packet into the networked computing environment; and
a comparison module comparing the validation checksum to the acknowledgement number of the acknowledgement packet.
2. A system according to claim 1, further comprising:
a firewall performing the request packet acknowledgement and the acknowledgement packet validation, the firewall functionally interposed between a server and the networked computing environment; and
the authentication module performing a session handshaking sequence between the firewall and the server.
3. A system according to claim 2, further comprising:
the authentication module forwarding a synchronize packet to the server, receiving a synchronize-acknowledgement packet from the server and forwarding an acknowledgement packet to the server.
4. A system according to claim 2, further comprising:
a translation module translating sequence numbers and acknowledgement numbers on packets transiting through the firewall to and from the server in accordance with the acknowledgement number of the acknowledgement packet.
5. A system according to claim 1, wherein the calculated checksum is a cryptographic checksum.
6. A system according to claim 5, wherein the cryptographic checksum comprises at least one cryptographic processing procedure selected from the group comprising a secure hash algorithm-1 (SHA-1) and a message authorization code (MAC).
7. A system according to claim 1, wherein the information included in the headers of the request packet comprises a source address, a destination address, a source port number, a destination port number, and a sequence number.
8. A system according to claim 1, wherein the session-oriented protocol layer comprises the Transmission Control Protocol (TCP).
9. A method for preventing a spoofed denial of service attack in a networked computing environment, comprising:
defining a hierarchical protocol stack comprising a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer;
receiving a packet from the networked computing environment requesting a session with the session-oriented protocol layer, the request packet comprising headers containing a source address of uncertain trustworthiness;
acknowledging the request packet, comprising:
calculating a checksum from information included in the request packet headers;
generating a request acknowledgement packet comprising headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address; and
sending the request acknowledgement packet into the networked computing environment;
receiving an acknowledgement packet from the networked computing environment comprising headers containing an acknowledgement number; and
validating the acknowledgement packet, comprising:
calculating a validation checksum from information included in the acknowledgement packet headers; and
comparing the validation checksum to the acknowledgement number of the acknowledgement packet.
10. A method according to claim 9, further comprising:
performing the request packet acknowledgement and the acknowledgement packet validation on a firewall functionally interposed between a server and the networked computing environment; and
performing a session handshaking sequence between the firewall and the server.
11. A method according to claim 10, further comprising:
forwarding a synchronize packet to the server;
receiving a synchronize-acknowledgement packet from the server; and
forwarding an acknowledgement packet to the server.
12. A method according to claim 10, further comprising:
translating sequence numbers and acknowledgement numbers on packets transiting through the firewall to and from the server in accordance with the acknowledgement number of the acknowledgement packet.
13. A method according to claim 9, wherein the calculated checksum is a cryptographic checksum.
14. A method according to claim 13, wherein the cryptographic checksum comprises at least one cryptographic processing procedure selected from the group comprising a secure hash algorithm-1 (SHA-1) and a message authorization code (MAC).
15. A method according to claim 9, wherein the information included in the headers of the request packet comprises a source address, a destination address, a source port number, a destination port number, and a sequence number.
16. A method according to claim 9, wherein the session-oriented protocol layer comprises the Transmission Control Protocol (TCP).
17. A storage medium for preventing a spoofed denial of service attack in a networked computing environment, comprising:
defining a hierarchical protocol stack comprising a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer;
receiving a packet from the networked computing environment requesting a session with the session-oriented protocol layer, the request packet comprising headers containing a source address of uncertain trustworthiness;
acknowledging the request packet, comprising:
calculating a checksum from information included in the request packet headers;
generating a request acknowledgement packet comprising headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address; and
sending the request acknowledgement packet into the networked computing environment;
receiving an acknowledgement packet from the networked computing environment comprising headers containing an acknowledgement number; and
validating the acknowledgement packet, comprising:
calculating a validation checksum from information included in the acknowledgement packet headers; and
comparing the validation checksum to the acknowledgement number of the acknowledgement packet.
18. A storage medium according to claim 17, further comprising:
performing the request packet acknowledgement and the acknowledgement packet validation on a firewall functionally interposed between a server and the networked computing environment; and
performing a session handshaking sequence between the firewall and the server.
19. A storage medium according to claim 18, further comprising:
forwarding a synchronize packet to the server;
receiving a synchronize-acknowledgement packet from the server; and
forwarding an acknowledgement packet to the server.
20. A storage medium according to claim 18, further comprising:
translating sequence numbers and acknowledgement numbers on packets transiting through the firewall to and from the server in accordance with the acknowledgement number of the acknowledgement packet.
Description
CROSS-REFERENCE TO RELATED APPLICATION

This patent application is related to commonly-assigned U.S. patent application Ser. No. 09/655,515, entitled “Apparatus And Method For Controlling Access To Network Resources Through Stateless Client Validity Checking,” filed Aug. 31, 2000, pending, the disclosure of which is incorporated herein by reference.

FILED OF THE INVENTION

The present invention relates in general to networked computing environment protection, and, in particular, to a system and method for preventing a spoofed denial of service attack in a networked computing environment.

BACKGROUND OF THE INVENTION

Computer networks form a central component of corporate information technology infrastructures. There are two types of networks. A local area network or “intranetwork” is a network operating within a single identifiable location, such as on one floor of a building. Individual computers and shared resources are interconnected over a single media segment. A wide area network or “internetwork” is a network consisting of interconnected intranetworks and geographically computational resources which, when taken as a whole, comprise a unified set of loosely associated computers. The Internet, for example, is a public internetwork interconnecting clients worldwide.

Structurally, most internetworks and intranetworks are based on a layered network model employing a stack of standardized protocol layers. The Transmission Control Protocol/Internet Protocol (TCP/IP) suite, such as described in W. R. Stevens, “TCP/IP Illustrated,” Vol. 1, Ch. 1 et seq., Addison-Wesley (1994), the disclosure of which is incorporated herein by reference, is a widely adopted network model. Computers and network resources using the TCP/IP suite implement hierarchical protocol stack that include link, network and transport layers. In addition, client and server end devices implement an application layer for providing or receiving services, such as electronic mail, content provision or resource sharing, to individual clients. The Transmission Control Protocol (TCP) provides a connection-oriented, reliable, byte stream service. Services offered in a TCP environment are session-based and TCP sessions must be initiated through a negotiated three-way handshaking sequence.

TCP-based networks are particularly susceptible to a type of attack known as a denial of service (“DoS”) attack. Ordinarily, a TCP server will reserve state, such as memory buffers, upon receiving a service request from a client in the expectation of having to process transient data packets during a session. However, a state consumption attack attempts to force a victim server to allocate state for unproductive uses during the three-way handshaking sequence. In a DoS attack, an attacker will cause a high volume of bogus service requests to be sent to a victim server which will continue to allocate state until all available state is expended. Thus, no state will be left for valid requesters and service will be denied. In addition, DoS attacks are difficult to detect because the bogus service requests are indistinguishable from normal network traffic.

One form of DoS attack employs “spoofed” packet source addresses. A spoofed packet is a data packet sent by a third party containing a source address other than the source address of that third party. The fraudulent source address could be the address of another system or might be a random source address that is valid yet not presently in use. Unfortunately, TCP does not provide means for ensuring that packet source addresses are not fraudulent. Attackers take advantage of this security hole by sending service request packets with fraudulent source addresses to disguise their identity. Consequently, tracing the source of spoofed DoS attacks is often meaningless and the attackers are virtually untraceable.

In the prior art, firewalls have traditionally provided a first line of defense against all types of attacks. Firewalls are placed at the boundary separating an intranetwork from a public internetwork and prevent network compromise by unauthorized users through packet filtering and proxies. However, firewalls fail to provide an adequate defense to DoS attacks for several reasons.

For instance, most firewalls filter packets by comparing the source addresses of incoming packets to lists of individual addresses and address ranges. However, these addresses and address ranges must be periodically loaded into the firewall. Loading this information once a DoS attack is underway is too late to be of practical use. Similarly, address range checking can be too restrictive and can filter out valid yet not presently connected addresses.

More importantly, though, most, if not all, of the packets used to produce a DoS attack will appear valid, as there is no a priori method to sort spoofed packets from non-spoofed packets. As well, application layer firewalls, such as might be incorporated directly into a server, risk running out of state at the TCP and IP protocol layers in the same manner as the underlying servers which they are attempting to protect. Finally, firewalls are typically installed within the infrastructure of an organization in front of the internal machine population, thereby providing no further protection beyond the protected machine boundary.

Therefore, there is a need for a solution providing protection against DoS attacks in a TCP-based computing environment. Preferably, such an approach would operate in a stateless fashion to protect the firewall from allocating state and thus ensure that the firewall would not run out of resources during a DoS. Moreover, such an approach would operate at a network layer in a protocol-independent manner.

There is a further need for a dynamic approach to packet validity checking which can detect spoofed, fictitious, and inactive addresses. Preferably, such an approach would validate all source addresses in an unintrusive manner.

SUMMARY OF THE INVENTION

The present invention provides a system and method for protecting against state consumption-type DoS attacks. Typically, DoS attacks can occur in TCP-based networked computing environments. A system, such as could be incorporated into a firewall, intercepts session request packets. A checksum, preferably cryptographic, is generated from information contained in the headers of each request packet. An request acknowledgement packet is sent to the source address indicated in the request packet headers with the checksum included as a pseudo sequence number. If an acknowledgement packet is received back, a second checksum is generated from information contained in the headers of the acknowledgement packet. The second checksum is compared to the acknowledgement number contained in the acknowledgement packet headers. If the checksum and acknowledgement number match, the session request is valid and a handshaking sequence is performed with the server. The sequence numbers of subsequent session packets are translated to account for the pseudo acknowledgement number.

An embodiment of the present invention is a system and a method for preventing a spoofed denial of service attack in a networked computing environment. A hierarchical protocol stack is defined. The hierarchical protocol stack includes a plurality of communicatively interfaced protocol layers with at least one session-oriented protocol layer. A packet requesting a session with the session-oriented protocol layer is received from the networked computing environment. The request packet includes headers containing a source address of uncertain trustworthiness. The request packet is acknowledged by performing the following operations. First, a checksum is calculated from information included in the request packet headers. A request acknowledgement packet is generated. The request acknowledgement packet includes headers containing the checksum as a pseudo sequence number and the source address in the request packet headers as a destination address. Finally, the request acknowledgement packet is sent into the networked computing environment. An acknowledgement packet is received from the networked computing environment. The acknowledgement packet includes headers containing an acknowledgement number. The acknowledgement packet is validated by performing the following operations. First, a validation checksum is calculated from information included in the acknowledgement packet headers. Then, the validation checksum is compared to the acknowledgement number. No state is maintained by the authenticating system until the comparison has succeeded.

Still other embodiments of the present invention will become readily apparent to those skilled in the art from the following detailed description, wherein is described embodiments of the invention by way of illustrating the best mode contemplated for carrying out the invention. As will be realized, the invention is capable of other and different embodiments and its several details are capable of modifications in various obvious respects, all without departing from the spirit and the scope of the present invention. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a networked computing environment, including a system for preventing a spoofed denial of service attack, in accordance with the present invention.

FIG. 2 is a network diagram illustrating, by way of example, the progression of a state consumption attack.

FIG. 3 is a timing diagram showing, by way of example, the three-way handshake performed during the initiation of a TCP session.

FIG. 4 is a block diagram showing an authentication system for preventing a spoofed denial of service attack for use in a networked computing environment.

FIG. 5 is a timing diagram showing, by way of example, the authentication of an incoming TCP packet.

FIG. 6 is a block diagram showing the functional software modules of the system of FIG. 4.

FIG. 7 is a flow diagram showing a method for preventing a spoofed denial of service attack in a networked computing environment in accordance with the present invention.

FIG. 8 is a flow diagram showing a routine for processing SYN packets for use in the method of FIG. 7.

FIG. 9 is a flow diagram showing a routine for processing ACK packets for use in the method of FIG. 7.

FIG. 10 is a flow diagram showing a routine for processing session packets for use in the method of FIG. 7.

DETAILED DESCRIPTION

FIG. 1 is a block diagram showing a networked computing environment 10, including a system, incorporated into a firewall 20 (FW), for preventing a spoofed denial of service attack, in accordance with the present invention. The environment 10 includes a intranetwork 12 interconnected with an internetwork 15, such as the Internet, by means of an Internetwork Service Provider (ISP) infrastructure 14. The intranetwork 12 includes a local server 11 (S) with a plurality of workstations (WS) 13 and similar network resources. Internally, the ISP infrastructure 14 includes a plurality of network service support machines, include high bandwidth routers, servers, and related support equipment, as is known in the art.

The intranetwork 12 interfaces to the internetwork 15 through a series of high- and low-bandwidth connections. A high-bandwidth connection 17, such as an optical carrier OC-3 (155.52 Mbps) or OC-12 (622.08 Mbps) line, connects the intranetwork 12 to a pair of routers 18 which exchange data over low-bandwidth commercial lines. The intranetwork 12 interfaces to the high-bandwidth connection 17 through a border router 16 (BR) or similar device. Similarly, the ISP infrastructure 14 interfaces to the router 18 over a high-bandwidth connection 17 through a border router 19.

The server 11 is susceptible to denial of service (DoS) attacks, particularly state consumption attacks, as further described below with respect to FIG. 2. As protection against DoS attacks, firewall 20 can be placed at between the border router 16 and the intranetwork 12. The firewall 20 incorporates a system for preventing a spoofed denial of service attack, as further described below beginning with reference to FIG. 4.

The individual computer systems 11 and 13 are general purpose, programmed digital computing devices consisting of a central processing unit (CPU), random access memory (RAM), non-volatile secondary storage, such as a hard drive or CD ROM drive, network interfaces, and peripheral devices, including user interfacing means, such as a keyboard and display. Program code, including software programs, and data are loaded into the RAM for execution and processing by the CPU and results are generated for display, output, transmittal, or storage.

State consumption attacks are a specific type of DoS attack that can cripple or disable network servers 11 through bogus session requests. FIG. 2 is a network diagram 30 illustrating, by way of example, the progression of a state consumption attack. The goal of a state consumption attack is to induce a victim server 36 into allocating state, such as memory buffers and similar limited resources, through incomplete service requests. State consumption attacks occur in TCP/IP compliant computing environments with connection-oriented protocols, such as TCP.

Although several forms of state consumption attacks exist, the SYN attack is the most notorious. This type of attack relies upon inherent limitations in the TCP protocol. Ordinarily, when opening a new connection, a server 36 performs a three-way handshake sequence with a requesting client. The three-way handshake is further described in W. R. Stevens, “TCP/IP Illustrated,” Vol. 1, Ch. 18, Addison-Wesley (1994), the disclosure of which is incorporated herein by reference. Briefly, the handshake begins when a requesting client sends a synchronize (SYN) request to a server with which the client wishes to establish a connection. The server allocates state upon receipt of the SYN request and sends a SYN/acknowledgement (ACK) response back to the requesting client. The client then sends back an ACK to confirm and establish the connection.

A state consumption attack progresses as follows. A plurality of individual intranetworks 33, 35 are interconnected via an internetwork 33 using conventional low- and high-bandwidth carriers interfaced via border routers 32 or similar devices. Other network topologies and configurations are feasible. An attacker 31 sends a stream of SYN request packets 37 with a fraudulent, that is, “spoofed,” source address to a victim server 36 (step {circle around (1)}). The attacker 31 might also induce a plurality of servers 34 to send fraudulent SYN request packets 37 (step {circle around (2)}), such as through broadcast messaging. In turn, the victim server 36 allocates state (step {circle around (3)}) and sends SYN/ACK response packets to the system indicated in the source address of each SYN request packet 37. However, since the source addresses are spoofed, no ACK packets are returned and the state on the victim server 36 remains allocated until each request times out. If a sufficient number of SYN request packets 37 are sent in rapid succession, all available state in the victim server 36 will be allocated in reliance on the fraudulent SYN request packets 37. Thus, no state will be available for valid requests and the service will be denied.

Unfortunately, the SYN request packets 37 used to attack victim servers 36 resemble valid, bona fide traffic. Consequently, DoS attacks are difficult to detect. Moreover, state consumption attacks can also target ISP infrastructures 14 (shown in FIG. 1) resulting in a wider impact.

FIG. 3 is a timing diagram showing, by way of example, the three-way handshake 40 performed during the initiation of a TCP session. TCP is a connection-oriented, session-based protocol. Individual TCP sessions are end-to-end communications with a three-way handshake sequence 40 executed between peer TCP layers. Ordinarily, the handshake sequence 40 is transacted between a requesting client 41 and a responding server 45. Upon successful completion of the handshake sequence, session-based programs in the application layers of the client 41 and server 45 communicate by exchanging TCP packets.

The TCP three-way handshake consists of three exchanges, such as described in W. R. Stevens, “TCP/IP Illustrated,” Vol. 1, Chs. 17-18, Addison-Wesley (1994), and Postel, J. B., “Transmission Control Protocol,” RFC 793 (September 1981), the disclosures of which are incorporated herein by reference. First, an initiating client sends a synchronize (SYN) packet 42 to a server. The SYN packet 42 has an internet protocol (IP) header containing fields for storing a source address and destination address and a TCP header containing fields for storing a source port number, destination port number, and sequence number n. The sequence number n is a 32-bit unsigned integer chosen by the initiating client.

The server responds by sending a SYN-acknowledgement (ACK) packet 43 addressed to the system located at the source address. The SYN-ACK packet 43 also has an IP header containing fields for storing a source address and destination address and a TCP header containing fields for storing a source port number, destination port number, sequence number n, plus an acknowledgement number m. The acknowledgement number m is also a 32-bit unsigned integer. The client's sequence number n is incremented by one to indicate acknowledgement by the server and is sent back to the system located at the source address in the acknowledgement number m field. In addition, the server includes its own sequence number in the sequence number n field.

Assuming the source address is valid, the initiating client returns an ACK packet 44 with the TCP header containing the sequence number n chosen by the server incremented by one in the acknowledgement field to indicate acknowledgement by the client. Thus, upon successful completion of the three-way handshake, the sequence number n field of the ACK packet 44 contains the client sequence number plus one and the acknowledgement number m field contains the server sequence number plus one.

FIG. 4 is a block diagram showing an authentication system 50, incorporated into firewall 52, for preventing a spoofed DoS attack for use in a networked computing environment 49. For the purpose of illustrating how to prevent spoofed DoS attacks, the networked computing environment 50 consists of three types of systems: a requesting client 51, a firewall 52, and a responding server 53. Each of these systems implement a TCP/IP network protocol stack which includes link 54, IP 55 and TCP 56, layers, such as described in W. R. Stevens, “TCP/IP Illustrated,” Vol. 1, Ch. 1 et seq., Addison-Wesley (1994), the disclosure of which is incorporated herein by reference. In addition, both the client 51 and server 52 implement client application 57 and server application 58 layers. In the case of a DoS attack on the server 53, the client application 57 is actually a malicious application that attempts to consume the state in the TCP layer 56 of the server 53.

In the described embodiment, a firewall 52 intercedes between the client 51 and the server 53 and performs the handshake sequence to prevent DoS attacks, as further described below with reference to FIG. 5. The firewall 52 functions as a pseudo server by exchanging client-firewall packets 59 with the client 51 and firewall-server packets 60 with the server 53. A client-firewall handshake sequence is first attempted and, if authenticated, a firewall-server handshake sequence is then performed.

In the described embodiment, the authentication system 50 is incorporated in a firewall 52 operating on a programmed digital computer. As is conventional in the art, the firewall 52 operates under the control of an operating system, such as the Unix or Windows NT operating systems. An exemplary firewall 52 is the Gauntlet firewall product, version 5.5, licensed by Network Associates, Inc., Santa Clara, Calif. Alternatively, the authentication system 50 could be incorporated directly into a network protocol stack, such as a TCP/IP stack running on either a firewall, server, or client. As well, the authentication system 50 could be implemented as a stand-alone program or as a program module working in conjunction with an operating system, protocol stack, or other application, procedure, driver, or module. Finally, the authentication system 50 could be implemented entirely or partly in hardware, firmware, or software, as would be recognized by one skilled in the art. In particular, the translation module, discussed below with reference to FIG. 6, could be efficiently implemented in hardware to optimize the firewall-server sequence number conversions.

FIG. 5 is a timing diagram 70 showing, by way of example, the authentication of an incoming TCP packet. Briefly, a firewall 72 intercepts a session request from a requesting client 71 and only forwards the session request to a server 73 after checking the existence or validity of the requesting client 71. Non-existent or invalid session requests are discarded, thereby preventing state consumption leading up to a DoS attack.

Chronologically, a requesting “client” 71 sends a client SYN packet 74 to the server 73. However, the firewall 72 intercepts the client SYN packet 74 and generates a firewall SYN-ACK packet 75. No state on the firewall 72 is consumed. The firewall SYN-ACK packet 75 contains a pseudo acknowledgement number, preferably generated by taking the lowest 32-bits of a cryptographic checksum of the source address, destination address, source port number, destination port number, and sequence number contained in the IP header and the TCP header of the client SYN packet 74.

The firewall SYN-ACK packet 75 is addressed to the system at the source address specified in the TCP header of the client SYN packet 74. If a valid requesting client 71 sent the client SYN packet 74, the requesting client 71 will respond to the firewall SYN-ACK packet 75 by sending a client ACK packet 76.

However, if the client SYN packet 74 was spoofed, that is, sent with a fraudulent source address, two outcomes exist. First, if the spoofed source address is not in use by another system, no responding client ACK packet 76 will be generated and the original client SYN packet 74 will be ignored. Alternatively, if the spoofed source address is in use by another system but that system did not sent the original client SYN packet 74, that system will send a client reset (RST) packet. In either case, since the firewall 72 intercepted the spoofed client SYN packet 74 before reaching the server 73, no state is consumed or wasted, both on the server 73 and on the firewall 72.

Assuming the client SYN packet 74 originated from a valid requesting client 71, the firewall 72 will perform a three-way handshake with the server 73. First, the firewall 72 sends a firewall SYN packet 77 to the server 73 upon receiving back a client ACK packet 76. The firewall SYN packet 77 contains the original sequence number contained in the TCP header of the client SYN packet 74. In response, the server 73 returns a server SYN-ACK packet 78 containing an acknowledgement number. Finally, the firewall 72 completes the three-way handshake by responding with a firewall ACK packet 79. Note that the sequence numbers contained in all subsequent packets exchanged between the requesting client 71 and the server 73 will need to be translated to account for the offset of the firewall-generated pseudo sequence number.

FIG. 6 is a block diagram showing the functional software modules of the system 50 of FIG. 4. The authentication system 50 is incorporated into the firewall 52 and works in conjunction with a protocol stack 91. However, the authentication 50 could equally work as a stand-alone system or directly in conjunction with a server.

The system 50 consists of four main modules: packet 92, checksum 93, comparison 94, and translation 95. Each module is a computer program or module written as source code in a conventional programming language, such as the C programming language, and is presented for execution by the CPU as object or byte code, as is known in the art. The various implementations of the source code and object and byte codes can be held on a computer-readable storage medium or embodied on a transmission medium in a carrier wave. The system 50 operates in accordance with a sequence of process steps, as further described below beginning with reference to FIG. 7.

The packet module 92 performs packet housekeeping chores, including interfacing to the protocol stack 91, parsing header information for incoming packets and building headers for outgoing packets. The checksum module 93 generates a checksum, preferably cryptographic, based on information contained in the IP and TCP headers of incoming SYN packets. The comparison module 94 determines whether received ACK packets are valid or forged based on checksum information. Finally, the translation module 95 converts sequence numbers of non-SYN and non-ACK packets, that is, session packets, to adjust for the offset of the firewall pseudo acknowledgement number.

FIG. 7 is a flow diagram showing a method 100 for preventing a spoofed denial of service attack in a networked computing environment in accordance with the present invention. The method 100 operates in two phases. During the first phase, initialization (block 101), the protocol stack 91 (shown in FIG. 6) and authentication system 50 (shown in FIG. 4) are started and their associated data structures initialized.

During the second phase, operation (blocks 102-108), packets are iteratively processed as follows. If the incoming packet is a client SYN packet 74 (shown in FIG. 5) (block 103), a routine for processing SYN packets is called I(block 104), as further described below with reference to FIG. 8. If the incoming packet is a client ACK packet 76 (block 105), a routine for processing ACK packets is called (block 106), as farther described below with reference to FIG. 9. Otherwise, if the incoming packet is neither a client SYN packet nor a client ACK packet, a routine for processing session packets is called (block 107), as further described below with reference to FIG. 10. The iterative loop (blocks 102-108) is repeated for each new packet. The method 100 terminates upon the unloading of the authentication system 50 or upon shutdown of the firewall 52.

FIG. 8 is a flow diagram showing a routine 120 for processing SYN packets 74 (shown in FIG. 5) for use in the method of FIG. 7. The purpose of this routine is to intercept TCP session request packets and generate a nearly non-forgeable response. Each client SYN packet 74 is received (block 121) before the session request is forwarded to the server. The sequence number contained in the TCP header of the client SYN packet 74 is incremented by one (block 122). A checksum, preferably cryptographic, is generated (block 123). In the described embodiment, the checksum is calculated using a secure cryptographic processing procedure, such as the Secure Hash Algorithm-1 (SHA-1), described in the Federal Information Processing Standards Publication, No. 180-1, or Message Authentication Code (MAC), described in the Federal Information Processing Standards Publication, No. 113. The cryptographic checksum is based on the source address (IP header), destination address (IP header), source port number (TCP header), destination port number (TCP header), and sequence number (TCP header), although more or fewer header fields, as well as other quantitative information, could be used in generating the checksum.

Next, a firewall sequence number is generated using the checksum (block 124). In the described embodiment, the lowest 32-bits of the cryptographic checksum are used. Finally, a firewall SYN-ACK packet 75 is sent addressed to the system located at the source address in the client SYN packet 74, after which the routine returns. Note that the foregoing routine 120 is alone sufficient to protect against a state consumption attack, as the server is sheltered from spoofed request packets and only client SYN packets 74 with valid source addresses are ultimately acknowledged.

FIG. 9 is a flow diagram showing a routine 140 for processing ACK packets 76 (shown in FIG. 5) for use in the method of FIG. 7. The purpose of this routine is to authenticate an acknowledgement reply from a possible client and, if authenticated, to complete a three-way handshake with the requested server. Each client ACK packet 76 is received (block 141) and a checksum, preferably cryptographic, is generated (block 142) using the same technique and information as used to generate the pseudo sequence number contained in the firewall SYN-ACK packet 75. The calculated checksum is compared to the acknowledgement number (minus one) contained in the client ACK packet 76 (block 143). If the checksum and acknowledgement number do not match (block 143), the client ACK packet 76 is discarded and the routine returns. Otherwise, the client is valid and a three-way handshake (blocks 144-147) is performed with the server.

First, a firewall SYN packet 77 is sent to the server using the sequence number contained in the TCP header of the original client SYN packet 74 (block 144). A server SYN-ACK packet 78 is received from the server (block 145) and the acknowledgement number contained in the TCP header is stored (block 146). The acknowledgement number needs to be stored to determine the offset between the pseudo acknowledgement number generated by the firewall 72 and exchanged with the receiving client 71 and the genuine acknowledgement number exchanged with the server 73. Finally, a firewall ACK packet 79 with the client and server sequence numbers is sent to the server (block 147) to complete the three-way handshake. The routine then returns.

FIG. 10 is a flow diagram showing a routine 160 for processing session packets for use in the method of FIG. 7. The purpose of this routine is to translate the sequence numbers contained in the TCP header of session packets transiting through the firewall 72. Thus, if the session packet is an incoming packet from an outside client, that is, originating from an authenticated client located outside the firewall 72, (block 161), the acknowledgement number in the TCP header is translated from a firewall acknowledgement number (which could also be called a “pseudo server acknowledgement number”) to a server acknowledgement number (block 162). Sequence numbers and acknowledgement numbers are continuously incremented by the amount of data transferred, measured in bytes, as the TCP session progresses. By translating the firewall acknowledgement number into the server acknowledgement number, the firewall acknowledgement number plus any number i is translated into the server acknowledgement number plus i. The session packet is then forwarded to the server (block 163). Otherwise, if the session packet is an outgoing packet, that is, originating from the server (block 161), the sequence number in the TCP header is translated from a server sequence number to a firewall sequence number (block 164). By translating the server sequence number into the firewall sequence number, the server sequence number plus any number j is translated into the firewall sequence number plus j. The session packet is then forwarded to the server (block 165). The routine then returns.

Note that TCP sessions are terminated by a four-way handshake using finish (FIN) packets. These packets are processed by the routine 160 is the usual fashion and no special handling is necessary.

Thus, the present invention provides an approach to protecting a server from DoS attacks by intercepting and authenticating session requests. Only authenticated session requests are allowed to proceed to create a session between the server and a requesting client.

This approach functions in a TCP-based environment, as well as in any other session-oriented environment, in which a three-way handshake is performed. In addition, this approach functions entirely with TCP and does not rely on acknowledgement packets sent in other protocols. For instance, “ping” packets could be used with the Internet Control Message Protocol (ICMP).

However, ICMP headers do not include ports and, consequently, ICMP-based solutions will not work in those networked computing environments using dynamic network address translation.

While the invention has been particularly shown and described as referenced to the embodiments thereof, those skilled in the art will understand that the foregoing and other changes in form and detail may be made therein without departing from the spirit and scope of the invention.

Patent Citations
Cited PatentFiling datePublication dateApplicantTitle
US4641274 *Aug 19, 1985Feb 3, 1987International Business Machines CorporationMethod for communicating changes made to text form a text processor to a remote host
US5070528 *Jun 29, 1990Dec 3, 1991Digital Equipment CorporationGeneric encryption technique for communication networks
US5325370 *Nov 12, 1991Jun 28, 1994Storage Technology CorporationMethod and apparatus for recording data on magnetic tape media
US5511122 *Jun 3, 1994Apr 23, 1996The United States Of America As Represented By The Secretary Of The NavyIntermediate network authentication
US5941998 *Jul 25, 1997Aug 24, 1999Samsung Electronics Co., Ltd.Disk drive incorporating read-verify after write method
US5950195 *Sep 18, 1996Sep 7, 1999Secure Computing CorporationGeneralized security policy management system and method
US6070243 *Jun 13, 1997May 30, 2000Xylan CorporationDeterministic user authentication service for communication network
US6092191 *Nov 29, 1996Jul 18, 2000Kabushiki Kaisha ToshibaPacket authentication and packet encryption/decryption scheme for security gateway
US6144744 *Jun 30, 1997Nov 7, 2000International Business Machines CorporationMethod and apparatus for the secure transfer of objects between cryptographic processors
Non-Patent Citations
Reference
1 *Bruce Schneier, 1996, Katherine Schowalter, "Applied Cryptography", 445,455.*
2 *S. Bellovin, 1996, AT&T Research, "Defending Against Sequence Number Attacks".*
3 *W. Richard Stevens, 1994, Addison Wesley, "TCP/IP Illustrated, vol. 1", 34, 225.
Referenced by
Citing PatentFiling datePublication dateApplicantTitle
US6944663 *Mar 6, 2002Sep 13, 2005Sun Microsystems, Inc.Method and apparatus for using client puzzles to protect against denial-of-service attacks
US7020784 *Aug 20, 2001Mar 28, 2006Yitran Communications Ltd.Mechanism for detecting intrusion and jamming attempts in a shared media based communications network
US7039954 *May 4, 2001May 2, 2006International Business Machines CorporationMethod for enabling a network-addressable device to detect use of its identity by a spoofer
US7047303 *Jul 26, 2001May 16, 2006International Business Machines CorporationApparatus and method for using a network processor to guard against a “denial-of-service” attack on a server or server cluster
US7058058 *Nov 4, 2004Jun 6, 2006Juniper Networks, Inc.Transparent optimization for transmission control protocol initial session establishment
US7069438 *Aug 19, 2002Jun 27, 2006Sowl Associates, Inc.Establishing authenticated network connections
US7114182 *May 31, 2002Sep 26, 2006Alcatel Canada Inc.Statistical methods for detecting TCP SYN flood attacks
US7120858Aug 21, 2002Oct 10, 2006Sun Microsystems, Inc.Method and device for off-loading message digest calculations
US7158522Mar 7, 2006Jan 2, 2007Juniper Networks, Inc.Transparent optimization for session establishment using characterized synchronization packet
US7177300 *May 24, 2002Feb 13, 2007Communications Research Laboratory, Independent Administrative InstitutionPacket communication method and proposal node
US7254133 *Jul 15, 2002Aug 7, 2007Intel CorporationPrevention of denial of service attacks
US7302705 *Aug 30, 2000Nov 27, 2007International Business Machines CorporationMethod and apparatus for tracing a denial-of-service attack back to its source
US7360245 *Jul 18, 2001Apr 15, 2008Novell, Inc.Method and system for filtering spoofed packets in a network
US7441272Jun 9, 2004Oct 21, 2008Intel CorporationTechniques for self-isolation of networked devices
US7454614 *Mar 25, 2002Nov 18, 2008Microsoft CorporationMethod and apparatus for fault tolerant TCP handshaking
US7464410 *Dec 11, 2001Dec 9, 2008At&T Corp.Protection against flooding of a server
US7529246Jan 2, 2007May 5, 2009Juniper Networks, Inc.Transparent optimization for session establishment using characterized synchronization packet
US7543070 *Sep 22, 2005Jun 2, 2009Mcafee, Inc.System and method for negotiating multi-path connections through boundary controllers in a networked computing environment
US7549159May 5, 2005Jun 16, 2009Liquidware Labs, Inc.System, apparatuses, methods and computer-readable media for determining the security status of a computer before establishing connection thereto
US7552323 *Aug 19, 2003Jun 23, 2009Liquidware Labs, Inc.System, apparatuses, methods, and computer-readable media using identification data in packet communications
US7564792Nov 4, 2004Jul 21, 2009Juniper Networks, Inc.Transparent optimization for transmission control protocol flow control
US7591001May 5, 2005Sep 15, 2009Liquidware Labs, Inc.System, apparatuses, methods and computer-readable media for determining the security status of a computer before establishing a network connection
US7610622 *Feb 6, 2006Oct 27, 2009Cisco Technology, Inc.Supporting options in a communication session using a TCP cookie
US7660980Mar 23, 2007Feb 9, 2010Liquidware Labs, Inc.Establishing secure TCP/IP communications using embedded IDs
US7673025Apr 29, 2004Mar 2, 2010Cisco Technology, Inc.Controlling access message flow
US7675854 *Feb 21, 2006Mar 9, 2010A10 Networks, Inc.System and method for an adaptive TCP SYN cookie with time validation
US7711790 *Sep 20, 2000May 4, 2010Foundry Networks, Inc.Securing an accessible computer system
US7725587Jun 29, 2001May 25, 2010Aol LlcDeep packet scan hacker identification
US7730137Dec 22, 2003Jun 1, 2010Aol Inc.Restricting the volume of outbound electronic messages originated by a single entity
US7743134 *Aug 16, 2001Jun 22, 2010Riverbed Technology, Inc.Thwarting source address spoofing-based denial of service attacks
US7743144Nov 3, 2003Jun 22, 2010Foundry Networks, Inc.Securing an access provider
US7788329Jan 12, 2006Aug 31, 2010Aol Inc.Throttling electronic communications from one or more senders
US7797738Dec 14, 2005Sep 14, 2010At&T Corp.System and method for avoiding and mitigating a DDoS attack
US7797749Nov 3, 2004Sep 14, 2010Intel CorporationDefending against worm or virus attacks on networks
US7823194Aug 13, 2003Oct 26, 2010Liquidware Labs, Inc.System and methods for identification and tracking of user and/or source initiating communication in a computer network
US7836494 *Feb 17, 2004Nov 16, 2010Intel CorporationSystem and method for regulating the flow of information to or from an application
US7865954Aug 24, 2007Jan 4, 2011Louisiana Tech Research Foundation; A Division Of Louisiana Tech University Foundation, Inc.Method to detect SYN flood attack
US7870608Nov 23, 2004Jan 11, 2011Markmonitor, Inc.Early detection and monitoring of online fraud
US7913302Nov 23, 2004Mar 22, 2011Markmonitor, Inc.Advanced responses to online fraud
US7930740 *Jul 7, 2005Apr 19, 2011International Business Machines CorporationSystem and method for detection and mitigation of distributed denial of service attacks
US7937586Jun 29, 2007May 3, 2011Microsoft CorporationDefending against denial of service attacks
US7940665Jul 15, 2009May 10, 2011Juniper Networks, Inc.Transparent optimization for transmission control protocol flow control
US7992204Nov 23, 2004Aug 2, 2011Markmonitor, Inc.Enhanced responses to online fraud
US8001244Apr 12, 2010Aug 16, 2011Aol Inc.Deep packet scan hacker identification
US8001601Jun 14, 2006Aug 16, 2011At&T Intellectual Property Ii, L.P.Method and apparatus for large-scale automated distributed denial of service attack detection
US8006285 *Jun 13, 2005Aug 23, 2011Oracle America, Inc.Dynamic defense of network attacks
US8041769Nov 23, 2004Oct 18, 2011Markmonitor Inc.Generating phish messages
US8108531May 7, 2010Jan 31, 2012Foundry Networks, Inc.Securing an access provider
US8112547 *Jun 8, 2010Feb 7, 2012International Business Machines CorporationEfficiently hashing packet keys into a firewall connection table
US8154987Jun 9, 2004Apr 10, 2012Intel CorporationSelf-isolating and self-healing networked devices
US8175096 *Jan 25, 2007May 8, 2012Hitachi, Ltd.Device for protection against illegal communications and network system thereof
US8225399Dec 14, 2005Jul 17, 2012At&T Intellectual Property Ii, LpSystem and method for avoiding and mitigating a DDoS attack
US8245298 *Aug 16, 2006Aug 14, 2012International Business Machines CorporationPort scanning method and device, port scanning detection method and device, port scanning system, computer program and computer program product
US8276204 *Dec 9, 2009Sep 25, 2012Fujitsu LimitedRelay device and relay method
US8307419 *Nov 12, 2010Nov 6, 2012Intel CorporationSystem and method for regulating communications to or from an application
US8537794 *Feb 13, 2008Sep 17, 2013Nec CorporationPseudo-response frame communication system, pseudo-response frame communication method, and pseudo-response frame transmitting device
US8584199Dec 15, 2012Nov 12, 2013A10 Networks, Inc.System and method to apply a packet routing policy to an application session
US8595791Oct 12, 2012Nov 26, 2013A10 Networks, Inc.System and method to apply network traffic policy to an application session
US8645537Jul 29, 2011Feb 4, 2014Citrix Systems, Inc.Deep packet scan hacker identification
US8719926 *Feb 11, 2011May 6, 2014Verizon Patent And Licensing Inc.Denial of service detection and prevention using dialog level filtering
US8769671May 2, 2004Jul 1, 2014Markmonitor Inc.Online fraud solution
US8782221Jul 5, 2012Jul 15, 2014A10 Networks, Inc.Method to allocate buffer for TCP proxy session based on dynamic network conditions
US20070044155 *Aug 16, 2006Feb 22, 2007International Business Machines CorporationPort scanning method and device, port scanning detection method and device, port scanning system, computer program and computer program product
US20070201474 *Jan 25, 2007Aug 30, 2007Hitachi, Ltd.Device for protection against illegal communications and network system thereof
US20100020780 *Feb 13, 2008Jan 28, 2010Yumi HiranoPseudo-response frame communication system, pseudo-response frame communication method, and pseudo-response frame transmitting device
US20100088764 *Dec 9, 2009Apr 8, 2010Fujitsu LimitedRelay device and relay method
US20110119751 *Nov 12, 2010May 19, 2011Intel CorporationSystem and method for regulating communications to or from an application
US20120210007 *Feb 11, 2011Aug 16, 2012Verizon Patent And Licensing Inc.Denial of service detection and prevention using dialog level filtering
US20130055383 *Aug 22, 2011Feb 28, 2013Cisco Technology, Inc.Coordinated detection of a grey-hole attack in a communication network
US20130139252 *Nov 28, 2011May 30, 2013International Business Machines CorporationSecuring network communications from blind attacks with checksum comparisons
US20130167234 *Aug 16, 2011Jun 27, 2013Siemens AktiengesellschaftMethod for Processing Messages in a Communication Network Comprising a Plurality of Network Nodes
USRE44701 *Mar 6, 2012Jan 14, 2014A10 Networks, Inc.System and method for an adaptive TCP SYN cookie with time validation
CN101030977BFeb 16, 2007Nov 6, 2013株式会社日立制作所Device for protection against illegal communications and network system thereof
EP2739002A1 *Nov 21, 2013Jun 4, 2014Verisign, Inc.Systems and methods for transparently monitoring network traffic for denial of service attacks
Classifications
U.S. Classification713/153, 726/23, 726/25, 726/22
International ClassificationH04L29/06
Cooperative ClassificationH04L69/163, H04L69/16, H04L69/161, H04L63/126, H04L63/1458, H04L63/1416
European ClassificationH04L29/06J7, H04L29/06J3, H04L63/12B, H04L63/14A1, H04L63/14D2, H04L29/06J
Legal Events
DateCodeEventDescription
Apr 3, 2012SULPSurcharge for late payment
Year of fee payment: 7
Apr 3, 2012FPAYFee payment
Year of fee payment: 8
Mar 19, 2012REMIMaintenance fee reminder mailed
Jan 7, 2008FPAYFee payment
Year of fee payment: 4
Aug 16, 2005ASAssignment
Owner name: MCAFEE, INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NETWORKS ASSOCIATES, INC., D/B/A NETWORK ASSOCIATES, INC.;REEL/FRAME:016893/0354
Effective date: 20050628
Jun 29, 2005ASAssignment
Owner name: MCAFEE, INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:NETWORKS ASSOCIATES, INC., D/B/A NETWORK ASSOCIATES, INC.;REEL/FRAME:016448/0014
Effective date: 20050628
Sep 5, 2000ASAssignment
Owner name: NETWORKS ASSOCIATES, INC., CALIFORNIA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GLAWITSCH, GREGOR A.;REEL/FRAME:011086/0975
Effective date: 20000831