CA2305587A1 - System and method for processing data packets - Google Patents
System and method for processing data packets Download PDFInfo
- Publication number
- CA2305587A1 CA2305587A1 CA002305587A CA2305587A CA2305587A1 CA 2305587 A1 CA2305587 A1 CA 2305587A1 CA 002305587 A CA002305587 A CA 002305587A CA 2305587 A CA2305587 A CA 2305587A CA 2305587 A1 CA2305587 A1 CA 2305587A1
- Authority
- CA
- Canada
- Prior art keywords
- header
- data packet
- processing
- registers
- header portion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/25—Routing or path finding in a switch fabric
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/30—Peripheral units, e.g. input or output ports
- H04L49/3009—Header conversion, routing tables or routing tags
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/30—Peripheral units, e.g. input or output ports
- H04L49/3027—Output queuing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L49/00—Packet switching elements
- H04L49/35—Switches specially adapted for specific applications
- H04L49/351—Switches specially adapted for specific applications for local area network [LAN], e.g. Ethernet switches
Abstract
An apparatus and method for processing a data packet to determine the routing of the data packet through a communications network is provided in which the data packet has a header portion stored in I/O header registers (150) and a data portion stored in data buffer (50). The header portion of an incoming data packet is received, stored and processed. The processing may include using a processing core such as ALU (164) for executing instructions for processing the header portion, searching through a route table (162) to determine a route of the data packet, and searching through a table memory (124) for information about the destination of the data packet. The route table search, the table memory search and the processing core operate simultaneously to process the header portion and generate an internal header or a network media header. A modified header portion is then generated to route the data packet through the communications network.
Description
This invention relates generally to a system and method for routing data packets in a data communications network, and in particular to a system and method for processing header portions of data packets in order to route the data packets through the data communications network.
The amount of Internet Protocol (IP) data packet traffic being communicated over the Internet, as well as Intranets, has grown exponentially over the past decade.
Communications networks are constantly being expanded to handle this growth.
The expansion of the communications networks is typically accomplished by partitioning a network that is too large into a number of smaller communication networks in which each smaller network is serviced by a separate router, and then relinking the smaller partitioned networks back together with inter-networking equipment. However, even with this cycle of partitioning and relinking of communication networks, in some environments, such as large-scale public and private data backbones, the data packet traffic has reached such tremendous amounts that these conventional routers cannot handle the data packet traffic and proposed enhancements to these conventional routers cannot keep pace with the ever increasing amount of data packet traffic.
Normally, a conventional router is used in a communications network to route data packets through the network from a first location, known as a source, to a second SUBSTITUTE SHEET (RULE 26) location, known as a destination. These conventional routers typically have a single shared data bus so that each data packet received by the router must wait for access to the shared bus before being forwarded to the next network "hop" along the data path to the destination address of the data packet. A portion of a route table for a router, which is used to determine the next network hop for a data packet, is typically stored in a cache memory. The portion of the route table stored in the cache memory is typically the most frequently used next network addresses. Using the route table data stored in the cache, the muter attempts to determine the next network address for a particular data packet. If the next network address of the data packets is not contained in the portion of the route table in the cache, the data packet is further delayed while a complete software-based route table look-up is performed. If the route table is large, the delay caused by the full software route table look-up may be substantial.
Thus, the delay associated with contention for the shared bus and the route table caching and look-up problems of conventional routers lead to poor data packet transfer performance or lost data packets when a conventional router is placed in a large, dynamic network, such as the Internet. Thus, these conventional routers cannot achieve the data packet processing rates necessary for the future needs of the Internet and other high speed computer networks.
Instead of a conventional router with a shared bus architecture, a new class of the network switches and packet processing systems have been developed that attempt to efficiently process and transfer data packets through a switch to the next hop network. These packet processing systems attempt to overcome the problems with a SUBSTITUTE SHEET (RULE 26) conventional router by processing data packets more rapidly than conventional routers.
The speed of the processing of the data packets may be increased in a number of different ways. For example, one conventional packet processing system and method receives an incoming data packet and splits the incoming data packet into a header portion and a data portion. The data portion is stored in a conventional buffer memory while the header portion is stored in a cache memory which has a faster access time than the buffer memory. The cache memory permits the processor to rapidly access the header portions of data packets. The quicker access to the header portions of the data packets, since the processor needs to process the header portion to determine the next hop network address for a data packet, increases the data packet processing speed.
Since the processor does not need quick access to the data portion to process a data packet, the storage of the data portions in a slower access buffer memory is acceptable.
Once the next hop network address has been determined, the header portion and the corresponding data portion are combined back together and the data packet is sent on to its next hop network. Although the header portion is stored in a fast access time cache memory, the time it takes for the processor to access the cache memory is still slow and ultimately limits the overall processing speed of the packet processing system. In addition, the time required to access a route table in these systems is slow.
Thus, there is a need for a system and method for processing data packets which avoid these and other problems of known systems and methods, and it is to this end that the present invention is directed.
SUBSTITUTE SHEET (RULE 26) In accordance with the invention, a system and method for processing data packets is provided in which a hardware based data packet processing system efficiently processes data packets by reducing the amount of time associated with retrieving header information from a memory and associated with a route table look-up.
To reduce the route table look-up time, a hardware implemented look-up system is used that requires a single memory access per node to retrieve information from a search tree that is used to search the route table for a next hop network address. To reduce the time associated with memory accesses, the data portion of a data packet is stored in a buffer memory while the header portions are stored directly into a plurality of hardware registers that are directly accessible by a processor. In addition, the system may have two sets of header registers so that while a header in one set of the registers is being accessed and processed by the processor, a previously processed header may be unloaded and a next unprocessed header rnay be loaded into the second set of registers so that there is minimal downtime in between the processing of headers.
Thus, when a first header processing is complete and a new header may be processed by the processor, the header portion is already present in the set of registers and loading the header from a memory location is not required.
In addition, the invention also provides a mechanism so that the data packets that are going to be dropped are immediately removed from memory. The system also provides a unique method for processing multicast data packets. The processor within SUBSTITUTE SHEET (RULE 26) the system may have a separate route table search engine and a table search engine so that a route table search, a table search and the processing of other information in a header may occur simultaneously which reduces the time to process a header. To further reduce the processing time of a header, most of the data being operated on by the system may be stored in hardware registers.
Thus, in accordance with the invention, an apparatus and method for processing a data packet to determine the routing of the data packet through a communications network is provided in which the data packet has a header portion and a data portion.
The apparatus stores the header portion of the data packet, and processes the header portion of the data packet. The processing rnay include using a processing core for executing instructions for processing the header portion, searching through a route table to determine a route of the data packet, and searching through a table memory for information about the destination of the data packet. The header loading/unloading, the route table search, the table memory search , and the execution of header processing instructions may occur simultaneously. A modified header portion is generated to route the data packet through the communications network.
In accordance with another aspect of the invention, a method for processing data packets to determine the route of the data packet is also provided in which a header portion is received from an incoming data packet, a media information search is performed for certain types of data packets based on the received header portion, an interface search based on the received header portion is performed, and a route look-up SUBSTITUTE SHEET (RULE 2B) search is performed simultaneously with said interface search. The information contained within the header portion is processed while said interface search and said route look-up search are being perfonmed for determining if the header portion is valid and generating an internal header based on the results of the route look-up search and the interface search.
Figure 1 is a block diagram of a typical communications network that includes a packet processing system in accordance with the invention;
Figure 2 is a more detailed diagram of the packet processing system.
Figure 3 is a block diagram illustrating a media card for the packet processing system shown in Figure 2;
Figure 4 is a diagram illustrating the data flow through the media card of Figure 3;
Figure 5 is a block diagram illustrating the details of the packet processing unit (PPU) of Figure 4;
Figure 6A and 6B are a flowchart illustrating a method for packet header processing in accordance with the invention;
Figure 7 is a diagram illustrating a header stored in a set of header registers prior to processing by the packet processing unit;
Figure 8 is a diagram illustrating a modified header stored in the set of header registers after the processing by the packet processing unit; and SUBSTITUTE SHEET (RULE 26) Figure 9 is a flowchart illustrating a method for mufti-cast data packet processing in accordance with the invention.
The invention is particularly applicable to a system and method for processing messages in a communications network. This communications network may be a network which processes data packets such as the Internet or may be a network that processes telephone calls such as a telephone switching network. It is in this data communications context that the invention will be described. It will be appreciated, however, that the system and method in accordance with the invention has greater utility.
Figure 1 is a block diagram illustrating a communications network 20 that may contain a plurality of computer networks, such as network A, network B, and network C, which are interconnected together by a packet processing system 28 in accordance with the invention. The packet processing system receives data packets from one of the computer networks, determines the appropriate next computer network that the data packet should be sent to, and communicates the data packet to the proper next computer network. For example, a data packet 30 originates at a source host computer 32 attached to computer network A and has a destination of a destination host computer 34 attached to network B. To communicate the data packet between the source host computer and the destination host computer, the data packet is communicated to the SUBSTITUTE SHEET (RULE 26) packet processing system 28 which determines, based on a header portion of the data packet as described below, that the path of the data packet is to computer network B
since the destination host computer 34 is attached to that network. Therefore, the data packet 30 is forwarded to computer network B and then on to the destination host computer. The communications network shown is a very simple network but the packet processing system may also route data packets for complex communication networks, such as the Internet.
A certain amount of time is required for a packet processing system such as shown in Figure 1 to process a data packet. In particular, the packet processing system must determine the proper route of the data packet and then communicate the data packet along the proper route. It is desirable that the time that the data packet is processed by the packet processing system be minimized so that the time that a data packet is delayed through the communications system is minimized. In accordance with the invention, the packet processing system significantly reduces the time required to process and route a data packet, as described below, which increases the speed that data packets may be communicated over the communications network. To understand how the time required to process and route data packets is reduced, more details of the packet processing system in accordance with the invention will now be described.
Figure 2 is a block diagram of an example of the packet processing system 28, which may include a plurality of media cards, such as a first media card MCI, a second media card MC2 and a third media card MC3. Each media card may receive and SUBSTITUTE SHEET (RULE 26) transmit data packets over a particular type of media. In the example shown, an Ethernet media card, a Fiber Distributed Data Interface (FDDI) media card, an asynchronous transfer mode (ATM) media card and a SONET media card are provided.
Each media card may have a plurality of input ports 36 for receiving data packets and a plurality of output ports 37 for outputting data packets. A particular port on a media card may be capable of receiving and transmitting data packets simultaneously.
A data packet may be received by one media card over a first media, but that data packet may be output by the first media card over a different port or by a second media card over a second media. When a packet is destined for a different media card, the packet may be sent to the second card over a switch fabric 38. For example, an incoming ATM
data packet may be received by MC3 and the media card determines that the data packet is going to be output over an Ethernet network to reach its destination.
Therefore, the data packet is processed by MC3, sent over the switch fabric to MC1 and output by MC 1 over the Ethernet network. Thus, the packet processing system handles data packets from a plurality of different media. Now, a media card will be described in more detail.
Figure 3 is a block diagram illustrating more details of a media card MC1 in the packet processing system 28 in accordance with the invention. Each media card may be similar so only one will be described here. Each media card may include an ingress portion 40 and an egress portion 42. The ingress portion receives data packets from an external source, such as the particular type of media and processes those data packets to determine the appropriate route for the data packet, as described below.
The egress SUBSTITUTE SHEET (RULE 28) portion reformats the received data packet, determines the appropriate physical network address based on the route determined by the ingress portion, and communicates the data packet to a computer network connected to the packet processing system. A
plurality of media ports 44 are attached to both the ingress and egress portions, and these media ports communicate the data packets with the computer networks connected to the packet processing system. In the example of the packet processing system shown in Figure 3, there are four media ports including port 0 through port 3.
At any time, there may be data packets entering and exiting each media card for all of these media ports simultaneously.
A data packet entering the media card from a media port has a header portion and a data portion. The header portion contains information about the data packet such as the address of its destination, the type of media it is being transmitted over and other characteristics about the data packet. The data portion contains the actual data being communicated. The details of the header portion and data portion will be described below. Once a data packet enters the media card at the ingress portion 40, the data packet is separated into the header portion and the data portion and each of these portions is transferred over a bus 46 to a header in store 48 and a data buffer memory 50, respectively. A direct memory access (DMA) controller 52 controls the transfer of data between the various components within the ingress portion 40. To ensure that the header portion can be later reunited with the correct data portion, a descriptor, stored with the header portion and separately from the data portion, may indicate which data portion is associated with each header portion. The header portion of the data packet in SUBSTITUTE SHEET (RULE 2B) the header in store is transferred to a packet processor unit (PPU) S4 that determines the next hop network address for the data packet based on the information contained in the header portion, as described below.
To determine the next hop network address, the PPU 54 may perform a route table search using a route table memory 124. Once the PPU has determined the next hop network address for the data packet, the PPU may add additional information, such as the next hop network address, into the header portion, as described below, and the DMA 52 is notified of the completion of header processing. The DMA then transfers the header portion of the data packet from the PPU to a header out store 58.
When the modified header portion has reached the header out store 58, the DMA senses that and then reunites the header portion with the data portion which is still contained in the buffer memory 50 using the descriptor and communicates the entire data packet to a switch fabric out port 60. The data packet is then communicated by the switch fabric out port 60 over a switch fabric 38, as shown in Figure 2, of the packet processing system to a switch fabric in port 62 which is part of the egress portion 42 of the packet processing system. In this example, the switch fabric in port and the switch fabric out port are located on the same media card since the data packet is entering and exiting on the same media, but the ports may also be located on different media cards since the system may include a plurality of media cards connected together to handle various types of media.
SUBSTITUTE SHEET (RULE 28) A DMA 63 of the egress portion is then alerted to the presence of the data packet in the switch fabric in port and the DMA splits the header portion from the data portion and directs the header portion to a header in store 64 and the data portion to a data buffer memory 66. The header portion, including any additional information added by the ingress PPU 54, is then transmitted to a second packet processor unit (PPU) 68 which processes the header portion and determines the appropriate physical address of the next hop network from the next hop network address stored in the header portion by the ingress PPU 54 based on a table stored in the table memory 126 connected to the egress PPU 68. Once the proper physical address is determined, the DMA reunites the header portion with its appropriate data portion and transfers the data packet to one or more queues 72. The queues holds the data packet until the media port is prepared to transmit the data packet to its next hop network. One or more queues are utilized to separate traffic destined for different output ports and to prioritize the data packets.
For incoming data packets that are destined for more than one destination {e.g., mufti-cast data packets), the packet processing system may advantageously store a single copy of the data portion in the buffer memory and then generate a plurality of header portions for each destination. To facilitate this, the descriptors that are generated by the first PPU 54 permit multiple headers portions to correspond to the same data portion. Thus, only a single copy of the data portion of the data packet is required for a mufti-cast data packet which reduces the storage required to route a multicast data packet and increases the speed with which the multicast data packet is SUBSTITUTE SHEET (RULE 26) processed. To facilitate this mufti-casting, the descriptor may have a free buffer bit which is normally set by the ingress PPU 54 for non-mufti-cast data packets so that the data portion is removed from the buffer memory once the data packet has been forwarded. For a mufti-cast data packet, on the other hand, the free buffer bit is only set for the last header that references that data portion so that the data portion remains in the buffer memory until the mufti-casting operation has been completed.
As described above, once a header of a data packet has been received, the DMA
is alerted and the header portion is transferred to the packet processing units 54 which processes the information contained within the header portion and determines, among other things, the next hop address for the data packet. In some embodiments of the invention, there may be multiple PPU's that simultaneously process headers.
Each PPU operates in the same manner and will be described in more detail below with reference to Figures 4 -5. In addition, the method of processing a header in accordance with the invention will be described below with reference to Figures 6A and 6B. Each PPU utilizes the set of memories 56 to process the header. The set of memories may include a table memory (TMEM) 120, an instruction memory (IMEM) 122 and a route table look-up memory (RTMEM) 124. The TMEM may hold tables to look up information about a data packet, the .IMEM may hold the instructions being executed by the PPU and the RTMEM may store a route table for determining a route of a data packet. The operation of these memories will be described below.
SUBSTITUTE SHEET (RULE 26) Once the header has been processed by one of the PPU's, an internal header is generated by the PPU and the internal header is included in the original header portion to form a modified header portion. Instead of an internal header, a network media header may also be generated by the PPU. The modified header portion is then stored in the header output store 58. Once the header reaches the header output store, the DMA 52 is alerted and the header portion with the internal header and the data portion are reunited in the switch fabric out port 60. Then the entire data packet may be transmitted over the switch fabric of the packet processing system. The data packet transmitted over the switch fabric may be received by the switch fabric in port 62. The data packet with the internal header is then processed by the PPU's 68, that may be a pair of PPU's that may simultaneously process two headers, and then transmit the processed headers to the queues 72. Each PPU may have a set of memories 70 that store data and instructions used by the PPU to process the header. In particular, a table memory 134 (TMEM) and an instruction memory 136 (IMEM) are provided. For the egress portion 42, a RTMEM is not necessary since the route of the data packet has already been determined. The PPU's in the egress portion remove the internal header, and generate the appropriate physical address for the particular media. Then the data packet with the proper header is transmitted out over the media. Now, the details of the PPU will be described with reference to Figures 4 and 5.
Figure 4 is a block diagram illustrating the major components of the PPU along with the associated memories and other external components. Figure 5 is a block diagram illustrating the details of the packet processing unit 116 in accordance with the SUBSTITUTE SHEET (RULE 26) invention. The PPU described is one of the PPU's that reside on the ingress portion of a media card. The egress PPU's may be similar and will not be described. The CPU
142 and the data buffer 50 are shown for reference and are not part of the PPU. As each of the plurality of data packets enter the media card (not shown), the header portion is split from the data portion. As described above, the data portion is stored in the data buffer memory 50. The header portion is transferred to the header in store 48 shown in Figure 3 and then into a set of header registers 150. In accordance with the invention, there may be a first set of header registers 152 and a second set of header registers 154 so that, as a header in the first set of registers is being processed by the sequence engine 176 as shown in Figure 5, the next header to be processed by the sequence engine may be loaded into the second set of header registers. Thus, once the PPU has completed processing a header, it can start processing the next header during the next clock cycle since the next header is already loaded into the second set of registers. In addition, a processed header may be unloaded from a set of registers while another header is being processed by the PPU. Therefore, the delay associated with loading and/or unloading the header into and out of the registers is minimized which increases the speed that headers may be processed by the PPU.
In accordance with another aspect of the invention, the PPU operates using a plurality of registers that may be more rapidly accessed than conventional memories.
These registers hold various data being operated on by the sequence engine 176, the ALU 164, the table engine 172, and the route table search engine 170, all of which are as shown in Figure 5, and instructions between the functional units within the PPU. As SUBSTITUTE SHEET (RULE 26) WO 99/18b99 PCTIUS98/20795 shown, the PPU may have the set of header registers 150, a set of general registers 156, a set of mailbox registers 158, a flat table look-up set of registers 160, and a set of route table look-up registers 162. Each of these registers will be described below.
The heart of the PPU may be a processing core, such as an arithmetic logic unit (ALU) 164, that receives microcode instructions from the instruction memory I22. Through the sequence engine 176, and in accordance with the invention, the ALU accesses information and data from any of the registers in order to process a header portion of a data packet and also executes instructions stored in the instruction memory.
In addition, due to the architecture of the PPU, the load and/or unloading of the headers, the flat table and route table searches and the other header processing may occur simultaneously which decreases the total time that it takes for a PPU to process a header. The ALU may execute a plurality of unique microcode instructions.
Some of these unique instructions may be specialized for processing data packets such as, for example, Internet Protocol (IP) data packets. Several examples of some of these specialized data packet processing instructions include a jump using vector register (JVEC) instruction, a SEND instruction, a SENDM instruction, an EXCP instruction, an EXCPM instruction, and a DROP instruction. Each of these instructions will be described below. The SEND instruction sends a processed header portion of a data packet to an output FIFO 184 of the PPU so that the data packet may be forwarded and loads a next unprocessed header. The SENDM instruction is used for multicast data packets as described below with reference to Figure 9. The EXCP
instruction sends a header portion of a data packet to an exception header buffer 180 SUBSTITUTE SHEET (RULE 26j and loads a next unprocessed header while the EXCPM instruction sends a header portion of a multi-cast data packet to the exception header buffer 180 and reuses the header portion in a similar manner to the SENDM instruction. The DROP
instruction discards the data packet, loads the next unprocessed header portion into the PPU and places information about a data packet to be discarded into a drop FIFO 186 so that the memory space occupied by the discarded data packet may be reused immediately.
Thus, these specialized instructions permit the PPU to quickly process data packets.
Now, the JVEC instruction will be described.
The JVEC instruction and the associated vector registers that are accessed by the JVEC instruction permit the CPU to steer the execution by the PPU to certain routines. In operation, the PPU may execute a set of instructions until a JVEC
instruction is encountered and then the next instructions to be executed by the PPU will be determined based on the address in the vector registers. The address in the vector registers is determined by the CPU so that the CPU may direct the PPU to different routines depending on the state of the PPU.
The JVEC instruction has three choices of routines to which it can direct the PPU including a idle routine, a packet routine and an interrupt routine. The selection of the vector routine is on a priority basis so that, for example, if the interrupt vector has been updated since the last time a JVEC instruction was executed, then the interrupt routine is selected regardless of the status of the other vector registers. Thus, the interrupt vector has the highest priority. The second highest priority vector routine SUBSTITUTE SHEET (RULE 2B) is the packet routine which is executed if the header Il0 engine indicates that a new header is ready for processing and there is no interrupt vector. The lowest priority routine is the idle routine which is the default routine and will be executed only if neither of the other vector conditions have occurred.
An example of the JVEC instruction as used during normal data packet processing may be:
IDLE:
JVEC;
NOP;
PACKET:
//process packets here //packet processing completed SEND;
JVEC;
NOP;
In this example, after the PPU has completed the processing of a header portion and has issued a SEND instruction which places the header portion in the output FIFO, as described above, the JVEC instruction is executed. In this example, there is no header portion to be processed by the PPU so the vector registers have been loaded with the address of the IDLE routine and the PPU executes the instructions within the IDLE routine. In the IDLE routine, the PPU executes another JVEC instruction which returns the PPU to the IDLE routine as long as there are no header portions ready for processing and the interrupt vector has not been written since the last JVEC
instruction.
When a header portion is ready for processing, the sequence engine is alerted and SUBSTITUTE SHEET (RULE 26) chooses the address in the packet vector register and the PPU will begin executing the instructions in the PACKET routine the next time that it executes the JVEC
instruction.
In addition, as conditions in the packet processing system change, the CPU may change the address in the vector registers to direct the PPU to another routine. For example, if the CPU determines that the packet processing system is low on buffer space, it may direct the PPU to execute a different packet routine which implements the selective discarding of data packets until more buffer space is available.
In addition, the use of the JVEC instruction may permit the CPU to update the packet processing code. To update the packet processing code, the CPU steers the PPU
to special routine where it will no longer process packets. The PPU, through the mailbox registers, will indicate that it is currently executing this special routine so that the CPU may halt the PPU and update the instruction memory with the desired changes. Once the code has been updated, the vector registers are re-initialized and the PPU is unhalted. The update method prevents the PPU and the packet processing system from losing header data and those buffer memory regions associated with a packet because the PPU was halted only after the current header being processed has been completed. Now, other components associated with the PPU will be described.
The set of general registers may store a variety of general information used by the ALU 164. The mailbox registers 158 permit the ALU to communicate with the CPU 142. The flat table look-up registers permit the ALU to request that a flat table look-up search, as described below, be performed by a table engine 172 shown in Figure 5 and the results of the flat table search be stored in the flat table registers 160 SUBSTITUTE SHEET (RULE 26) so that the ALU may access the results. The route table look-up registers 162 permit the ALU to request a route table look-up search from a route table search engine 170 shown in Figure 5, as described below, and the results of the route table search are stored in the route table registers 162. The ALU may then retrieve the results of either a route table search or a flat table search from the appropriate registers. In accordance with the invention, the flat table and route table searches may be performed simultaneously by the respective table engines and do not require any processing time of the ALU, as described below. In accordance with the invention, the data being accessed by the ALU and the search engines are contained in hardware registers since these are more rapidly accessible than conventional memories. This has the result of increasing the speed with which a header may be processed by the PPU.
The processing of the header portion will now be briefly described. It will be described in more detail below with reference to Figures 6A and 6B. Once the header portion has been loaded into one of the header registers 152, 154, the ALU
accesses a portion of the header register, as described below, and requests that a flat table search be conducted by the table search engine 172 using a portion of the header by writing an instruction into the flat table registers 160. The ALU then reads a second portion of the header and requests that a route table search be performed by the route table search engine 170 based on the portion of the header by writing an instruction into the route table search registers 162. While the flat table and route table searches are being performed by the respective search engines, the ALU may perform additional processing on the header such as verifying the checksum contained in the header, SUBSTITUTE SHEET (RULE 28) decrementing a Time To Live (TTL) value in a portion of the header and recalculating a checksum value. Once the searches have been completed and the results are returned to the registers, the ALU may generate an internal header which includes the results of the flat table search and the route table search and attach the internal header to the existing header stored in one of the header registers 152, 154. The modified header portion is then reunited with the data portion and output to the switch fabric, as described above. Additional details about the structure of the PPU will now be described.
Figure 5 is a block diagram illustrating the details of the packet processing unit 116 in accordance with the invention. In addition to the ALU 164 and the various registers described above, the PPU may also include a route table search engine 170, a flat table engine 172, a program counter 174, a sequence engine 176 and a CPU
interface 179 that permits the CPU 142 shown in Figure 4 to access some of the PPU's registers and each of the PPU's associated memories. The various engines are tightly coupled together which further increases the processing speed of the PPU. To tightly couple these components together, these components may reside on the same semiconductor chip. The CPU may control some of these functional blocks. The route table search engine 170 performs a route table look-up search when an instruction is placed in the route table registers 162. The flat table look-up search engine 172 may perform a flat table search when an instruction is placed in the flat table registers 160.
The program counter 174 keeps track of the address of the microcode being executed and the sequence engine 176 fetches and decodes the next microcode instruction to be SUBSTITUTE SHEET (RULE 26) executed by the ALU 164. The sequence engine also has the ability to execute conditional branches based on the results of the ALU.
To route the data between these units within the PPU, a plurality of multiplexers 178 are used. In addition, a header input/output (I/O) engine 247 is provided which controls the flow of headers into and out of the PPU. To facilitate the movement of the headers within the PPU, the header I/O engine 247 may control an input buffer 182, an output buffer 184, a dump buffer 186, a header exception buffer 180 and a CPU header buffer 181. The header I/O engine controls header movement between the sets of header registers 152, 154 and the input buffer, the output buffer, the dump buffer and the header exception buffer. The header I/O engine also controls header movement between the CPU header buffer and the input buffer, the output buffer, the dump buffer and the header exception buffer. The header I/O engine may control the input buffer 182 that receives unprocessed headers entering the PPU and the output buffer 184 for sending headers that have been processed to the output header buffer of the packet processing system. In addition, the header I/O engine may also control the dump buffer 186 which holds descriptors for data packets that are going to be dropped by the packet processing system because, for example, the header contains corrupt information. The dump buffer may be directly connected to the DMA 52 shown in Figure 4 so that the data portion that corresponds to the dumped header portion may be discarded and the buffer space occupied by the data portion in the buffer memory 50 shown in Figure 4 may be immediately reused. The header I/O
engine may also control a header exception buffer for data packets that can be SUBSTITUTE SHEET (RULE 26) WO 99/18b99 PCT/US98/20795 processed more efficiently by the CPU. The header I/O engine may control a CPU
header buffer for headers that were processed by the CPU. In this manner, the header I/O engine has a plurality of different options for routing an incoming or outgoing header portion from the PPU. Now, a method for processing a header portion of a data packet in accordance with the invention will be described with reference to Figures 6A, 6B, 7, and 8.
Figures 6A and 6B illustrate a flowchart detailing the steps of a method 200 for processing a header portion of a data packet in accordance with the invention.
Figures 7 and 8 are diagrams which illustrate an example of the contents of the header registers prior to processing by the PPU and after processing by the PPU. Returning to Figure 6A, the method begins at step 202 in which the next header portion to be processed by the ALU is stored in one of the set of header registers and is ready for processing. As shown in Figure 7, in this example, the first set of header registers 152 has a total of thirty-two registers and the header portion of the data packet starts at header register 9 (HD9) and fills the rest of the registers. In this example, a frame relay data packet header is shown which may include the frame relay header 204, an IP packet header 206, payload data 208, a DMA descriptor 210 and a media descriptor 212. A data packet from another type of media may have a slightly different header fomlat and may be stored starting at a different individual header register. The PPU in accordance with the invention may process a plurality of different types of data packets transmitted over a plurality of different media, such as FDDI or Ethemet or frame relay and the header portions for each of these data packets is slightly different, but can also be processed by SUBSTITUTE SHEET (RULE 26) the PPU. The first nine registers (HDO - HD 8) 214 are used to store the internal header generated by the PPU, as described below.
Once the header portion has been received, the PPU extracts the frame relay header 204 in step 216, and requests a flat table search to be conducted to retrieve an entry for this frame relay circuit in a circuit table contained in the table memory. Step 216 may be executed for fi~ame relay or ATM data packets, but is not executed for other types of data packets. Next, in step 218, a second flat table search is conducted into the interface table using the inbound interface number contained in the header portion. Next, in step 220, while the other flat table search is being performed, the PPU requests that a route table search is conducted using the destination IP
address contained in HD14 in this example. While these searches are being conducted by the appropriate search engines, the PPU may, in step 222, perform additional header processing and header information checks that do not require the results of the searches, such as verifying the checksum in the header, decrernenting the Time To Live (TTL) value and recalculating the checksum. Once these checks have been performed, the PPU determines if the interface table look-up has been completed in step 224 and waits, if it has not been completed. When the results of the interface table have been received, the PPU ensures that the appropriate inbound interface has been correctly configured, increments the interface statistics and writes the record back into the table memory in step 226. In step 228, the PPU determines if the route table search has been completed and waits until it has been completed. Once the results of the route table search are available, the next hop address is placed in the route table registers in step SUBSTITUTE SHEET (RULE 2B) 230 by the route table search engine. In step 232, assuming the next hop address is correct, the PPU generates an internal header in HDO - HD9 214 using the next hop address information. A modified header that contains the internal header is shown in Figure 8. Once the header has been modified to include the internal header, the PPU
issues a SEND command in step 234. The header register sets are then switched, the processed header is unloaded, and the modified header is reunited with the data portion by the DMA and transmitted across the switch fabric. After transmission, the storage for the data portion in the buffer is released. The method loops back to step immediately upon the switching of the header register sets and begins processing another header portion. As described above, due to the architecture of the PPU, only a single clock cycle elapses between the end of a header processing and the start of the next header processing. In addition, the PPU performs a plurality of operations in parallel, such as header loading and/or unloading, the flat table and route table searches, and other header processing, which substantially reduces the time required to process a header. Now, a method for processing multicast data packets in accordance with the invention will be described.
Figure 9 is a flowchart illustrating a method 240 for processing the header of a multicast data packet in accordance with the invention. In step 242, the PPU
determines, from data in the header portion, whether the data packet is a multicast data packet (e.g., a data packet destined for more than one destination). If the data packet is not a multicast data packet, normal header processing as described above occurs. If a multicast data packet is encountered, the header is processed, as described above, in SUBSTITUTE SHEET (RULE 26) step 244. Once the data packet has been processed, a SENDM command is issued by the PPU in step 246. The SENDM command differs from a SEND command in that the SENDM command does not release the processed header registers. Instead, the processed header portion is returned to the working register set after a copy of it has been unloaded. In this manner, the SENDM instruction duplicates headers. Each header generated will point to the same buffer space occupied by the data portion.
Once the SENDM command is issued, the header I/O engine 247, as shown in Figure S, copies the processed header to the output buffer 184 and then returns the same header to the working set of header registers. Once the header has been returned to the working header registers, the PPU modifies the internal header for the next address in step 250. Next, in step 252, the PPU determines if the current address is the last address for the multicast data packet. If there are no additional addresses for the multicast data packet, then, in step 254, the PPU issues a SEND command which routes the modified header to the packet processing system and allows the removal of the header portion from the I/O header registers. If there is an additional address for the multicast data packet, the method loops back to step 246 in which the SENDM
command is issued. Thus, in accordance with the invention, a multicast data packet may be rapidly communicated to each of the destination addresses without needing a separate data portion for each destination which substantially reduces the amount of buffer memory required to store a multicast data packet.
While the foregoing has been with reference to a particular embodiment of the invention, it wi 11 be appreciated by those skilled in the art that changes in this SUBSTITUTE SHEET (RULE 26) embodiment may be made without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims.
SUBSTITUTE SHEET (RULE 26)
The amount of Internet Protocol (IP) data packet traffic being communicated over the Internet, as well as Intranets, has grown exponentially over the past decade.
Communications networks are constantly being expanded to handle this growth.
The expansion of the communications networks is typically accomplished by partitioning a network that is too large into a number of smaller communication networks in which each smaller network is serviced by a separate router, and then relinking the smaller partitioned networks back together with inter-networking equipment. However, even with this cycle of partitioning and relinking of communication networks, in some environments, such as large-scale public and private data backbones, the data packet traffic has reached such tremendous amounts that these conventional routers cannot handle the data packet traffic and proposed enhancements to these conventional routers cannot keep pace with the ever increasing amount of data packet traffic.
Normally, a conventional router is used in a communications network to route data packets through the network from a first location, known as a source, to a second SUBSTITUTE SHEET (RULE 26) location, known as a destination. These conventional routers typically have a single shared data bus so that each data packet received by the router must wait for access to the shared bus before being forwarded to the next network "hop" along the data path to the destination address of the data packet. A portion of a route table for a router, which is used to determine the next network hop for a data packet, is typically stored in a cache memory. The portion of the route table stored in the cache memory is typically the most frequently used next network addresses. Using the route table data stored in the cache, the muter attempts to determine the next network address for a particular data packet. If the next network address of the data packets is not contained in the portion of the route table in the cache, the data packet is further delayed while a complete software-based route table look-up is performed. If the route table is large, the delay caused by the full software route table look-up may be substantial.
Thus, the delay associated with contention for the shared bus and the route table caching and look-up problems of conventional routers lead to poor data packet transfer performance or lost data packets when a conventional router is placed in a large, dynamic network, such as the Internet. Thus, these conventional routers cannot achieve the data packet processing rates necessary for the future needs of the Internet and other high speed computer networks.
Instead of a conventional router with a shared bus architecture, a new class of the network switches and packet processing systems have been developed that attempt to efficiently process and transfer data packets through a switch to the next hop network. These packet processing systems attempt to overcome the problems with a SUBSTITUTE SHEET (RULE 26) conventional router by processing data packets more rapidly than conventional routers.
The speed of the processing of the data packets may be increased in a number of different ways. For example, one conventional packet processing system and method receives an incoming data packet and splits the incoming data packet into a header portion and a data portion. The data portion is stored in a conventional buffer memory while the header portion is stored in a cache memory which has a faster access time than the buffer memory. The cache memory permits the processor to rapidly access the header portions of data packets. The quicker access to the header portions of the data packets, since the processor needs to process the header portion to determine the next hop network address for a data packet, increases the data packet processing speed.
Since the processor does not need quick access to the data portion to process a data packet, the storage of the data portions in a slower access buffer memory is acceptable.
Once the next hop network address has been determined, the header portion and the corresponding data portion are combined back together and the data packet is sent on to its next hop network. Although the header portion is stored in a fast access time cache memory, the time it takes for the processor to access the cache memory is still slow and ultimately limits the overall processing speed of the packet processing system. In addition, the time required to access a route table in these systems is slow.
Thus, there is a need for a system and method for processing data packets which avoid these and other problems of known systems and methods, and it is to this end that the present invention is directed.
SUBSTITUTE SHEET (RULE 26) In accordance with the invention, a system and method for processing data packets is provided in which a hardware based data packet processing system efficiently processes data packets by reducing the amount of time associated with retrieving header information from a memory and associated with a route table look-up.
To reduce the route table look-up time, a hardware implemented look-up system is used that requires a single memory access per node to retrieve information from a search tree that is used to search the route table for a next hop network address. To reduce the time associated with memory accesses, the data portion of a data packet is stored in a buffer memory while the header portions are stored directly into a plurality of hardware registers that are directly accessible by a processor. In addition, the system may have two sets of header registers so that while a header in one set of the registers is being accessed and processed by the processor, a previously processed header may be unloaded and a next unprocessed header rnay be loaded into the second set of registers so that there is minimal downtime in between the processing of headers.
Thus, when a first header processing is complete and a new header may be processed by the processor, the header portion is already present in the set of registers and loading the header from a memory location is not required.
In addition, the invention also provides a mechanism so that the data packets that are going to be dropped are immediately removed from memory. The system also provides a unique method for processing multicast data packets. The processor within SUBSTITUTE SHEET (RULE 26) the system may have a separate route table search engine and a table search engine so that a route table search, a table search and the processing of other information in a header may occur simultaneously which reduces the time to process a header. To further reduce the processing time of a header, most of the data being operated on by the system may be stored in hardware registers.
Thus, in accordance with the invention, an apparatus and method for processing a data packet to determine the routing of the data packet through a communications network is provided in which the data packet has a header portion and a data portion.
The apparatus stores the header portion of the data packet, and processes the header portion of the data packet. The processing rnay include using a processing core for executing instructions for processing the header portion, searching through a route table to determine a route of the data packet, and searching through a table memory for information about the destination of the data packet. The header loading/unloading, the route table search, the table memory search , and the execution of header processing instructions may occur simultaneously. A modified header portion is generated to route the data packet through the communications network.
In accordance with another aspect of the invention, a method for processing data packets to determine the route of the data packet is also provided in which a header portion is received from an incoming data packet, a media information search is performed for certain types of data packets based on the received header portion, an interface search based on the received header portion is performed, and a route look-up SUBSTITUTE SHEET (RULE 2B) search is performed simultaneously with said interface search. The information contained within the header portion is processed while said interface search and said route look-up search are being perfonmed for determining if the header portion is valid and generating an internal header based on the results of the route look-up search and the interface search.
Figure 1 is a block diagram of a typical communications network that includes a packet processing system in accordance with the invention;
Figure 2 is a more detailed diagram of the packet processing system.
Figure 3 is a block diagram illustrating a media card for the packet processing system shown in Figure 2;
Figure 4 is a diagram illustrating the data flow through the media card of Figure 3;
Figure 5 is a block diagram illustrating the details of the packet processing unit (PPU) of Figure 4;
Figure 6A and 6B are a flowchart illustrating a method for packet header processing in accordance with the invention;
Figure 7 is a diagram illustrating a header stored in a set of header registers prior to processing by the packet processing unit;
Figure 8 is a diagram illustrating a modified header stored in the set of header registers after the processing by the packet processing unit; and SUBSTITUTE SHEET (RULE 26) Figure 9 is a flowchart illustrating a method for mufti-cast data packet processing in accordance with the invention.
The invention is particularly applicable to a system and method for processing messages in a communications network. This communications network may be a network which processes data packets such as the Internet or may be a network that processes telephone calls such as a telephone switching network. It is in this data communications context that the invention will be described. It will be appreciated, however, that the system and method in accordance with the invention has greater utility.
Figure 1 is a block diagram illustrating a communications network 20 that may contain a plurality of computer networks, such as network A, network B, and network C, which are interconnected together by a packet processing system 28 in accordance with the invention. The packet processing system receives data packets from one of the computer networks, determines the appropriate next computer network that the data packet should be sent to, and communicates the data packet to the proper next computer network. For example, a data packet 30 originates at a source host computer 32 attached to computer network A and has a destination of a destination host computer 34 attached to network B. To communicate the data packet between the source host computer and the destination host computer, the data packet is communicated to the SUBSTITUTE SHEET (RULE 26) packet processing system 28 which determines, based on a header portion of the data packet as described below, that the path of the data packet is to computer network B
since the destination host computer 34 is attached to that network. Therefore, the data packet 30 is forwarded to computer network B and then on to the destination host computer. The communications network shown is a very simple network but the packet processing system may also route data packets for complex communication networks, such as the Internet.
A certain amount of time is required for a packet processing system such as shown in Figure 1 to process a data packet. In particular, the packet processing system must determine the proper route of the data packet and then communicate the data packet along the proper route. It is desirable that the time that the data packet is processed by the packet processing system be minimized so that the time that a data packet is delayed through the communications system is minimized. In accordance with the invention, the packet processing system significantly reduces the time required to process and route a data packet, as described below, which increases the speed that data packets may be communicated over the communications network. To understand how the time required to process and route data packets is reduced, more details of the packet processing system in accordance with the invention will now be described.
Figure 2 is a block diagram of an example of the packet processing system 28, which may include a plurality of media cards, such as a first media card MCI, a second media card MC2 and a third media card MC3. Each media card may receive and SUBSTITUTE SHEET (RULE 26) transmit data packets over a particular type of media. In the example shown, an Ethernet media card, a Fiber Distributed Data Interface (FDDI) media card, an asynchronous transfer mode (ATM) media card and a SONET media card are provided.
Each media card may have a plurality of input ports 36 for receiving data packets and a plurality of output ports 37 for outputting data packets. A particular port on a media card may be capable of receiving and transmitting data packets simultaneously.
A data packet may be received by one media card over a first media, but that data packet may be output by the first media card over a different port or by a second media card over a second media. When a packet is destined for a different media card, the packet may be sent to the second card over a switch fabric 38. For example, an incoming ATM
data packet may be received by MC3 and the media card determines that the data packet is going to be output over an Ethernet network to reach its destination.
Therefore, the data packet is processed by MC3, sent over the switch fabric to MC1 and output by MC 1 over the Ethernet network. Thus, the packet processing system handles data packets from a plurality of different media. Now, a media card will be described in more detail.
Figure 3 is a block diagram illustrating more details of a media card MC1 in the packet processing system 28 in accordance with the invention. Each media card may be similar so only one will be described here. Each media card may include an ingress portion 40 and an egress portion 42. The ingress portion receives data packets from an external source, such as the particular type of media and processes those data packets to determine the appropriate route for the data packet, as described below.
The egress SUBSTITUTE SHEET (RULE 28) portion reformats the received data packet, determines the appropriate physical network address based on the route determined by the ingress portion, and communicates the data packet to a computer network connected to the packet processing system. A
plurality of media ports 44 are attached to both the ingress and egress portions, and these media ports communicate the data packets with the computer networks connected to the packet processing system. In the example of the packet processing system shown in Figure 3, there are four media ports including port 0 through port 3.
At any time, there may be data packets entering and exiting each media card for all of these media ports simultaneously.
A data packet entering the media card from a media port has a header portion and a data portion. The header portion contains information about the data packet such as the address of its destination, the type of media it is being transmitted over and other characteristics about the data packet. The data portion contains the actual data being communicated. The details of the header portion and data portion will be described below. Once a data packet enters the media card at the ingress portion 40, the data packet is separated into the header portion and the data portion and each of these portions is transferred over a bus 46 to a header in store 48 and a data buffer memory 50, respectively. A direct memory access (DMA) controller 52 controls the transfer of data between the various components within the ingress portion 40. To ensure that the header portion can be later reunited with the correct data portion, a descriptor, stored with the header portion and separately from the data portion, may indicate which data portion is associated with each header portion. The header portion of the data packet in SUBSTITUTE SHEET (RULE 2B) the header in store is transferred to a packet processor unit (PPU) S4 that determines the next hop network address for the data packet based on the information contained in the header portion, as described below.
To determine the next hop network address, the PPU 54 may perform a route table search using a route table memory 124. Once the PPU has determined the next hop network address for the data packet, the PPU may add additional information, such as the next hop network address, into the header portion, as described below, and the DMA 52 is notified of the completion of header processing. The DMA then transfers the header portion of the data packet from the PPU to a header out store 58.
When the modified header portion has reached the header out store 58, the DMA senses that and then reunites the header portion with the data portion which is still contained in the buffer memory 50 using the descriptor and communicates the entire data packet to a switch fabric out port 60. The data packet is then communicated by the switch fabric out port 60 over a switch fabric 38, as shown in Figure 2, of the packet processing system to a switch fabric in port 62 which is part of the egress portion 42 of the packet processing system. In this example, the switch fabric in port and the switch fabric out port are located on the same media card since the data packet is entering and exiting on the same media, but the ports may also be located on different media cards since the system may include a plurality of media cards connected together to handle various types of media.
SUBSTITUTE SHEET (RULE 28) A DMA 63 of the egress portion is then alerted to the presence of the data packet in the switch fabric in port and the DMA splits the header portion from the data portion and directs the header portion to a header in store 64 and the data portion to a data buffer memory 66. The header portion, including any additional information added by the ingress PPU 54, is then transmitted to a second packet processor unit (PPU) 68 which processes the header portion and determines the appropriate physical address of the next hop network from the next hop network address stored in the header portion by the ingress PPU 54 based on a table stored in the table memory 126 connected to the egress PPU 68. Once the proper physical address is determined, the DMA reunites the header portion with its appropriate data portion and transfers the data packet to one or more queues 72. The queues holds the data packet until the media port is prepared to transmit the data packet to its next hop network. One or more queues are utilized to separate traffic destined for different output ports and to prioritize the data packets.
For incoming data packets that are destined for more than one destination {e.g., mufti-cast data packets), the packet processing system may advantageously store a single copy of the data portion in the buffer memory and then generate a plurality of header portions for each destination. To facilitate this, the descriptors that are generated by the first PPU 54 permit multiple headers portions to correspond to the same data portion. Thus, only a single copy of the data portion of the data packet is required for a mufti-cast data packet which reduces the storage required to route a multicast data packet and increases the speed with which the multicast data packet is SUBSTITUTE SHEET (RULE 26) processed. To facilitate this mufti-casting, the descriptor may have a free buffer bit which is normally set by the ingress PPU 54 for non-mufti-cast data packets so that the data portion is removed from the buffer memory once the data packet has been forwarded. For a mufti-cast data packet, on the other hand, the free buffer bit is only set for the last header that references that data portion so that the data portion remains in the buffer memory until the mufti-casting operation has been completed.
As described above, once a header of a data packet has been received, the DMA
is alerted and the header portion is transferred to the packet processing units 54 which processes the information contained within the header portion and determines, among other things, the next hop address for the data packet. In some embodiments of the invention, there may be multiple PPU's that simultaneously process headers.
Each PPU operates in the same manner and will be described in more detail below with reference to Figures 4 -5. In addition, the method of processing a header in accordance with the invention will be described below with reference to Figures 6A and 6B. Each PPU utilizes the set of memories 56 to process the header. The set of memories may include a table memory (TMEM) 120, an instruction memory (IMEM) 122 and a route table look-up memory (RTMEM) 124. The TMEM may hold tables to look up information about a data packet, the .IMEM may hold the instructions being executed by the PPU and the RTMEM may store a route table for determining a route of a data packet. The operation of these memories will be described below.
SUBSTITUTE SHEET (RULE 26) Once the header has been processed by one of the PPU's, an internal header is generated by the PPU and the internal header is included in the original header portion to form a modified header portion. Instead of an internal header, a network media header may also be generated by the PPU. The modified header portion is then stored in the header output store 58. Once the header reaches the header output store, the DMA 52 is alerted and the header portion with the internal header and the data portion are reunited in the switch fabric out port 60. Then the entire data packet may be transmitted over the switch fabric of the packet processing system. The data packet transmitted over the switch fabric may be received by the switch fabric in port 62. The data packet with the internal header is then processed by the PPU's 68, that may be a pair of PPU's that may simultaneously process two headers, and then transmit the processed headers to the queues 72. Each PPU may have a set of memories 70 that store data and instructions used by the PPU to process the header. In particular, a table memory 134 (TMEM) and an instruction memory 136 (IMEM) are provided. For the egress portion 42, a RTMEM is not necessary since the route of the data packet has already been determined. The PPU's in the egress portion remove the internal header, and generate the appropriate physical address for the particular media. Then the data packet with the proper header is transmitted out over the media. Now, the details of the PPU will be described with reference to Figures 4 and 5.
Figure 4 is a block diagram illustrating the major components of the PPU along with the associated memories and other external components. Figure 5 is a block diagram illustrating the details of the packet processing unit 116 in accordance with the SUBSTITUTE SHEET (RULE 26) invention. The PPU described is one of the PPU's that reside on the ingress portion of a media card. The egress PPU's may be similar and will not be described. The CPU
142 and the data buffer 50 are shown for reference and are not part of the PPU. As each of the plurality of data packets enter the media card (not shown), the header portion is split from the data portion. As described above, the data portion is stored in the data buffer memory 50. The header portion is transferred to the header in store 48 shown in Figure 3 and then into a set of header registers 150. In accordance with the invention, there may be a first set of header registers 152 and a second set of header registers 154 so that, as a header in the first set of registers is being processed by the sequence engine 176 as shown in Figure 5, the next header to be processed by the sequence engine may be loaded into the second set of header registers. Thus, once the PPU has completed processing a header, it can start processing the next header during the next clock cycle since the next header is already loaded into the second set of registers. In addition, a processed header may be unloaded from a set of registers while another header is being processed by the PPU. Therefore, the delay associated with loading and/or unloading the header into and out of the registers is minimized which increases the speed that headers may be processed by the PPU.
In accordance with another aspect of the invention, the PPU operates using a plurality of registers that may be more rapidly accessed than conventional memories.
These registers hold various data being operated on by the sequence engine 176, the ALU 164, the table engine 172, and the route table search engine 170, all of which are as shown in Figure 5, and instructions between the functional units within the PPU. As SUBSTITUTE SHEET (RULE 26) WO 99/18b99 PCTIUS98/20795 shown, the PPU may have the set of header registers 150, a set of general registers 156, a set of mailbox registers 158, a flat table look-up set of registers 160, and a set of route table look-up registers 162. Each of these registers will be described below.
The heart of the PPU may be a processing core, such as an arithmetic logic unit (ALU) 164, that receives microcode instructions from the instruction memory I22. Through the sequence engine 176, and in accordance with the invention, the ALU accesses information and data from any of the registers in order to process a header portion of a data packet and also executes instructions stored in the instruction memory.
In addition, due to the architecture of the PPU, the load and/or unloading of the headers, the flat table and route table searches and the other header processing may occur simultaneously which decreases the total time that it takes for a PPU to process a header. The ALU may execute a plurality of unique microcode instructions.
Some of these unique instructions may be specialized for processing data packets such as, for example, Internet Protocol (IP) data packets. Several examples of some of these specialized data packet processing instructions include a jump using vector register (JVEC) instruction, a SEND instruction, a SENDM instruction, an EXCP instruction, an EXCPM instruction, and a DROP instruction. Each of these instructions will be described below. The SEND instruction sends a processed header portion of a data packet to an output FIFO 184 of the PPU so that the data packet may be forwarded and loads a next unprocessed header. The SENDM instruction is used for multicast data packets as described below with reference to Figure 9. The EXCP
instruction sends a header portion of a data packet to an exception header buffer 180 SUBSTITUTE SHEET (RULE 26j and loads a next unprocessed header while the EXCPM instruction sends a header portion of a multi-cast data packet to the exception header buffer 180 and reuses the header portion in a similar manner to the SENDM instruction. The DROP
instruction discards the data packet, loads the next unprocessed header portion into the PPU and places information about a data packet to be discarded into a drop FIFO 186 so that the memory space occupied by the discarded data packet may be reused immediately.
Thus, these specialized instructions permit the PPU to quickly process data packets.
Now, the JVEC instruction will be described.
The JVEC instruction and the associated vector registers that are accessed by the JVEC instruction permit the CPU to steer the execution by the PPU to certain routines. In operation, the PPU may execute a set of instructions until a JVEC
instruction is encountered and then the next instructions to be executed by the PPU will be determined based on the address in the vector registers. The address in the vector registers is determined by the CPU so that the CPU may direct the PPU to different routines depending on the state of the PPU.
The JVEC instruction has three choices of routines to which it can direct the PPU including a idle routine, a packet routine and an interrupt routine. The selection of the vector routine is on a priority basis so that, for example, if the interrupt vector has been updated since the last time a JVEC instruction was executed, then the interrupt routine is selected regardless of the status of the other vector registers. Thus, the interrupt vector has the highest priority. The second highest priority vector routine SUBSTITUTE SHEET (RULE 2B) is the packet routine which is executed if the header Il0 engine indicates that a new header is ready for processing and there is no interrupt vector. The lowest priority routine is the idle routine which is the default routine and will be executed only if neither of the other vector conditions have occurred.
An example of the JVEC instruction as used during normal data packet processing may be:
IDLE:
JVEC;
NOP;
PACKET:
//process packets here //packet processing completed SEND;
JVEC;
NOP;
In this example, after the PPU has completed the processing of a header portion and has issued a SEND instruction which places the header portion in the output FIFO, as described above, the JVEC instruction is executed. In this example, there is no header portion to be processed by the PPU so the vector registers have been loaded with the address of the IDLE routine and the PPU executes the instructions within the IDLE routine. In the IDLE routine, the PPU executes another JVEC instruction which returns the PPU to the IDLE routine as long as there are no header portions ready for processing and the interrupt vector has not been written since the last JVEC
instruction.
When a header portion is ready for processing, the sequence engine is alerted and SUBSTITUTE SHEET (RULE 26) chooses the address in the packet vector register and the PPU will begin executing the instructions in the PACKET routine the next time that it executes the JVEC
instruction.
In addition, as conditions in the packet processing system change, the CPU may change the address in the vector registers to direct the PPU to another routine. For example, if the CPU determines that the packet processing system is low on buffer space, it may direct the PPU to execute a different packet routine which implements the selective discarding of data packets until more buffer space is available.
In addition, the use of the JVEC instruction may permit the CPU to update the packet processing code. To update the packet processing code, the CPU steers the PPU
to special routine where it will no longer process packets. The PPU, through the mailbox registers, will indicate that it is currently executing this special routine so that the CPU may halt the PPU and update the instruction memory with the desired changes. Once the code has been updated, the vector registers are re-initialized and the PPU is unhalted. The update method prevents the PPU and the packet processing system from losing header data and those buffer memory regions associated with a packet because the PPU was halted only after the current header being processed has been completed. Now, other components associated with the PPU will be described.
The set of general registers may store a variety of general information used by the ALU 164. The mailbox registers 158 permit the ALU to communicate with the CPU 142. The flat table look-up registers permit the ALU to request that a flat table look-up search, as described below, be performed by a table engine 172 shown in Figure 5 and the results of the flat table search be stored in the flat table registers 160 SUBSTITUTE SHEET (RULE 26) so that the ALU may access the results. The route table look-up registers 162 permit the ALU to request a route table look-up search from a route table search engine 170 shown in Figure 5, as described below, and the results of the route table search are stored in the route table registers 162. The ALU may then retrieve the results of either a route table search or a flat table search from the appropriate registers. In accordance with the invention, the flat table and route table searches may be performed simultaneously by the respective table engines and do not require any processing time of the ALU, as described below. In accordance with the invention, the data being accessed by the ALU and the search engines are contained in hardware registers since these are more rapidly accessible than conventional memories. This has the result of increasing the speed with which a header may be processed by the PPU.
The processing of the header portion will now be briefly described. It will be described in more detail below with reference to Figures 6A and 6B. Once the header portion has been loaded into one of the header registers 152, 154, the ALU
accesses a portion of the header register, as described below, and requests that a flat table search be conducted by the table search engine 172 using a portion of the header by writing an instruction into the flat table registers 160. The ALU then reads a second portion of the header and requests that a route table search be performed by the route table search engine 170 based on the portion of the header by writing an instruction into the route table search registers 162. While the flat table and route table searches are being performed by the respective search engines, the ALU may perform additional processing on the header such as verifying the checksum contained in the header, SUBSTITUTE SHEET (RULE 28) decrementing a Time To Live (TTL) value in a portion of the header and recalculating a checksum value. Once the searches have been completed and the results are returned to the registers, the ALU may generate an internal header which includes the results of the flat table search and the route table search and attach the internal header to the existing header stored in one of the header registers 152, 154. The modified header portion is then reunited with the data portion and output to the switch fabric, as described above. Additional details about the structure of the PPU will now be described.
Figure 5 is a block diagram illustrating the details of the packet processing unit 116 in accordance with the invention. In addition to the ALU 164 and the various registers described above, the PPU may also include a route table search engine 170, a flat table engine 172, a program counter 174, a sequence engine 176 and a CPU
interface 179 that permits the CPU 142 shown in Figure 4 to access some of the PPU's registers and each of the PPU's associated memories. The various engines are tightly coupled together which further increases the processing speed of the PPU. To tightly couple these components together, these components may reside on the same semiconductor chip. The CPU may control some of these functional blocks. The route table search engine 170 performs a route table look-up search when an instruction is placed in the route table registers 162. The flat table look-up search engine 172 may perform a flat table search when an instruction is placed in the flat table registers 160.
The program counter 174 keeps track of the address of the microcode being executed and the sequence engine 176 fetches and decodes the next microcode instruction to be SUBSTITUTE SHEET (RULE 26) executed by the ALU 164. The sequence engine also has the ability to execute conditional branches based on the results of the ALU.
To route the data between these units within the PPU, a plurality of multiplexers 178 are used. In addition, a header input/output (I/O) engine 247 is provided which controls the flow of headers into and out of the PPU. To facilitate the movement of the headers within the PPU, the header I/O engine 247 may control an input buffer 182, an output buffer 184, a dump buffer 186, a header exception buffer 180 and a CPU header buffer 181. The header I/O engine controls header movement between the sets of header registers 152, 154 and the input buffer, the output buffer, the dump buffer and the header exception buffer. The header I/O engine also controls header movement between the CPU header buffer and the input buffer, the output buffer, the dump buffer and the header exception buffer. The header I/O engine may control the input buffer 182 that receives unprocessed headers entering the PPU and the output buffer 184 for sending headers that have been processed to the output header buffer of the packet processing system. In addition, the header I/O engine may also control the dump buffer 186 which holds descriptors for data packets that are going to be dropped by the packet processing system because, for example, the header contains corrupt information. The dump buffer may be directly connected to the DMA 52 shown in Figure 4 so that the data portion that corresponds to the dumped header portion may be discarded and the buffer space occupied by the data portion in the buffer memory 50 shown in Figure 4 may be immediately reused. The header I/O
engine may also control a header exception buffer for data packets that can be SUBSTITUTE SHEET (RULE 26) WO 99/18b99 PCT/US98/20795 processed more efficiently by the CPU. The header I/O engine may control a CPU
header buffer for headers that were processed by the CPU. In this manner, the header I/O engine has a plurality of different options for routing an incoming or outgoing header portion from the PPU. Now, a method for processing a header portion of a data packet in accordance with the invention will be described with reference to Figures 6A, 6B, 7, and 8.
Figures 6A and 6B illustrate a flowchart detailing the steps of a method 200 for processing a header portion of a data packet in accordance with the invention.
Figures 7 and 8 are diagrams which illustrate an example of the contents of the header registers prior to processing by the PPU and after processing by the PPU. Returning to Figure 6A, the method begins at step 202 in which the next header portion to be processed by the ALU is stored in one of the set of header registers and is ready for processing. As shown in Figure 7, in this example, the first set of header registers 152 has a total of thirty-two registers and the header portion of the data packet starts at header register 9 (HD9) and fills the rest of the registers. In this example, a frame relay data packet header is shown which may include the frame relay header 204, an IP packet header 206, payload data 208, a DMA descriptor 210 and a media descriptor 212. A data packet from another type of media may have a slightly different header fomlat and may be stored starting at a different individual header register. The PPU in accordance with the invention may process a plurality of different types of data packets transmitted over a plurality of different media, such as FDDI or Ethemet or frame relay and the header portions for each of these data packets is slightly different, but can also be processed by SUBSTITUTE SHEET (RULE 26) the PPU. The first nine registers (HDO - HD 8) 214 are used to store the internal header generated by the PPU, as described below.
Once the header portion has been received, the PPU extracts the frame relay header 204 in step 216, and requests a flat table search to be conducted to retrieve an entry for this frame relay circuit in a circuit table contained in the table memory. Step 216 may be executed for fi~ame relay or ATM data packets, but is not executed for other types of data packets. Next, in step 218, a second flat table search is conducted into the interface table using the inbound interface number contained in the header portion. Next, in step 220, while the other flat table search is being performed, the PPU requests that a route table search is conducted using the destination IP
address contained in HD14 in this example. While these searches are being conducted by the appropriate search engines, the PPU may, in step 222, perform additional header processing and header information checks that do not require the results of the searches, such as verifying the checksum in the header, decrernenting the Time To Live (TTL) value and recalculating the checksum. Once these checks have been performed, the PPU determines if the interface table look-up has been completed in step 224 and waits, if it has not been completed. When the results of the interface table have been received, the PPU ensures that the appropriate inbound interface has been correctly configured, increments the interface statistics and writes the record back into the table memory in step 226. In step 228, the PPU determines if the route table search has been completed and waits until it has been completed. Once the results of the route table search are available, the next hop address is placed in the route table registers in step SUBSTITUTE SHEET (RULE 2B) 230 by the route table search engine. In step 232, assuming the next hop address is correct, the PPU generates an internal header in HDO - HD9 214 using the next hop address information. A modified header that contains the internal header is shown in Figure 8. Once the header has been modified to include the internal header, the PPU
issues a SEND command in step 234. The header register sets are then switched, the processed header is unloaded, and the modified header is reunited with the data portion by the DMA and transmitted across the switch fabric. After transmission, the storage for the data portion in the buffer is released. The method loops back to step immediately upon the switching of the header register sets and begins processing another header portion. As described above, due to the architecture of the PPU, only a single clock cycle elapses between the end of a header processing and the start of the next header processing. In addition, the PPU performs a plurality of operations in parallel, such as header loading and/or unloading, the flat table and route table searches, and other header processing, which substantially reduces the time required to process a header. Now, a method for processing multicast data packets in accordance with the invention will be described.
Figure 9 is a flowchart illustrating a method 240 for processing the header of a multicast data packet in accordance with the invention. In step 242, the PPU
determines, from data in the header portion, whether the data packet is a multicast data packet (e.g., a data packet destined for more than one destination). If the data packet is not a multicast data packet, normal header processing as described above occurs. If a multicast data packet is encountered, the header is processed, as described above, in SUBSTITUTE SHEET (RULE 26) step 244. Once the data packet has been processed, a SENDM command is issued by the PPU in step 246. The SENDM command differs from a SEND command in that the SENDM command does not release the processed header registers. Instead, the processed header portion is returned to the working register set after a copy of it has been unloaded. In this manner, the SENDM instruction duplicates headers. Each header generated will point to the same buffer space occupied by the data portion.
Once the SENDM command is issued, the header I/O engine 247, as shown in Figure S, copies the processed header to the output buffer 184 and then returns the same header to the working set of header registers. Once the header has been returned to the working header registers, the PPU modifies the internal header for the next address in step 250. Next, in step 252, the PPU determines if the current address is the last address for the multicast data packet. If there are no additional addresses for the multicast data packet, then, in step 254, the PPU issues a SEND command which routes the modified header to the packet processing system and allows the removal of the header portion from the I/O header registers. If there is an additional address for the multicast data packet, the method loops back to step 246 in which the SENDM
command is issued. Thus, in accordance with the invention, a multicast data packet may be rapidly communicated to each of the destination addresses without needing a separate data portion for each destination which substantially reduces the amount of buffer memory required to store a multicast data packet.
While the foregoing has been with reference to a particular embodiment of the invention, it wi 11 be appreciated by those skilled in the art that changes in this SUBSTITUTE SHEET (RULE 26) embodiment may be made without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims.
SUBSTITUTE SHEET (RULE 26)
Claims (21)
1. An apparatus for processing a data packet to determine the routing of the data packet through a communications network, the data packet having a header portion and a data portion, the apparatus comprising:
means for storing the header portion of the data packet;
means, connected to the header storing means, for processing the header portion of the data packet;
said processing means comprising a processing core for executing instructions for processing the header portion, means, responsive to an instruction from the processing core, for searching through a route table to determine a route of the data packet, means, responsive to an instruction from the processing core, for searching through a table memory for information about the data packet, said processing core, said route table search means and said table memory search means operating simultaneously to process said header portion and generate one of an internal header and a network media header; and means for generating a modified header portion containing one of the internal header and the network media header to route the data packet through the communications network.
means for storing the header portion of the data packet;
means, connected to the header storing means, for processing the header portion of the data packet;
said processing means comprising a processing core for executing instructions for processing the header portion, means, responsive to an instruction from the processing core, for searching through a route table to determine a route of the data packet, means, responsive to an instruction from the processing core, for searching through a table memory for information about the data packet, said processing core, said route table search means and said table memory search means operating simultaneously to process said header portion and generate one of an internal header and a network media header; and means for generating a modified header portion containing one of the internal header and the network media header to route the data packet through the communications network.
2. The apparatus of Claim 1, wherein said header storing means comprises a first set of header registers and a second set of header registers so that while the processing means is processing the header contained in one of the set of header registers a new header portion may be stored in the other set of header registers.
3. The apparatus of Claim 2, wherein said header storing means further comprises means for switching between said set of header registers.
4. The apparatus of Claim 3, wherein said header storing means further comprises means for unloading a header portion from the first set of registers that has been processed by said processing core and loading a header portion into the first set of registers that has not been processed while the processing means is processing a header contained in the second set of registers.
5. The apparatus of Claim 1, wherein said route table search means comprises a plurality of registers for communications between said processing core and said route table search means.
6. The apparatus of Claim 1, wherein said table memory search means comprises a plurality of registers for communications between said processing core and said table memory search means.
7. The apparatus of Claim 1 further comprising a central processor interface for controlling said processing core, said header storing means, said route table search means and said table memory search means and means for communicating between said central processor interface and said processing core, said header storing means, said route table search means and said table memory search means.
8. The apparatus of Claim 7, wherein said communications means comprises a set of registers.
9. The apparatus of Claim 1, wherein said processing core comprises an arithmetic logic unit and a sequence engine for executing instructions to process the header portion.
10. The apparatus of Claim 2, wherein said header storing means further comprises means for storing information about a data packet that is being discarded so that the space in the header storing means occupied by the discarded packet and space in a data store are immediately reused.
11. The apparatus of Claim 1, wherein said processing core comprises means for executing predetermined instructions that facilitate the processing of data packets.
12. The apparatus of Claim 1 further comprising a central processing unit for controlling the processing core, the central processing unit comprising means for steering the processing core to one of a predetermined set of routines.
13. A method for processing a data packet to determine the routing of the data packet through a communications network, the data packet having a header portion and a data portion, the method comprising:
storing the header portion of the data packet;
processing the header portion of the data packet;
said processing comprising executing instructions for processing the header portion, searching through a route table to determine a route of the data packet, searching through a table memory for information about the destination of the data packet, said route table search and said table memory search operating simultaneously to process said header portion and generate one of an internal header and a network media header; and generating a modified header portion containing one of the internal header and the network media header to route the data packet through the communications network.
storing the header portion of the data packet;
processing the header portion of the data packet;
said processing comprising executing instructions for processing the header portion, searching through a route table to determine a route of the data packet, searching through a table memory for information about the destination of the data packet, said route table search and said table memory search operating simultaneously to process said header portion and generate one of an internal header and a network media header; and generating a modified header portion containing one of the internal header and the network media header to route the data packet through the communications network.
14. The method of Claim 13, wherein said header storing comprises using a first set of header registers and a second set of header registers so that while processing the header contained in one of the set of header registers a new header portion is stored in the other set of header registers.
15. The method of Claim 14, wherein said header storing further comprises switching between said set of header registers.
16. The method of Claim 15, wherein said header storing further comprises unloading a header portion that has been processed by said processing core from the first set of registers and loading an unprocessed header portion into the first set of registers while the processing core processes a header in the second set of registers.
17. The method of Claim 15, wherein said header storing further comprises storing a information about a data packet that is being discarded so that the space in the header storing means occupied by the discarded packet and space in a data store are immediately reused.
18. The method of Claim 13 further comprising controlling the processing core using a central processing unit, the controlling comprises steering the processing core to one of a predetermined set of routines.
19. A method for processing data packets to determine the route of the data packet, the data packet having a header portion and a data portion, the method comprising:
receiving a header portion of an incoming data packet;
performing a search based on the received header portion;
performing a route look-up search simultaneously with said search;
checking information contained within the header portion while said search and said route look-up search are being performed for determining if the header portion is valid;
generating one of an internal header and a network media header based on the results of the route look-up search and the search; and generating a modified header portion including the received header portion and one of the internal header and the network media header.
receiving a header portion of an incoming data packet;
performing a search based on the received header portion;
performing a route look-up search simultaneously with said search;
checking information contained within the header portion while said search and said route look-up search are being performed for determining if the header portion is valid;
generating one of an internal header and a network media header based on the results of the route look-up search and the search; and generating a modified header portion including the received header portion and one of the internal header and the network media header.
20. The method of Claim 19, wherein checking information in the header portion further comprises verifying a checksum value contained in the received header portion, decrementing a time-to-live counter contained with the header portion and generating a new checksum value to be inserted into the header portion.
21. The method of Claim 19, further comprising processing a multicast data packet destined for a plurality of addresses comprising generating a header portion for each destination address, outputting a header and the same data portion for each destination address, and deleting the data portion from a buffer memory when the processing of the multicast data packet is completed.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US08/943,512 US6032190A (en) | 1997-10-03 | 1997-10-03 | System and method for processing data packets |
| US08/943,512 | 1997-10-03 | ||
| PCT/US1998/020795 WO1999018699A1 (en) | 1997-10-03 | 1998-10-02 | System and method for processing data packets |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2305587A1 true CA2305587A1 (en) | 1999-04-15 |
Family
ID=25479796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002305587A Abandoned CA2305587A1 (en) | 1997-10-03 | 1998-10-02 | System and method for processing data packets |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US6032190A (en) |
| EP (1) | EP1031216A1 (en) |
| AU (1) | AU9783398A (en) |
| CA (1) | CA2305587A1 (en) |
| WO (1) | WO1999018699A1 (en) |
Families Citing this family (142)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6791947B2 (en) | 1996-12-16 | 2004-09-14 | Juniper Networks | In-line packet processing |
| US6266706B1 (en) * | 1997-09-15 | 2001-07-24 | Effnet Group Ab | Fast routing lookup system using complete prefix tree, bit vector, and pointers in a routing table for determining where to route IP datagrams |
| US6415315B1 (en) * | 1997-12-01 | 2002-07-02 | Recursion Software, Inc. | Method of moving objects in a computer network |
| US6519653B1 (en) * | 1997-12-01 | 2003-02-11 | Recursion Software, Inc. | Method of communicating between agent objects in a computer network |
| US6442586B1 (en) * | 1997-12-01 | 2002-08-27 | Recursion Software, Inc. | Method of moving objects across multiple locations in a computer network |
| US6549519B1 (en) | 1998-01-23 | 2003-04-15 | Alcatel Internetworking (Pe), Inc. | Network switching device with pipelined search engines |
| US6161144A (en) * | 1998-01-23 | 2000-12-12 | Alcatel Internetworking (Pe), Inc. | Network switching device with concurrent key lookups |
| US6401171B1 (en) * | 1998-02-27 | 2002-06-04 | Cisco Technology, Inc. | Method and device for storing an IP header in a cache memory of a network node |
| JPH11261649A (en) * | 1998-03-12 | 1999-09-24 | Hitachi Ltd | Data processor and router bridge adopting the same |
| US6392996B1 (en) * | 1998-04-13 | 2002-05-21 | At&T Corp. | Method and apparatus for frame peeking |
| US6275508B1 (en) * | 1998-04-21 | 2001-08-14 | Nexabit Networks, Llc | Method of and system for processing datagram headers for high speed computer network interfaces at low clock speeds, utilizing scalable algorithms for performing such network header adaptation (SAPNA) |
| US6185635B1 (en) * | 1998-05-30 | 2001-02-06 | Alcatel Networks Corporation | Method and circuit for transporting data based on the content of ingress data words and egress data words |
| US7158532B2 (en) * | 1998-07-06 | 2007-01-02 | Intel Corporation | Half duplex link with isochronous and asynchronous arbitration |
| US6920146B1 (en) | 1998-10-05 | 2005-07-19 | Packet Engines Incorporated | Switching device with multistage queuing scheme |
| US6678269B1 (en) | 1998-10-05 | 2004-01-13 | Alcatel | Network switching device with disparate database formats |
| JP3735471B2 (en) * | 1998-10-05 | 2006-01-18 | 株式会社日立製作所 | Packet relay device and LSI |
| US6385661B1 (en) * | 1998-10-19 | 2002-05-07 | Recursion Software, Inc. | System and method for dynamic generation of remote proxies |
| US6425021B1 (en) | 1998-11-16 | 2002-07-23 | Lsi Logic Corporation | System for transferring data packets of different context utilizing single interface and concurrently processing data packets of different contexts |
| US6359911B1 (en) * | 1998-12-04 | 2002-03-19 | Koninklijke Philips Electronics N.V. (Kpenv) | MPEG-2 transport demultiplexor architecture with non-time-critical post-processing of packet information |
| US6434170B1 (en) * | 1998-12-04 | 2002-08-13 | Koninklijke Philips Electronics N.V. | Memory-based circular queue with local descriptive information to implement a storage area for filtering mpeg-2 packets that are distributed and/or processed under the control of a host microprocessor by a direct memory access mechanism |
| US6463059B1 (en) * | 1998-12-04 | 2002-10-08 | Koninklijke Philips Electronics N.V. | Direct memory access execution engine with indirect addressing of circular queues in addition to direct memory addressing |
| US6308220B1 (en) * | 1999-01-29 | 2001-10-23 | Neomagic Corp. | Circulating parallel-search engine with random inputs for network routing table stored in a wide embedded DRAM |
| US6363428B1 (en) * | 1999-02-01 | 2002-03-26 | Sony Corporation | Apparatus for and method of separating header information from data in an IEEE 1394-1995 serial bus network |
| US6367026B1 (en) | 1999-02-01 | 2002-04-02 | Sony Corporation | Unbalanced clock tree for a digital interface between an IEEE 1394 serial bus system and a personal computer interface (PCI) |
| JP4182180B2 (en) * | 1999-02-24 | 2008-11-19 | 株式会社日立製作所 | Network relay device and network relay method |
| US6683885B1 (en) * | 1999-02-24 | 2004-01-27 | Hitachi, Ltd. | Network relaying apparatus and network relaying method |
| JP3645734B2 (en) * | 1999-02-24 | 2005-05-11 | 株式会社日立製作所 | Network relay device and network relay method |
| US6366961B1 (en) * | 1999-03-03 | 2002-04-02 | Nokia Telecommunications, Oy | Method and apparatus for providing mini packet switching in IP based cellular access networks |
| US6708233B1 (en) * | 1999-03-25 | 2004-03-16 | Microsoft Corporation | Method and apparatus for direct buffering of a stream of variable-length data |
| US6643705B1 (en) * | 1999-03-29 | 2003-11-04 | Microsoft Corporation | Routing of electronic messages using a routing map and a stateful script engine |
| JP2000295274A (en) * | 1999-04-05 | 2000-10-20 | Nec Corp | Packet exchange |
| US6574231B1 (en) * | 1999-05-21 | 2003-06-03 | Advanced Micro Devices, Inc. | Method and apparatus for queuing data frames in a network switch port |
| US6650660B1 (en) * | 1999-07-27 | 2003-11-18 | Pluris, Inc. | Apparatus and method for synchronization of multiple data paths and recovery from lost synchronization |
| US6449656B1 (en) * | 1999-07-30 | 2002-09-10 | Intel Corporation | Storing a frame header |
| US6983350B1 (en) * | 1999-08-31 | 2006-01-03 | Intel Corporation | SDRAM controller for parallel processor architecture |
| US6606704B1 (en) * | 1999-08-31 | 2003-08-12 | Intel Corporation | Parallel multithreaded processor with plural microengines executing multiple threads each microengine having loadable microcode |
| US7167443B1 (en) * | 1999-09-10 | 2007-01-23 | Alcatel | System and method for packet level restoration of IP traffic using overhead signaling in a fiber optic ring network |
| US6678743B1 (en) | 1999-11-30 | 2004-01-13 | Recursion Software, Inc. | Method for moving objects in a distributed computing environment |
| US6947965B2 (en) | 1999-11-30 | 2005-09-20 | Recursion Software, Inc. | System and method for communications in a distributed computing environment |
| US6532509B1 (en) | 1999-12-22 | 2003-03-11 | Intel Corporation | Arbitrating command requests in a parallel multi-threaded processing system |
| JP2001186142A (en) * | 1999-12-27 | 2001-07-06 | Toshiba Corp | Packet buffer device and packet switching device |
| US6694380B1 (en) | 1999-12-27 | 2004-02-17 | Intel Corporation | Mapping requests from a processing unit that uses memory-mapped input-output space |
| US6661794B1 (en) * | 1999-12-29 | 2003-12-09 | Intel Corporation | Method and apparatus for gigabit packet assignment for multithreaded packet processing |
| US6952824B1 (en) * | 1999-12-30 | 2005-10-04 | Intel Corporation | Multi-threaded sequenced receive for fast network port stream of packets |
| US6810031B1 (en) | 2000-02-29 | 2004-10-26 | Celox Networks, Inc. | Method and device for distributing bandwidth |
| WO2001067237A2 (en) * | 2000-03-03 | 2001-09-13 | Tenor Networks, Inc. | High-speed data processing using internal processor memory space |
| DE10011667C2 (en) * | 2000-03-10 | 2002-11-21 | Infineon Technologies Ag | High Speed Router |
| US6965603B1 (en) | 2000-03-20 | 2005-11-15 | Cortina Systems, Inc. | Circuits for combining ATM and packet data on an optical fiber |
| US6947430B2 (en) * | 2000-03-24 | 2005-09-20 | International Business Machines Corporation | Network adapter with embedded deep packet processing |
| US7050437B2 (en) * | 2000-03-24 | 2006-05-23 | International Business Machines Corporation | Wire speed reassembly of data frames |
| US6751224B1 (en) | 2000-03-30 | 2004-06-15 | Azanda Network Devices, Inc. | Integrated ATM/packet segmentation-and-reassembly engine for handling both packet and ATM input data and for outputting both ATM and packet data |
| US6810039B1 (en) | 2000-03-30 | 2004-10-26 | Azanda Network Devices, Inc. | Processor-based architecture for facilitating integrated data transfer between both atm and packet traffic with a packet bus or packet link, including bidirectional atm-to-packet functionally for atm traffic |
| US6751214B1 (en) | 2000-03-30 | 2004-06-15 | Azanda Network Devices, Inc. | Methods and apparatus for dynamically allocating bandwidth between ATM cells and packets |
| US7146422B1 (en) | 2000-05-01 | 2006-12-05 | Intel Corporation | Method and apparatus for validating documents based on a validation template |
| US6732175B1 (en) | 2000-04-13 | 2004-05-04 | Intel Corporation | Network apparatus for switching based on content of application data |
| US7333489B1 (en) * | 2000-05-08 | 2008-02-19 | Crossroads Systems, Inc. | System and method for storing frame header data |
| JP4265087B2 (en) * | 2000-06-29 | 2009-05-20 | ソニー株式会社 | Data conversion apparatus and method, data transmission / reception apparatus and method, and network system |
| US6944162B1 (en) | 2000-10-03 | 2005-09-13 | Alcatel | Tuple-based lookup scheme for packet switching node |
| US6816489B1 (en) * | 2000-10-05 | 2004-11-09 | Cisco Technology, Inc. | Efficient method for packet switching on asynchronous transfer mode switch based platforms |
| US9639553B2 (en) * | 2000-11-02 | 2017-05-02 | Oracle International Corporation | TCP/UDP acceleration |
| US7089293B2 (en) | 2000-11-02 | 2006-08-08 | Sun Microsystems, Inc. | Switching system method for discovering and accessing SCSI devices in response to query |
| US7865596B2 (en) * | 2000-11-02 | 2011-01-04 | Oracle America, Inc. | Switching system for managing storage in digital networks |
| US7313614B2 (en) * | 2000-11-02 | 2007-12-25 | Sun Microsystems, Inc. | Switching system |
| US6985956B2 (en) * | 2000-11-02 | 2006-01-10 | Sun Microsystems, Inc. | Switching system |
| US20030128987A1 (en) * | 2000-11-08 | 2003-07-10 | Yaron Mayer | System and method for improving the efficiency of routers on the internet and/or cellular networks an/or other networks and alleviating bottlenecks and overloads on the network |
| WO2002039208A2 (en) * | 2000-11-08 | 2002-05-16 | Yaron Mayer | System and method for improving network router efficiency and alleviating network bottlenecks and overloads |
| WO2003060624A2 (en) * | 2001-12-28 | 2003-07-24 | Yaron Mayer | System and method for improving network router efficiency and alleviating network bottlenecks and overloads |
| KR100501080B1 (en) * | 2000-12-19 | 2005-07-18 | 노병희 | A method and system for distinguishing higher layer protocols of the internet traffic |
| US6788896B1 (en) * | 2000-12-26 | 2004-09-07 | Guo-Qiang Wang | Technique for all-optical packet switching |
| WO2002062021A1 (en) * | 2001-01-31 | 2002-08-08 | International Business Machines Corporation | Providing control information to a management processor of a communications switch |
| US6658546B2 (en) * | 2001-02-23 | 2003-12-02 | International Business Machines Corporation | Storing frame modification information in a bank in memory |
| US7039060B2 (en) * | 2001-03-07 | 2006-05-02 | Mips Tech Inc | System and method for extracting fields from packets having fields spread over more than one register |
| JP4822598B2 (en) * | 2001-03-21 | 2011-11-24 | ルネサスエレクトロニクス株式会社 | Cache memory device and data processing device including the same |
| US20030012181A1 (en) * | 2001-03-30 | 2003-01-16 | Andre The | Simulating high-speed access on a low-bandwidth network connection |
| US6904057B2 (en) * | 2001-05-04 | 2005-06-07 | Slt Logic Llc | Method and apparatus for providing multi-protocol, multi-stage, real-time frame classification |
| US7042848B2 (en) * | 2001-05-04 | 2006-05-09 | Slt Logic Llc | System and method for hierarchical policing of flows and subflows of a data stream |
| US6901052B2 (en) * | 2001-05-04 | 2005-05-31 | Slt Logic Llc | System and method for policing multiple data flows and multi-protocol data flows |
| US6944168B2 (en) * | 2001-05-04 | 2005-09-13 | Slt Logic Llc | System and method for providing transformation of multi-protocol packets in a data stream |
| US6937607B2 (en) * | 2001-06-21 | 2005-08-30 | Alcatel | Random early discard for cell-switched data switch |
| US7027446B2 (en) * | 2001-07-18 | 2006-04-11 | P-Cube Ltd. | Method and apparatus for set intersection rule matching |
| US6606322B2 (en) * | 2001-08-17 | 2003-08-12 | Mcdata Corporation | Route lookup caching for a fiber channel switch |
| JP2003067321A (en) * | 2001-08-28 | 2003-03-07 | Hitachi Ltd | Data transfer device and aligner provided inside data transfer device |
| US7126952B2 (en) * | 2001-09-28 | 2006-10-24 | Intel Corporation | Multiprotocol decapsulation/encapsulation control structure and packet protocol conversion method |
| US7958199B2 (en) * | 2001-11-02 | 2011-06-07 | Oracle America, Inc. | Switching systems and methods for storage management in digital networks |
| US7389360B1 (en) | 2001-11-05 | 2008-06-17 | Juniper Networks, Inc. | Context switched route lookup key engine |
| US7113518B2 (en) * | 2001-12-19 | 2006-09-26 | Agere Systems Inc. | Processor with reduced memory requirements for high-speed routing and switching of packets |
| US7254138B2 (en) * | 2002-02-11 | 2007-08-07 | Optimum Communications Services, Inc. | Transparent, look-up-free packet forwarding method for optimizing global network throughput based on real-time route status |
| US6934787B2 (en) * | 2002-02-22 | 2005-08-23 | Broadcom Corporation | Adaptable switch architecture that is independent of media types |
| US7130920B1 (en) * | 2002-02-27 | 2006-10-31 | Pmc-Sierra, Inc. | Load balancing multicast connection scheduling in 3-stage TDM fabrics |
| US7254632B2 (en) * | 2002-04-26 | 2007-08-07 | P-Cube Ltd. | Apparatus and method for pattern matching in text based protocol |
| US7161904B2 (en) * | 2002-06-04 | 2007-01-09 | Fortinet, Inc. | System and method for hierarchical metering in a virtual router based network switch |
| US7471688B2 (en) * | 2002-06-18 | 2008-12-30 | Intel Corporation | Scheduling system for transmission of cells to ATM virtual circuits and DSL ports |
| US7404015B2 (en) * | 2002-08-24 | 2008-07-22 | Cisco Technology, Inc. | Methods and apparatus for processing packets including accessing one or more resources shared among processing engines |
| AU2003252054B2 (en) * | 2002-08-24 | 2008-09-25 | Cisco Technology, Inc. | Packet processing engine |
| US7304999B2 (en) * | 2002-08-24 | 2007-12-04 | Cisco Technology Inc. | Methods and apparatus for processing packets including distributing packets across multiple packet processing engines and gathering the processed packets from the processing engines |
| KR100474302B1 (en) * | 2002-09-07 | 2005-03-10 | 엘지전자 주식회사 | Buffer control method of radio link control layer |
| US7433307B2 (en) * | 2002-11-05 | 2008-10-07 | Intel Corporation | Flow control in a network environment |
| US7983239B1 (en) | 2003-01-07 | 2011-07-19 | Raytheon Bbn Technologies Corp. | Systems and methods for constructing a virtual model of a multi-hop, multi-access network |
| JP4257151B2 (en) * | 2003-02-28 | 2009-04-22 | 富士通株式会社 | Packet control system, packet control device, packet relay device, and packet control program |
| US7835397B2 (en) * | 2003-04-25 | 2010-11-16 | Alcatel-Lucent Usa Inc. | Frame processing |
| US7937495B2 (en) * | 2003-06-25 | 2011-05-03 | Cisco Technology, Inc. | System and method for modifying data transferred from a source to a destination |
| DE602004021807D1 (en) * | 2003-07-10 | 2009-08-13 | Computer Ass Think Inc | SYSTEM AND METHOD FOR BUFFING DATA OF VARIABLE LENGTH |
| US7881229B2 (en) * | 2003-08-08 | 2011-02-01 | Raytheon Bbn Technologies Corp. | Systems and methods for forming an adjacency graph for exchanging network routing data |
| US7606927B2 (en) * | 2003-08-27 | 2009-10-20 | Bbn Technologies Corp | Systems and methods for forwarding data units in a communications network |
| US7515584B2 (en) * | 2003-09-19 | 2009-04-07 | Infineon Technologies Ag | Switching data packets in an ethernet switch |
| US7460531B2 (en) * | 2003-10-27 | 2008-12-02 | Intel Corporation | Method, system, and program for constructing a packet |
| US7668083B1 (en) | 2003-10-28 | 2010-02-23 | Bbn Technologies Corp. | Systems and methods for forwarding data in a communications network |
| US7369512B1 (en) | 2003-11-06 | 2008-05-06 | Bbn Technologies Corp. | Systems and methods for efficient packet distribution in an ad hoc network |
| US7509373B2 (en) * | 2003-11-24 | 2009-03-24 | At&T Intellectual Property I, L.P. | Methods for providing communications services |
| US20050114224A1 (en) * | 2003-11-24 | 2005-05-26 | Hodges Donna K. | Methods for providing communications services |
| US7467219B2 (en) * | 2003-11-24 | 2008-12-16 | At&T Intellectual Property I, L.P. | Methods for providing communications services |
| US7562363B1 (en) | 2003-11-25 | 2009-07-14 | Cisco Technology, Inc. | Gang scheduling among one or more components or systems |
| JP4238720B2 (en) * | 2003-12-19 | 2009-03-18 | ソニー株式会社 | Image signal processing apparatus and processing method, image signal processing board, image signal processing system, program, and medium recording the same |
| US7480308B1 (en) | 2004-03-29 | 2009-01-20 | Cisco Technology, Inc. | Distributing packets and packets fragments possibly received out of sequence into an expandable set of queues of particular use in packet resequencing and reassembly |
| US7406080B2 (en) * | 2004-06-15 | 2008-07-29 | International Business Machines Corporation | Method and structure for enqueuing data packets for processing |
| US9160649B2 (en) * | 2004-09-27 | 2015-10-13 | Alcatel Lucent | Method for routing traffic using traffic weighting factors |
| US20060078127A1 (en) * | 2004-10-08 | 2006-04-13 | Philip Cacayorin | Dispersed data storage using cryptographic scrambling |
| US7869453B2 (en) * | 2004-12-17 | 2011-01-11 | Lantiq Deutschland Gmbh | Apparatus and method for data transfer |
| US7551617B2 (en) | 2005-02-08 | 2009-06-23 | Cisco Technology, Inc. | Multi-threaded packet processing architecture with global packet memory, packet recirculation, and coprocessor |
| US7561589B2 (en) * | 2005-02-23 | 2009-07-14 | Cisco Technology, Inc | Virtual address storage which may be of particular use in generating fragmented packets |
| US7561573B2 (en) * | 2005-03-23 | 2009-07-14 | Fujitsu Limited | Network adaptor, communication system and communication method |
| US7606250B2 (en) | 2005-04-05 | 2009-10-20 | Cisco Technology, Inc. | Assigning resources to items such as processing contexts for processing packets |
| US20060268913A1 (en) * | 2005-05-27 | 2006-11-30 | Utstarcom, Inc. | Streaming buffer system for variable sized data packets |
| US7965708B2 (en) * | 2005-06-07 | 2011-06-21 | Cisco Technology, Inc. | Method and apparatus for using meta-packets in a packet processing system |
| US7739426B1 (en) | 2005-10-31 | 2010-06-15 | Cisco Technology, Inc. | Descriptor transfer logic |
| US7852843B2 (en) * | 2006-07-21 | 2010-12-14 | Cortina Systems, Inc. | Apparatus and method for layer-2 to layer-7 search engine for high speed network application |
| US8223788B1 (en) * | 2007-10-24 | 2012-07-17 | Ethernity Networks Ltd | Method and system for queuing descriptors |
| US8040888B1 (en) | 2007-12-17 | 2011-10-18 | Integrated Device Technology, Inc. | Packet switch with port route tables |
| US7843919B2 (en) * | 2008-03-20 | 2010-11-30 | International Business Machines Corporation | Ethernet virtualization using a network packet alteration |
| TWI356304B (en) * | 2008-04-21 | 2012-01-11 | Ralink Technology Corp | Network device of processing packets efficiently a |
| US8139504B2 (en) * | 2009-04-07 | 2012-03-20 | Raytheon Bbn Technologies Corp. | System, device, and method for unifying differently-routed networks using virtual topology representations |
| US8924767B2 (en) * | 2009-12-17 | 2014-12-30 | Texas Instruments Incorporated | Minimizing the use of chip routing resources when using timestamped instrumentation data by transmitting the most significant bits of the timestamp in series and transmitting the least significant bits of the timestamp in parallel |
| US9152423B2 (en) * | 2011-03-25 | 2015-10-06 | Avaya Inc. | Method and apparatus for efficient loop instruction execution using bit vector scanning |
| US9151255B2 (en) | 2011-09-23 | 2015-10-06 | Carter Fuel Systems, Llc | Marine fuel system with spill control feature |
| US9112708B1 (en) * | 2012-01-27 | 2015-08-18 | Marvell Israel (M.I.S.L) Ltd. | Processing multicast packets in a network device |
| US20140019570A1 (en) * | 2012-07-12 | 2014-01-16 | Raytheon Company | Data buffer exchange |
| US10963254B2 (en) * | 2013-07-15 | 2021-03-30 | Texas Instruments Incorporated | Mechanism to queue multiple streams to run on streaming engine |
| US9571541B1 (en) | 2013-10-03 | 2017-02-14 | Marvell Israel (M.I.S.L.) Ltd. | Network device architecture using cache for multicast packets |
| US9880784B2 (en) * | 2016-02-05 | 2018-01-30 | Knuedge Incorporated | Data routing and buffering in a processing system |
| DE102016110078A1 (en) * | 2016-06-01 | 2017-12-07 | Intel IP Corporation | Data processing apparatus and method for offloading data to a remote data processing apparatus |
| US10367758B2 (en) | 2016-09-12 | 2019-07-30 | Marvell World Trade Ltd. | Merging read requests in network device architecture |
| FR3079695B1 (en) * | 2018-04-03 | 2022-11-18 | Silkan Rt | DATA ANALYSIS AND FILTERING IN A DATA TRANSMISSION SYSTEM |
Family Cites Families (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5842224A (en) * | 1989-06-16 | 1998-11-24 | Fenner; Peter R. | Method and apparatus for source filtering data packets between networks of differing media |
| JPH04138739A (en) * | 1990-09-28 | 1992-05-13 | Toshiba Corp | Communication control system using asynchronous transmission mode network |
| US5398245A (en) * | 1991-10-04 | 1995-03-14 | Bay Networks, Inc. | Packet processing method and apparatus |
| EP0689748B1 (en) * | 1993-03-20 | 1998-09-16 | International Business Machines Corporation | Method and apparatus for extracting connection information from protocol headers |
| EP0676878A1 (en) * | 1994-04-07 | 1995-10-11 | International Business Machines Corporation | Efficient point to point and multi point routing mechanism for programmable packet switching nodes in high speed data transmission networks |
| US5623605A (en) * | 1994-08-29 | 1997-04-22 | Lucent Technologies Inc. | Methods and systems for interprocess communication and inter-network data transfer |
| US5598410A (en) * | 1994-12-29 | 1997-01-28 | Storage Technology Corporation | Method and apparatus for accelerated packet processing |
| US5541920A (en) * | 1995-06-15 | 1996-07-30 | Bay Networks, Inc. | Method and apparatus for a delayed replace mechanism for a streaming packet modification engine |
| US5841764A (en) * | 1995-10-30 | 1998-11-24 | Ericsson Inc. | Method and apparatus for permitting a radio to originate and receive data messages in a data communications network |
| US5757799A (en) * | 1996-01-16 | 1998-05-26 | The Boeing Company | High speed packet switch |
| US5781549A (en) * | 1996-02-23 | 1998-07-14 | Allied Telesyn International Corp. | Method and apparatus for switching data packets in a data network |
-
1997
- 1997-10-03 US US08/943,512 patent/US6032190A/en not_active Expired - Fee Related
-
1998
- 1998-10-02 CA CA002305587A patent/CA2305587A1/en not_active Abandoned
- 1998-10-02 AU AU97833/98A patent/AU9783398A/en not_active Abandoned
- 1998-10-02 EP EP98952038A patent/EP1031216A1/en not_active Withdrawn
- 1998-10-02 WO PCT/US1998/020795 patent/WO1999018699A1/en not_active Application Discontinuation
Also Published As
| Publication number | Publication date |
|---|---|
| EP1031216A1 (en) | 2000-08-30 |
| AU9783398A (en) | 1999-04-27 |
| US6032190A (en) | 2000-02-29 |
| WO1999018699A1 (en) | 1999-04-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US6032190A (en) | System and method for processing data packets | |
| US6683885B1 (en) | Network relaying apparatus and network relaying method | |
| US7167474B2 (en) | Network relaying apparatus and network relaying method capable of high-speed routing and packet transfer | |
| US6847645B1 (en) | Method and apparatus for controlling packet header buffer wrap around in a forwarding engine of an intermediate network node | |
| EP1226501B1 (en) | Network switch and components and method of operation | |
| EP1208447B1 (en) | Network processor, memory organization and methods | |
| EP1208676B1 (en) | Network switch using network processor | |
| US6831923B1 (en) | Pipelined multiple issue packet switch | |
| US7546399B2 (en) | Store and forward device utilizing cache to store status information for active queues | |
| US6769033B1 (en) | Network processor processing complex and methods | |
| US20050232303A1 (en) | Efficient packet processing pipeline device and method | |
| US5418781A (en) | Architecture for maintaining the sequence of packet cells transmitted over a multicast, cell-switched network | |
| US20040052254A1 (en) | Distributed lookup based on packet contents | |
| US6658003B1 (en) | Network relaying apparatus and network relaying method capable of high-speed flow detection | |
| EP1222517A1 (en) | Vlsi network processor and methods | |
| US7124196B2 (en) | Processing a network packet using queues | |
| US7174394B1 (en) | Multi processor enqueue packet circuit | |
| US6671277B1 (en) | Network relaying apparatus and network relaying method capable of high quality transfer of packets under stable service quality control | |
| US6976149B1 (en) | Mapping technique for computing addresses in a memory of an intermediate network node | |
| EP1631906B1 (en) | Maintaining entity order with gate managers | |
| Tantawy et al. | On the design of a multigigabit IP router | |
| JP3645733B2 (en) | Network relay device and network relay method | |
| JP2000151696A (en) | Information processing system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued |