US 20090141719 A1
Methods, systems, and apparatuses related to a communication switch are disclosed herein. In some embodiments, the communication switch may be configured to transmit TDM, ATM and/or packet data from an ingress service processor, through a plurality of switch elements, to an egress service processor. Other embodiments may be described and claimed.
8. A method comprising:
transmitting a bandwidth request element;
receiving a grant corresponding to the bandwidth request element;
generating a protocol data unit (PDU) including routing information, to facilitate routing of the PDU through a plurality of stages of a communication switch, and the PDU further including an indication of a first stage of the plurality of stages at which multicasting begins; and
transmitting, based at least in part on said receiving of the grant, the PDU to the first stage.
9. The method according to
copying the PDU at the first stage to produce one or more copies of the PDU; and
replacing the routing information in the one or more copies of the PDU with updated routing information.
10. The method according to
transmitting the bandwidth request element in row N of a data frame; and
transmitting the PDU in row N+1 of the data frame.
11. A method comprising:
generating a bandwidth request element including routing information, to facilitate routing of the bandwidth request element through a plurality of stages of a communication switch, and the bandwidth request element further including an indication of a first stage of the plurality of stages at which multicasting begins; and
transmitting the bandwidth request element to the first stage.
12. The method according to
transmitting the bandwidth request element in row N of the data frame.
13. The method according to
transmitting the data frame in 125 microseconds.
14. The method according to
receiving a grant, corresponding to the bandwidth request element, via an out-of-band link.
15. A system comprising:
a receive switch controller configured to generate a bandwidth request element including routing information, to facilitate routing of the bandwidth request element through a plurality of stages of a communication switch, and the bandwidth request element further including an indication of a first stage of the plurality of stages at which multicasting begins; and
an interface coupled to the receive switch controller and configured to transmit the bandwidth request element to the first stage.
16. The system according to
a switch element acting as the first stage and including a controller configured to receive the bandwidth request element;
copy the bandwidth request element to produce one or more copies of the bandwidth request element; and
replace the routing information in the one or more copies of the bandwidth request element with updated routing information.
17. The system according to
18. The system according to
a mapper configured to generate a repeating data frame including a plurality of rows with the bandwidth request element in row N to request bandwidth in row N+1.
19. A system comprising:
a receive switch controller to generate a protocol data unit (PDU) including routing information, to facilitate routing of the PDU through a plurality of stages of a communication switch, and the PDU further including an indication of a first stage of the plurality of stages at which multicasting begins; and
an interface coupled to the receive switch controller and configured to transmit a bandwidth request element, to receive a grant corresponding to the bandwidth request element, and to transmit the PDU to the first stage based at least in part on said receiving of the grant.
20. The system according to
a switch element acting as the first stage and including a controller configured to receive the PDU;
copy the PDU to produce one or more copies of the PDU; and
replace the routing information in the one or more copies of the PDU with updated routing information.
21. The system according to
a mapper to generate a repeating data frame including a plurality of rows with the bandwidth request element in row N to request bandwidth in row N+1.
22. A communications switch comprising:
a first switch element including one or more ports;
a second switch element including one or more ports; and
a port processor including
a first switch element interface coupled to a first port of the one or more ports of the first switch element;
a second switch element interface coupled to a first port of the one or more ports of the second switch element; and
a receive switch controller configured to automatically redirect traffic to either of said first or second switch element interfaces based at least in part on a switch congestion event.
23. The communications switch according to
24. The communications switch according to
25. A communications switch comprising:
a switch element including a first port and a second port; and
a port processor including
a first switch element interface coupled to the first port;
a second switch element interface coupled to the second port; and
a receive switch controller configured to automatically redirect traffic to either of said first or second switch element interfaces based at least in part on a switch failure or congestion event.
26. The communications switch according to
27. A communications switch comprising:
a first switch fabric including a plurality of first switch elements;
a second switch fabric including a plurality of second switch elements; and
a port processor including
a first switch fabric interface coupled to one of the plurality of first switch elements,
a second switch fabric interface coupled to one of the plurality of second switch elements, and
a receive switch controller configured to automatically redirecting network traffic through either of said first switch fabric or said second switch fabric based at least in part on switch congestion.
28. The communications switch according to
29. A communications switch comprising:
a plurality of switch elements; and
a port processor configured to repeatedly transmit a request element through a first subset of the plurality of switch elements based at least in part on a first routing tag in the request element until a grant corresponding to the request is received or the request is transmitted through the first subset a predetermined number of times, and, in an event in which the request is transmitted through the first subset a predetermined number of times, to transmit the request element through a second subset of the plurality of switch elements based at least in part on a second routing tag in the request element.
30. The communication switch of
a first switch fabric including the plurality of switch elements;
a second switch fabric; and
the port processor is further configured to transmit the request element over the first switch fabric or the second switch fabric based at least in part on the channel tag.
This application claims priority to U.S. patent application Ser. No. 10/155,517, filed May 24, 2002, which is a continuation-in-part of U.S. Pat. No. 6,631,130, issued Oct. 7, 2003, the specifications of which are hereby incorporated in their entirety.
Disclosed embodiments relate to telecommunications networks and, more particularly, to transmitting data through telecommunication switches in said networks.
One of the earliest techniques for employing broadband telecommunications networks was called time division multiplexing (TDM). The basic operation of TDM is simple to understand. A high frequency signal is divided into multiple time slots within which multiple lower frequency signals can be carried from one point to another. The actual implementation of TDM is quite complex, however, requiring sophisticated framing techniques and buffers in order to accurately multiplex and demultiplex signals. The North American standard for TDM (known as T1 or DS1) utilizes twenty-four interleaved channels together having a rate of 1.544 Mbits/sec. The European standard for TDM is known as E-1 and utilizes thirty interleaved channels having a rate of 2.048 Mbits/sec. A hierarchy of multiplexing is based on multiples of the T1 or E-1 signal, one of the most common being T3 or DS3. A T3 signal has 672 channels, the equivalent of twenty-eight T1 signals. TDM was originally designed for voice channels. Today, however, it is used for both voice and data.
An early approach to broadband data communication was called packet switching. One of the differences between packet switching and TDM is that packet switching includes methods for error correction and retransmission of packets which become lost or damaged in transit. Another difference is that, unlike the channels in TDM, packets are not necessarily fixed in length. Further, packets are directed to their destination based on addressing information contained within the packet. In contrast, TDM channels are directed to their destination based on their location in the fixed frame. Today, a widely used packet switching protocol is known as IP (Internet Protocol).
More recently, broadband technologies known as ATM and SONET have been developed. The ATM network is based on fixed length packets (cells) of 53-bytes each (48-bytes payload with 5-bytes overhead). One of the characteristics of the ATM network is that users contract for a quality of service (QOS) level. Thus, ATM cells are assigned different priorities based on QOS. For example, constant bit rate (CBR) service is the highest priority service and is substantially equivalent to a provisioned TDM connection. Variable bit rate (VBR) service is an intermediate priority service which permits the loss of cells during periods of congestion. Unspecified bit rate (UBR) service is the lowest priority and is used for data transmission which can tolerate high latency such as e-mail transmissions.
The SONET network is based on a frame of 810-bytes within which a 783-byte synchronous payload envelope (SPE) floats. The payload envelope floats because of timing differences throughout the network. The exact location of the payload is determined through a relatively complex system of stuffs/destuffs and pointers. In North America, the basic SONET signal is referred to as STS-1 (or OC-1). The SONET network includes a hierarchy of SONET signals wherein up to 768 STS-1 signals are multiplexed together providing the capacity of 21,504 T1 signals (768 T3 signals). STS-1 signals have a frame rate of 51.84 Mbit/sec, with 8,000 frames per second, and 125 microseconds per frame. In Europe, the base (STM-1) rate is 155.520 Mbit/sec, equivalent to the North American STS-3 rate (3*51.84=155.520), and the payload portion is referred to as the virtual container (VC). To facilitate the transport of lower-rate digital signals, the SONET standard uses sub-STS payload mappings, referred to as Virtual Tributary (VT) structures. (The ITU calls these Tributary Units or TUs.) Four virtual tributary sizes are defined: VT-1.5, VT-2, VT-3 and VT-6. VT-1.5 has a data transmission rate of 1.728 Mbit/s and accommodates a T1 signal with overhead. VT-2 has a data transmission rate of 2.304 Mbit/s and accommodates an E1 signal with overhead. VT-3 has a data transmission rate of 3.456 Mbit/s and accommodates a T2 signal with overhead. VT-6 has a data transmission rate of 6.912 Mbit/s and accommodates a DS2 signal with overhead.
Each of the above described broadband technologies can be categorized as TDM, ATM, or Packet technologies, with SONET being a complex form of TDM. From the foregoing, it will be appreciated that TDM, ATM and Packet each desire their own unique transmission characteristics. Consequently, different kinds of switches are used to route these different kinds of signals. In particular, TDM desires careful time synchronization; ATM desires careful attention to the priority of cells and QOS; and packet (e.g. IP) desires the ability to deal with variable length packets. For these reasons, switching technologies for TDM, ATM, and variable length packet switching have evolved in different ways. Service providers and network designers have thus been forced to deal with these technologies separately, often providing overlapping networks with different sets of equipment which can only be used within a single network.
Appendix A is an engineering specification (Revision 0.3) for a port processor according to some embodiments; and
Appendix B is an engineering specification (Revision 0.3) for a switch element according to some embodiments.
The apparatus of some embodiments generally includes a port processor and a switch element.
The ingress portion of the port processor 10 also includes a switch mapper 52, a parallel to serial switch fabric interface 54, and a request arbitrator 56. The egress portion of the port processor also includes a serial to parallel switch fabric interface 58, a switch demapper 60, and a grant generator 62.
For processing ATM and packet traffic, the port processor 10 utilizes, at the ingress portion, a descriptor constructor 64, an IPF and ATM lookup processor 66, an IP classification processor 68, an RED/Policing processor 70, all of which may be located off-chip. These units process ATM cells and packets before handing them to a (receive) data link manager 72. At the egress portion of the port processor, a (transmit) data link manager 74 and a transmit scheduler and shaper 76 are provided. Both of these units may be located off-chip. The port processor is also provided with a host interface 78 and a weighted round robin scheduler 80.
The purpose of the port processor at ingress to the switch is to unpack TDM, Packet, and ATM data and frame it according to the data frame described below with respect to
Though not shown in
Turning now to
The forward datapaths of each “datapath and link bandwidth arbitration module” 102 include a data stream deserializer 126, a data stream demapper 128, a row buffer mapper 130, a row buffer 132, a request arbitration module 134, a data stream mapper 136, and a data stream serializer 138. The return grant path for each module 102 includes a grant stream deserializer 140, a grant stream demapper 142, a grant arbitration module 144, a grant stream mapper 146, and a grant stream serializer 148.
The switch element 100 also includes the following modules which are instantiated only once and which support the functions of the twelve “datapath and link bandwidth arbitration modules” 102: a link synchronization and timing control 150, a request parser 152, a grant parser 154, and a link RISC processor 156. The switch element 100 also includes the following modules which are instantiated only once and which support the other modules, but which are not directly involved in “switching”: a configuration RISC processor 158, a system control module 160, a test pattern generator and analyzer 162, a test interface bus multiplexer 164, a unilink PLL 166, a core PLL 168, and a JTAG interface 170.
A typical switch according to some embodiments includes multiple port processors 10 and multiple switch elements 100. For example, as shown in
Before describing in detail the functions of the port processor 10 and the switch element 100, it should be appreciated that some embodiments utilize a unique framing technique which is well adapted to carry combinations of TDM, ATM, and Packet data in the same frame. Turning now to
The contents of the LOH slots may be inserted by the switch mapper (52 in
For ATM and packet data, a PDU (protocol data unit) of sixteen slots may be defined for a sixty-four-byte payload (large enough to accommodate an ATM cell with overhead). The format of the PDU is illustrated in
As shown in
As mentioned above, since ATM and Packet traffic are typically not provisioned, bandwidth may be arbitrated among ATM and Packet connections as traffic enters the system. Moreover, since TDM traffic shares the same frame as ATM and Packet traffic, bandwidth may be arbitrated while maintaining TDM timing. According to some embodiments, bandwidth is arbitrated by a system of requests and grants which is implemented for each PDU in each row of the frame. The request elements, which are generated by the port processors include “hop-by-hop” internal switch routing tags, switch element stage, and priority information. According to some embodiments two request elements are sent in a three contiguous slot bundle and at least eight slots of non-request element traffic must be present between request element bundles. The time separation between request element bundles may be used by the arbitration logic in the switch elements and the port processors to process the request elements. The request element format is shown in section 7.1.5 of Appendix B.
The structure shown in
According to some embodiments, TDM traffic may be switched through the switch elements with a finest granularity of one slot per row. The TDM traffic may be switched through the same path for a given slot for every row. The switch elements may not allow different switch paths for the same TDM data slot for different rows within the frame. This means that the switch does not care about what the current row number is (within a frame). The only time row numbering matters is when interpreting the contents of the Link Overhead slots.
With a finest granularity of one slot per row, the switch elements can switch TDM traffic with a minimum of 2.52 Mbps of switching bandwidth. Since a slot can carry the equivalent of four columns of traffic from a SONET SPE, it can be said that the switch elements switch TDM traffic with a granularity of a VT1.5 or VT2 channel. Although a VT1.5 channel may only occupies three columns in the SONET SPE, it will still be mapped to the slot format which is capable of holding four SPE columns. As mentioned above, the format of the contents of the thirty-six-bit slot carrying TDM traffic is a four-bit tag and a thirty-two bits of payload. The tag field definitions are shown in Table 1 below.
The switch elements know whether or not a slot contains TDM data via preconfigured connection tables. These tables may be implemented as an Input Cross Connect RAM for each input link. The input slot number is the address into the RAM, while the data output of the RAM contains the destination output link and slot number. The connection table can be changed by a centralized system controller which can send control messages, to the switch elements via either of two paths: (1) a host interface port or (2) in-band control messages which are sent via the link data channel. Since TDM connections will be changed infrequently, this relatively slow control message approach to update the connection tables is acceptable. It is the responsibility of an external software module to determine and configure the connection tables within the switch elements such that no TDM data will be lost.
Returning now to
Parallel data is sent to the SONET framer and transport overhead (TOH) block 14. All incoming signals may be framed according to the BELLCORE GR-253 standard which is incorporated herein by reference. The byte boundary and the frame boundary are found by scanning a series of sixteen bit words for the F628 pattern. The framer frames on the pattern F6F6F62828288. Independent SONET SPEs within the STS-N frame are demultiplexed by the framer 14. There is a maximum of four independent line interfaces, therefore the framer 14 includes four independent framers. The inputs to the framer include a sixteen-bit parallel data input, and a one-bit clock which will accept 155 MHz, 38.87 MHz, or 9.72 MHz. The outputs of the framer include a sixteen-bit parallel data output, a one-bit start of frame (SOF) indication, a six-bit SPE ID used to indicate SONET SPE number. The SPEs are numbered 1 through 48 with respect to the line side port configuration.
The block 14 also terminates the transport (section and line) overhead for each independent SONET SPE. Since there are a maximum of forty-eight OC-1s on the line side, forty-eight transport overhead blocks are provided unless blocks are time-shared. The inputs to the TOH termination are the same as those discussed above with respect to the framer. The six-bit SPE ID enables data into this block. There may be no need for an output data bus as the traffic is routed to this block and to the next block (Ptr Proc 16) on the same data bus. The data path may only flow into this block, not through it.
The pointer processor 16 uses the SONET pointer (H1, H2 and H3 bytes in the TOH) to correctly locate the start of the payload data being carried in the SONET envelope. The SONET pointer identifies the location of byte #1 of the path overhead. The pointer processor 16 is responsible for accommodating pointer justifications that were inserted in order to justify the frequency difference between the payload data and the SONET envelope. Since there may be a maximum of forty-eight OC-1s, forty-eight pointer processor blocks are mated to the forty-eight transport overhead termination blocks unless blocks are time-shared. The inputs to the pointer processor 16 are the same as those to the framer and TOH terminator 14. The outputs include a sixteen-bit parallel data output, a one-bit start of SPE indicator which coincides with word 1 of SPE 3, a one-bit SPE valid indicator which gaps out overhead and accommodates pointer movements, and a one-bit POH valid indicator which indicates when a path overhead byte is on the output bus.
The POH processor 18 processes the nine bytes of Path Overhead in each of the forty-eight SONET SPES. Since there are a maximum of forty-eight SPEs, forty-eight path overhead processors are provided unless processors are time-shared. The inputs to the path overhead processor 18 include an eight-bit parallel data input, a four-bit SPE ID, the one-bit start of SPE indicator, and the one-bit POH valid indicator. The outputs include a one-bit V1 indicator, J1 info, alarms, and path status. Further details about blocks 14, 16, and 18 are provided by the GR-253 standard and documentation accompanying standard SONET mapper/demappers such as those available from Lucent or TranSwitch.
Once the frame boundaries of the incoming SONET/SDH signals are found and the location of the SPEs has been identified either through pointer processing or through Telecom bus I/F control signals, and the Path Overhead is processed, the payload is extracted from the SPE. The SPEs may be carrying TDM traffic, ATM cells or IP packets. The type of traffic for each SPE may be configured through the microprocessor interface 78. Each SPE can carry only one type of traffic. The data from each SPE is routed directly to the correct payload extractor.
SPEs containing packets and ATM cells may be sent to the HDLC framer 20 and the cell delineation block 22, respectively. Each SPE may be configured to carry packet data (packet over SONET). The Port Processor 10 supports packet over SONET for the following SONET (SDH) signals: STS-1 (VC-3), STS-3c (VC-4), STS-12c (VC-4-4c), and STS-48c (VC-4-16c). The datagrams may be encapsulated in PPP packets which are framed using the HDLC protocol. The HDLC frames are mapped byte wise into SONET SPEs and high order SDH VCs. The HDLC framer 20 performs HDLC framing and forwards the PPP packet to a FIFO buffer 24 where it awaits assembly into PDUs. The framer 20 has an input which includes a sixteen-bit parallel data input, a six-bit SPE ID, a one-bit SPE valid indicator, and a one-bit PYLD valid indicator. The output of the framer 20 includes a sixteen-bit data bus, a one-bit start of packet indicator, and a one-bit end of packet indicator. Further details about packet extraction from SONET are found in IETF (Internet Engineering Task Force) RFC 1619 (1999) which is incorporated herein by reference.
The cell delineation block 22 is based on ITU-T G.804, “ATM Cell Mapping into Plesiochronous Digital Hierarch (PDH)”, 1998, the complete disclosure of which is hereby incorporated herein by reference. The cell delineation block 22 has inputs that include a sixteen-bit parallel data bus, a six-bit SPE ID, a one-bit SPE valid indicator, and a one-bit POH valid indicator. The outputs include a sixteen-bit parallel data bus and a one-bit start of cell indicator. Cells are placed in a FIFO 24 while awaiting assembly into PDUS. Further details regarding ATM extraction from SONET are found in ITU-T G.804.
The TDM data is routed to a TDM demultiplexer and low order pointer processor block 26 where the low order VTs and VCs are identified. If a particular SPE is configured for TDM data, then the TDM mapping is described using the host interface 78. Each SPE can carry a combination of VC-11, VC-12, VC-2, VC-3 & VC-4. There are seven VT groups in a single STS-1 payload, each VT group has twelve columns. Within one VT Group all of the VTs must be the same. Different VT groups within the same STS-1 SPE can carry different VT types, but within the group all VTs may be of the same type. The VCs and VTs are demultiplexed out of the SONET signal based on the configuration for each of the SPEs. There is no interpretation of the traffic required to locate the containers and tributaries as all of this information is found in the configuration table (not shown) which is configured via the host interface 78. Frames are located inside of the VCs and the VTs through the H4 byte in the path overhead of the SPE. Pointer processing is performed as indicated by the V bytes in the VT superframe. The TDM demultiplexer and low order pointer processor block 26 has inputs which include sixteen bits of parallel data, a six-bits SPE ID, a one-bit start of SPE indicator, a one-bit SPE valid indicator, a one-bit V1 indicator, and one-bit POH valid indicator. The TDM demultiplexer and low order pointer processor block 26 provides the following outputs to the switch mapper 52: sixteen bits of parallel data, a one-bit VT/VC valid indicator, a six-bit SPE ID, and a five-bit VT/VC Number (0-27). The TDM data is placed in reserved slots in the frame as mentioned above and described in more detail below with reference to the switch mapper 52. Further details regarding TDM extraction are found in the GR-253 specification
IP packets and ATM cells from the UTOPIA interface 44 may be placed in FIFO 46. Packets and cells from the FIFOs 24 may be merged with the packets and cells from the FIFO 46. The descriptor constructor 64 determines whether the data is an ATM cell or an IP packet and generates a corresponding interrupt to trigger the IPF/ATM look-up processor 66 to perform either IP routing look-up or ATM look-up. IP routing look-up is performed by searching for the IP destination address for every packet and the IP source address for packets that need classification. ATM look-up is performed by searching the VPI/VCI fields of the cells. Outputs of the IPF/ATM look-up processor 66 for both IP packets and ATM cells include a seventeen-bit flow; index, a five-bit QOS index, and an indicator showing whether the IP packet-needs classification. If the IP packet needs classification, the packet is passed to the IP classification processor 68 for classification; otherwise it is passed to the next stage of packet processing, the RED/policing processor 70. IP classification is described in detail in section 6.4 of Appendix A. The RED/Policing processor 70 performs random early detection and weighted random early detection for IP congestion control, performs leaky bucket policing for ATM traffic control, and performs early packet and partial packet discard for controlling ATM traffic which contains packets. The RED/Policing traffic control is described in detail in sections 7.5 et seq. of Appendix A. Some embodiments of the port processor 10 includes a mode register (not shown) which can be placed in a bypass mode to globally turn off the IP/ATM forwarding. In bypass mode, an external device is used for IP/ATM forwarding, and the data descriptors generated by the descriptor constructor 64 are routed directly to an output FIFO (not shown).
All of the data stored in the FIFOs 24 and 46 may be in fifty-two-byte “chunks”. If an IP packet is longer than fifty-two-bytes, it may be segmented into multiple fifty-two-byte chunks. The input data descriptor for each chunk includes indications of whether the chunk is an ATM cell or a packet, whether it is the start of a packet or the end of a packet, packet length, and the source and destination port numbers. After processing by the IPF/ATM lookup processor 66 and the IP classification processor 68, an output data descriptor is written to a FIFO (not shown) which is read by the RED/Policing processor 70.
Cells and packets which survive RED/policing are read by the receive data link manager 72, which creates the PDUs described above with reference to
As shown in
The switch mapper 52 includes a state machine (not shown) which is associated with the ATM/IP PDUS. The data link manager 72 writes the PDU's using a sixty-four-bit interface to the external FIFO (not shown). The data may be transmitted from the external FIFO to the switch mapper 52 in thirty-two-bit slots with four bits of parity. The state machine associated with the external PDU FIFO monitors the status of the FIFO and maintains data integrity.
In section 9 of Appendix A, the data link manager 72, arbiter block 56, switch mapper 52, and weighted round robin scheduler 80, together with memory and other support circuits (not shown in
The scheduler 80 may be responsible for scheduling cell containers through the switch. The scheduling algorithm used may be a weighted round robin, which operates on the QOS queues. Once cells have been scheduled from these queues the control headers from these queues are forwarded to the arbiter 56 and are stored in a request control table (not shown). The request arbiter 56 forms request elements from the control headers and forwards these requests to the switch data mapper 52 for transmission through the switch. The grants received in response to these requests may be deserialized by block 58, deframed and transferred back to the arbiter block 56 by the grant block 62. For granted requests, the cell containers may be dequeued from external memory by the data link manager 72 and transferred to the switch mapper 52 for transmission through the switch.
As mentioned above, the port processor 10 supports redundancy in order to improve reliability. Two redundancy schemes are supported. In the first redundancy scheme, the switch controller supports redundant routing tags and transparent route switch-over. In the second redundancy scheme, the port processor supports redundant data channels in both input and output directions. The redundant data channels connect to two separate switch fabrics. In the Appendices they are referred to as the A and B data channels. Each control header contains two routing tags, and each routing tag has a corresponding AB channel tag. This provides for two routes through the switch for data transmission. If both routing tags have the same channel tag, this allows for two alternate paths through the same switch fabric. If both routing tags have different channel tags, this allows for a redundant switch fabric and any route failing in one switch fabric will cause a switch-over to use the redundant switch fabric. An AB channel tag may be used to indicate whether the data is to be routed using the A data channel or the B data channel. If, after a programmable number of consecutive tries, no grant is received in response to request elements using the A channel routing tag, a bit may be set to switch over to the B channel routing tag. Further details of the redundancy feature are provided in sections 10.2.3 and 9.2.3 of Appendix A.
As mentioned above, the arbiter 56 may be responsible for sending requests to the switch mapper 52 and processing the grants that arrive from the grant demapper 62. The arbiter dequeues requests from a routing tag FIFO, copies this information into a request control table, writes the FLOWID into FLOWID RAM, resets a request trial counter that counts the number of times a request has been tried, and resets the grant bit. Each request message has a unique request ID which is returned in the grant message. The request ID is the index in the arbiter request control table into which the routing tag is copied. The routing tag along with the request ID may be forwarded to a routing tag formatter block, which formats the routing tag into a request message and inserts the request into a request FIFO in the switch mapper 52.
The grant demapper in the grant block 62 stores the request ID and the grant in a FIFO called the grant_reqid FIFO. In the arbiter_block 56, the request IDs are dequeued from A and B grant reqid FIFOS alternatively depending on whether the switchover bit is set. The request IDs dequeued from the FIFO are used to set a grant bit in the grant register at the bit position indicated by the request ID, to index the FLOWID.RAM, and read the FLOWID associated with the request ID. This FLOWID is written into a deq-flowid FIFO for the appropriate channel, i.e., if the request ID is dequeued from the A reqid_fifo, the FLOWID is written into the A deq_flowid fifo. The data link manager 72 monitors the deqflowid_fifo and uses the FLOWID to dequeue data PDUs from external memory and send them to the switch mapper 52 for transmission in the next row time.
An end_of_grants signal is asserted by the grant demapper 62, when no more grants can be received at the grant demapper. In most switch implementations the end_of_grants signal is rarely, if ever, asserted. It is only in switches having many stages that the end_of_grants signal is more likely to be asserted. Once the end_of_grant signal has been received the arbiter 56 begins the process of updating the request control table. If a grant has not been returned for a routing tag stored in the request control table, the request trial counter is incremented and a new request is generated using the routing tag. If a routing tag in the request control table has been sent as a RE a (programmed) maximum number of times, the most significant fifteen bits of the FLOWID are used to index into the redundancy control table and update the bit to indicate failure of the current path and to select the alternate routing path. Further details regarding the arbiter block 56 are provided at section 9.2.4 of Appendix A.
As described above, the TDM data, ATM/IP PDU's and the request messages may be combined into a single data stream for transmission through the switch fabric. This combination may be performed by the switch mapper 52 on the receive side of the port processor. On the transmit side of the port processor, a switch demapper 60 separates TDM data from ATM/IP PDUs. According to some embodiments, the demapper 60 may be provided with external memory for a PDU FIFO. For ATM/IP data, the demapper writes PDUs to the FIFO and interrupts the data link manager 74. The data link manager 74 reads the header information from the PDU FIFO, and extracts the FLOWID. Based on the FLOWID, the datalink manager 74 retrieves a Linked List/Shaping/Scheduling data structure from external memory. The data link manager 74 writes the linked list pointers to the PDU FIFO, then initiates a DMA transfer to move the PDU to external memory. The data link manager updates the head, tail, and count fields in the Linked List/Shaping/Scheduling data structure and passes the data structure to the Shaping/Scheduling processor 76 through a Shaping/Scheduling FIFO. The Shaping/Scheduling processor 76 performs the Shaping and Scheduling functions and updates the Linked List/Shaping/Scheduling datastructure.
The data-flow from external memory to the SONET/UTOPIA Data FIFOs 30 and 48 may be as follows. The data link manager 74 polls the PDU FIFO and SONET/UTOPIA FIFO status flags. If the PDU FIFO is not empty and the SONET/UTOPIA FIFO is not full for a particular output port, the data link manager 74 retrieves the Link List/Shaping/Scheduling data structure for the Flow ID read from the PDU FIFO. (Note that for an IP packet flow, the data link manager will continue to retrieve PDUs from the Linked List until a PDU with an End of Packet indicator is found.) The data link manager then initiates a DMA transfer from external memory to the SONET/UTOPIA FIFOs 30, 48. The data link manager 74 then updates the Link List/Shaping/Scheduling data structure and writes it back to external memory.
On the transmit side of the port processor 10, the grant framer, deframer, serializer and deserializer in the grant block 62, the switch demapper 60, the transmit datalink manager 74, and the transmit scheduler and shaper 76 are referred to collectively as the transmit (TX) switch controller in Appendix A. The TX switch controller is responsible for either accepting or rejecting requests that come into the port processor for output transmission. To do this, the TX switch controller checks if the queue identified by the output port number of the request can accept a cell container. These one hundred twenty-eight queues may be managed by the TX data link manager 74. According to some embodiments, these queues are stored in external memory. The scheduling of these cell containers may be performed by the TX scheduler 76. If the queue can accept the cell container, the request is turned into a grant and inserted into a grant_fifo. The grant-framer and serializer 62 reads this information and creates a grant message for transmission through the grant path.
The TX switch controller-monitors the status of the data queues for each of the one hundred twenty-eight output ports using the following three rules. If the full_status bit for the requested output port is set, there is no buffer space in the queue for any data PDUs destined for that output port and all requests to that output port may be denied. If the full_status bit is not set and the nearly_full_status bit is set, there is some space in the queue for data PDUs destined for that output port; however this space may be reserved for higher priority traffic. In this instance the QOS number is checked against a threshold (programmed) QOS number and if the QOS number is less than the threshold, the request will be accepted. If the nearly full_status bit is not set, all incoming requests may be granted. If a request is accepted, the corresponding output port counter is incremented. This reserves space in the data buffer (30 or 48) for the arrival of the data PDU at that output port. The transmit data link manager 74 constantly monitors the one hundred twenty-eight output port counters and sets/resets the one hundred twenty-eight full and nearly full status bits.
The port processor 10 creates complete outgoing SONET signals. All of the transport and path overhead functions are supported. The SONET interfaces can run in source timing mode or loop timing mode.
The high order pointer is adjusted by the high order pointer generator 38 through positive and negative pointer justifications to accommodate timing differences in the clocks used to generate the SONET frames and the clock used to generate the SONET SPEs. At initialization, SPE FIFOs may be allowed to fill to halfway before data is taken out. The variations around the center point are monitored to determine if the rate of the SONET envelope is greater than or less than the rate of the SPE. If the rate of the SONET envelope is greater than the rate of the SPE, then the SPE FIFO will gradually approach a more empty state. In this case, positive pointer movements will be issued in order to give the SPE an opportunity to send additional data. If the rate of the SONET envelope is less than the rate of the SPE, then the SPE FIFO will gradually approach a more full state. In this case, negative pointer movements will be issued in order to give the SPE an opportunity to output an extra byte of data from the FIFO. The SONET framer and TOH generator 40 generate transport overhead according to the BELLCORE GR-253 standard.
The outgoing SONET frames may be generated from either the timing recovered from the receive side SONET interface or from the source timing of the Port Processor. Each signal is configured separately and they can be configured differently. The frame orientation of the outgoing SONET frames may be arbitrary. Each of the four signals can be running off different timing so there is no need to try to synchronize them together as they will constantly drift apart. There is no need to frame align the Tx ports to the Rx ports as this would result in realigning the Tx port after every realignment of the Rx port.
For OC-3 and OC-12 the 16-bit wide internal bus is serialized to 155 Mbps or 622 Mbps by the serializer 42. For OC-48 applications, the entire sixteen bit bus is output under the control of an external serializer (not shown).
There is a potential for forty-eight different SPEs being generated for the outgoing SONET interfaces. All of these SPEs may be generated from a single timing reference. This allows all of the SPE generators to be shared among all of the SONET and Telecom bus interfaces without multiplexing between the different clocks of the different SONET timing domains. The SPE consists of the Path level overhead and the payload data. The payload data can be TDM, ATM or packet. All of these traffic types are mapped into single SPEs or concatenated SPEs as desired by their respective standards. As the SPEs are generated, they are deposited into SPE FIFOs. For each SPE there is a sixty-four-byte FIFO and these individual SPE FIFOs are concatenated through SPE concatenation configuration registers. As described above, the fill status of the SPE FIFOs is used to determine the correct time to perform a positive or negative pointer justification.
TDM, ATM and packet data may be all mapped into SONET SPEs as specified by their respective standards. The type of data carried in each of the potential forty-eight SPEs may be configured through the external host processor. Based on this configuration, each SPE generator may be allocated the correct type of mapper. All of this configuration may be performed at initialization and may be changed when the particular SPE is first disabled. Once the configuration is complete, there may be an isolated set of functional blocks allocated to each SPE. This set of functional blocks includes one of each of the following: payload mapper, payload FIFO, POH generator, SPE FIFO and SPE generator. Each of the ATM and packet payload mappers has a payload FIFO into which it writes payload data for a particular SPE. For TDM traffic, each potential Virtual Container is allocated its own FIFO.
Returning now to
The data-stream demapper 128 may be responsible for extracting the data from the incoming serial data links. It demaps the input link slots based on the input slot number and determines whether the traffic is TDM, PDU, or a request element (RE). For TDM traffic, the demapper determines the destination link and row buffer 132 memory address. This information is stored in a demapper RAM (not shown), which may be configured by software when TDM connections are added or torn down. For PDU traffic, the demapper 128 assembles all sixteen slots which make up the PDU into a single 64-byte PDU word, then forwards this entire PDU word to the row buffer mapper logic 130. The PDUs may be assembled prior to forwarding them to the row buffer 132 so that the row buffer mapper 130 can write the entire PDU to the row buffer 132 in a single clock cycle. This provides the maximum possible write-side memory bandwidth to the row buffer 132. It is a significant feature of the switch element that twelve entire PDUs are written to a single row buffer in six link slot times (twelve core clock cycles). For request elements, the demapper 128 assembles the three-slot block of REs into two forty-eight-bit REs and forwards them to the request parser module 152. A detailed description of the data stream demapper 128 is provided in Sections 4.3.1 et seq. of Appendix B.
The row buffer mapper 130 may be responsible for mapping traffic which is received from the data stream demapper 128 into the row buffer 132. The mapper 130 provides FIFO buffers for the TDM traffic as it is received from the data stream demapper 128, then writes it to the row buffer 132. The row buffer memory address is actually preconfigured in the demapper RAM (not shown) within the data stream demapper module 128. That module forwards the address to the row buffer mapper 130 along with the TDM slot data. The mapper 130 also writes PDU traffic from the data stream demapper 128 to the row buffer 132 and computes the address within the row buffer 132 where each PDU will be written. PDUs are written into the row buffers starting at address 0 and then every sixteen-slot address boundary thereafter, up to the maximum configured number of PDU addresses for the row buffer 132. A detailed description of the row buffer mapper 130 is provided in Section 220.127.116.11 of Appendix B.
The row buffer 132 contains the row buffer memory elements. According to some embodiments, it provides double buffered row storage which allows one row buffer to be written during row N while the row data which was written during row N−1 is being read out by the data stream mapper 136. Each row buffer is capable of storing 1536 slots of data. This allows the row buffer to store ninety-six PDUs or 1536 TDM slots or a combination of the two traffic types. Request elements and link overhead slots may not be sent to the row buffer 132. Therefore the row buffer may not need to be sized to accommodate the entire 1700 input link slots. According to some embodiments, the row buffer write port is 16*36=576 bits wide and it supports writing of only one thirty-six-bit slot (TDM data) or writing of an entire 576-bit word (PDU data) in a single clock cycle. A detailed description of the row buffer 132 is provided in Section 18.104.22.168 of Appendix B.
Request arbitration utilizes two components: a centralized request parser module 152 and a request arbitration module 134 for each of the output links. Request elements are extracted from the input slot stream by the data stream demapper 128 and are forwarded to the request parser 152. The request parser 152 forwards the forty-eight-bit request elements to the appropriate request arbitration module 134 via two request buses (part of the input link bus 120). Each request bus may contain a new request element each core clock cycle. This timing allows the request arbitration logic to process thirteen request sources in less than eight core clock cycles. The thirteen request sources are the twelve input data streams and the internal multicast and in band control messaging module 156. The request arbitration module 134 monitors the two request element buses and reads in all request elements which are targeted for output links the request arbitration module is implementing. According to some embodiments, the request arbitration module 134 provides buffering for up to twenty-four request elements. When a new request element is received, it is stored in a free RE buffer (not shown). If there are not any free RE buffers, then the lowest priority RE which is already stored in a buffer is replaced with the new RE if the new RE is a higher priority. If the new RE is equal to or lower in priority than all REs currently stored in the RE buffers then the new RE is discarded. On the output side, when the data stream mapper module 138 is ready to receive the next RE, the request arbitration module 134 forwards the highest priority RE which is stored in the RE buffers to the data stream mapper module 136. If the RE buffers are empty, then an “Idle” RE may be forwarded. A detailed description of the request arbitration module 134 is provided in Section 7 of Appendix B.
The data stream mapper 136 may be responsible for inserting data and request elements into the outgoing serial data links. This includes mapping of the output link slots based on the output slot number to determine if the traffic is TDM, PDU, request element, or test traffic. The determination is based on the contents of the mapper RAM (not shown). For TDM traffic, the row buffer memory address may be determined from the mapper RAM which is configured by software as TDM connections are added or torn down. For PDU traffic, the data stream mapper 136 use one slot at a time from the row buffer 132. The row buffer memory address may be stored in the mapper RAM by software. If the target PDU is not valid (i.e., a PDU was not written to that row buffer location during the previous row time), then the mapper 136 transmits an idle pattern in order to ensure that a data PDU is not duplicated within the switch. For request elements, the mapper assembles the three-slot block of REs-from two forty-eight-bit REs. The REs are read from—the request arbitration module 134. For test patterns, the mapper 136 inserts the appropriate test pattern from the output link bus 122. These test patterns are created by either the test pattern generator 162 or test interface bus 164 modules.
The data stream mapper supports slot multicasting at the output stage. For example, the data stream mapper for any output link is able to copy whatever any other output link is sending out on the current slot time. This copying is controlled via the mapper RAM and allows the mapper to copy the output data from another output link on a slot-by-slot basis. A detailed description of the data stream mapper 136 is provided in Section 4 of Appendix B.
The data stream serializer 138 creates the output link serial stream. Data slots are received via the data stream mapper module 136 and the link overhead is generated internally by the data stream serializer 138. The serializer 138 also splits the row data stream into two streams for transmission on the two paths 110, 114. A detailed description of this module is provided in, Section 11 of Appendix B.
The grant stream deserializer 140 in each module 102 works in much the same manner as the data stream deserializer 126. The primary difference is that the grant data only utilizes a single path, thus eliminating the need for deskewing and deinterleaving to recover a single input serial stream. Since this serial link is only one half the data stream rate of the forward link, there are 850 slots per row time. A single FIFO (not shown) may be used to allow for deskewing of the input serial grant streams for all 12 links. A detailed description of the grant stream deserializer 140 may be provided in Section 11 of Appendix B.
The grant stream demapper 142 may be responsible for extracting the data from the incoming serial grant links. This includes demapping of the received grant link slots based on the input slot number to determine if the traffic is a grant element or another kind of traffic. The determination is based on the contents of the grant demapper RAM (not-shown). According to some embodiments, traffic other than grant elements is not yet defined. For grant elements, the grant stream demapper 142 assembles the three-slot block of GEs into two forty-eight-bit GEs and forwards them to the single grant parser module 154. A detailed description of the grant stream demapper 142 is provided in Section 22.214.171.124 of Appendix B.
The grant arbitration module 144 operates in a similar manner to the request arbitration logic 134. In some embodiments, this module is identical to the request arbitration module. The only difference may be that it processes grant elements in the reverse path instead of request elements in the forward path. It will be recalled that grant elements are, in fact, the request elements which have been returned.
The grant stream mapper 146 may be responsible for inserting data into the outgoing serial grant links. It maps the output grant slots based on the output slot number to determine if the traffic is a grant element or test traffic. The determination is based on the contents of the grant mapper RAM (not shown). For grant elements, it assembles the three-slot block of GEs from two forty-eight-bit GEs. The GEs are read from the grant arbitration module 144. For test patterns, it inserts the appropriate test pattern from the output link bus 122. These test patterns may be created by either the test pattern generator 162 or the test interface bus 164 modules. A detailed description of the grant stream mapper 146 is provided in Section 126.96.36.199.
The grant stream serializer 148 works in much the same manner as the data stream serializer 138. The primary difference is that the grant data only utilizes a single path, thus eliminating the need for interleaving the transmit serial stream across multiple output serial streams. Since this serial link is only one half the forward data stream rate, there are only 850 slots per row time. A detailed description of the grant stream serializer 148 is provided in Section 11 of Appendix B.
The modules described above (except for the request parser and the grant parser) may be instantiated for each link module 102 of which there are twelve for each switch element 100. The following modules may be instantiated only once for each switch element.
The link synchronization & timing control 150 provides the global synchronization and timing signals used in the switch element. It generates transmission control signals so that all serial outputs-start sending row data synchronized to the RSYNC (row synchronization) input reference. It also controls the deskewing FIFOs in the data stream deserializers so that all twelve input links will drive the data for slot N at the same time, onto the input link bus 120. This same deskewing mechanism may be implemented on the grant stream deserializers. A detailed description of the link synchronization and timing control 150 is provided in Section 10 of Appendix B.
The request parser 152 receives inputs from all thirteen request element sources and forwards the REs to the appropriate request arbitration modules via the two request element buses. A detailed description of the request parser 152 is provided in Section 188.8.131.52 of Appendix B.
The grant parser 154 physically operates in a similar manner to and may be identical to the request parser 152. The only difference is that it processes grant elements in the reverse path instead of request elements in the forward path. As mentioned above, the grant elements contain the same information as the request elements, i.e. the link address through the switch from one port processor to another.
The link RISC processor 156 controls the ranging synchronization on the input links with the source port processors in the first stage of the switch fabric. It also controls the ranging synchronization on the output link grant stream input with the source port processors in the last stage of the switch fabric. It also handles the Req/Grant processing needed to transmit multicast messages and controls the reception and transmission of the in-band communications PDUs. All in-band communications PDUs are forwarded to the Configuration RISC Processor 158 which interprets the messages. The link RISC processor 156 only handles the Req/Grant processing needed to transmit multicast and in-band communications messages.
The configuration RISC controller 158 processes configuration and status messages received from an external controller module (not shown) and in-band communication messages as described above. The system control module 160 handles all the reset inputs and resets the appropriate internal modules. The configuration RISC controller 158 and the system control module 160 may be implemented with an Xtensa™ processor from Tensilica, Inc., Santa Clara, Calif.
The test pattern generator and analyzer 162 may be used for the generation of various test patterns which can be sent out on any slot on the data stream or grant stream outputs. It is also capable of monitoring input slots from either the received data stream or grant stream.
The test interface bus multiplexer 164 allows for sourcing transmit data from the external I/O pins and forwarding data to the I/O pins. This is used for testing the switch element when a port processor is not available.
The unilink PLL 166 may be used to create the IF clock needed by the unilink-macros. Within each unilink macro another PLL multiplies the IF clock up to the serial clock rate. The core PLL 168 may be used to create the clock used by the switch element core logic. In some embodiments, the core clock is approximately 250 MHz. A detailed description of both PLLs is provided in Section 9 of Appendix B.
The JTAG interface 170 may be used for two purposes: (1) boundary scan testing of the switch element at the ASIC fab and (2) Debug interface for the Configuration RISC Processor.
As shown in
According to some embodiments, each switch element includes a multicast controller and a separate multicast PDU buffer. Multicast request elements flow through the switch in the same manner as standard unicast request elements. At the point where the message needs to be multicast, the hop-by-hop field's bit code for that switch stage indicates that the request is multicast. The request is forwarded to the multicast controller. On the grant path, the multicast controller sources a grant if there is room for the data in the multicast recirculating buffers. Once the data has been transmitted to the multicast buffer, the multicast controller examines the data header and determines which output links it needs to be sent out on. At this point, the multicast controller sources a number of request messages which are handled in the same manner as unicast requests.
There have been described and illustrated herein several embodiments of a network switch which supports TDM, ATM, and IP traffic. While particular embodiments of the invention have been described, it is not intended that the invention be limited thereto, as it is intended that the invention be as broad in scope as the art will allow and that the specification be read likewise. It will therefore be appreciated by those skilled in the art that yet other modifications could be made to the provided invention without deviating from its spirit and scope as so claimed.