|Publication number||US6408006 B1|
|Application number||US 09/201,759|
|Publication date||Jun 18, 2002|
|Filing date||Dec 1, 1998|
|Priority date||Dec 1, 1997|
|Also published as||CA2223193A1|
|Publication number||09201759, 201759, US 6408006 B1, US 6408006B1, US-B1-6408006, US6408006 B1, US6408006B1|
|Inventors||Mark S. Wolff|
|Original Assignee||Alcatel Canada Inc.|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (3), Referenced by (70), Classifications (10), Legal Events (8)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This invention relates to a method of allocating buffers in a fast packet switched network, such as an ATM network or frame relay network, offering different QoS (Quality of Service) classes.
The intention of QoS in a communication system, for example, one which is frame relay or ATM based for the purposes of discussion, is to allow the system to offer a differentiated service for applications with differing performance requirements, particularly in the areas of any information loss or delay which could occur during transfer. An example of two disparate applications which might simultaneously use QoS would be real time packet voice, and bulk electronic mail transfer. To achieve the goal of supporting multiple services, such communication networks and the switches that provide these services may need to add service categories with defined performance objectives for each quality of service.
Focusing on a switch, bridge, router etc. which is implementing QoS for packet (or cell) based communications, the architecture of such a node must be capable of buffering incoming packets. After or during packet processing, the transmit frame or cell scheduling algorithm creates the various service categories by determining the relative order with which the packets of multiple QoS levels should be transmitted. By virtue of the ordering of transmissions, a resulting delay can be incurred, particularly for lower performance service classes which may be large depending on the amount of higher priority traffic present at any given time. Packets which are delayed too long are considered stale and usually are dropped rather than delivered late. To limit the delay encountered by given packets or cells, and also limit the memory resources used by the switch, each QoS level is normally provided with either a buffering resource limit (memory), an aging limit (time) or both.
As examples of possible aging and resource implementations, enforcing aging could be done by recording the time when a packet arrived at the node and comparing it to the time when the packet is at the front of the transmission queue. In FIG. 1, packets A . . . G are placed in queues 1, 2, 3 of relating to different classes of service and having respectively highest, intermediate, and lowest priority. The enqueued packets are passed by the scheduler 5 into the interface 5 in a manner known per se. Packet D might need to wait for packets A-C to be transmitted first, depending on the scheduling algorithm applied. If the aging limit has expired by the time D is determined the next to be sent by the scheduling algorithm, D is discarded. For buffering resource limits, in FIG. 1 packet G is dropped immediately from the lowest priority QoS level since there is already packets D-F buffered and adding G would exceed the buffer limit. One could increase the buffer limit to allow packet G, but only at the cost of increased memory demands.
The problem presented is how to optimally allocate memory to a priority level. For certain periods of time, there may not be any higher priority traffic and all packets admitted inside the buffer limit to a lower priority level will be able to be transmitted within the aging limit. In FIG. 2, packets A-C will all be transmitted within the aging limit.
For other periods of time, there may be large amounts of priority traffic, where lower priority traffic will not have the opportunity to be transmitted without exceeding aging or buffering limits. Focusing only on the lowest priority queue in FIG. 3, packet K will be dropped as expected due to buffering limits. However packets G-J will also eventually be dropped since they will not meet aging limits after packets A-F are transmitted first. Thus packets G-J are buffered despite having no chance of being transmitted.
Therefore, under congestion scenarios there will be node buffering resources uselessly allocated for lower priority class use. If the node has many different priorities per transmission interface and numerous transmission interfaces, the potential waste of buffer resources could become extremely significant.
An object of the invention is to reduce the amount of waste buffering under QoS transmission.
Accordingly the present invention provides a method of allocating buffers in a packet switched communication system offering different classes of service, each class of service being associated with a QoS queue, comprising the steps of computing the optimal buffering when a new packet for a particular QoS class arrives, setting the buffer limit for the queue associated with said particular class in accordance with said determination, and handling said new packet according to said set buffer limit.
Typically, the newly arrived packet is accepted into the queue or discarded according to the newly set queue size.
If the buffer limit is already set at the appropriate value, of course no change to the limit is actually required in order to set it to the desired value.
The buffering allocation for QoS in accordance with the invention is preferably performed by taking into account the timestamp of the frame or cell at the head of a given QoS queue, the transmission line speed, and the aging limit configured for the QoS class. As each newly processed frame or cell is placed into a particular QoS transmission queue, the optimal buffering at that point in time is determined and used to set the buffer limit for the queue, which in turn is used to determine how the new frame or cell is handled.
It is important to note that the invention is not dependent on the priority scheduling algorithm and can be applied to any QoS level. It is not computationally intensive, has low variable storage requirements and can be implemented in a real time system.
The term packet will be used herein to denote both frame and cell based systems.
The invention also provides a packet switched communication system offering different classes of service, each class of service being associated with a QoS queue, comprising means for determining the optimal buffering when a new packet for a particular QoS class arrives, means for setting the buffer limit for the queue associated with said particular class in accordance with said determination, and means for handling said new packet according to said newly set buffer limit.
The invention will now be described in more detail, by way of example, only with reference to the accompanying drawings, in which:
FIG. 1 shows an example of a known Multiple Quality of Service System;
FIG. 2 shows buffering when only low priority traffic is present in the known system;
FIG. 3 shows buffering when high and low priority traffic is present in the known system;
FIG. 4 shows the Start of Quality of Service Adaptive Buffering Frame Sequence in accordance with one embodiment of the invention;
FIG. 5 shows Low Priority Traffic Only;
FIG. 6 shows Low Priority Traffic Only Continued;
FIG. 7 shows the higher priority traffic added;
FIG. 8 shows the Higher Priority Traffic Transmitted in Preference to Lower Priority Traffic;
FIG. 9 shows the Lower Priority Traffic Discarded due to Adaptive Buffering; and
FIG. 10 shows the Lower Priority Traffic Only.
In order to fully understand the invention, the following definitions are required per QoS queue on a given transmission line:
Pqx is the timestamp associated with packet x in the queue for the QoS class, determined when the packet first arrived at the node (in seconds, relative to a fixed point in time, to an adequate number of decimal places).
Sqx is the length of packet x.
Hq is the timestamp for the packet at the head of a given QoS queue, taken from Pqx.
Tq is the configured aging limit for the QoS class (in seconds, to an adequate number of decimal places).
Lq is the buffer limit for the QoS class (in octets).
Bq is the current amount of buffering being used for the QoS class.
The following definitions are used by all QoS queues:
R is the transmission line speed (in bits per second).
C is the current time, the timestamp associated with a newly arrived packet (in seconds, relative to a fixed point in time, to an adequate number of decimal places).
The preferred method works as follows in accordance with the principles of the invention.
At the start, R is known, Tq is configured, Bq is zero, C is system defined, and Hq is undefined.
When a packet arrives at a node, the current value of C is associated as a timestamp with the packet, stored as Pqx. If/when that packet makes it to the head of a given QoS queue, this timestamp becomes the variable Hq. If there is no packet at the head of a given QoS queue, then Hq=Pqx=C. Lq is calculated by subtracting from the desired delay of the entering packet the present delay already encountered by the packet at the head of the queue:
where Tq and[C−Hq]must be≧0
Bq for the QoS queue is incremented by the number of bytes in the packet Sqx. If the result is that Bq>Lq, then the packet will not meet the aging limit, the packet is dropped and Bq is decremented by Sqx to return it to its original value. When a packet ‘x’ finishes being transmitted, Bq is decremented by the number of bytes in the packet, and the next packet in the queue becomes the head of queue, where its Pqx how defines Hq. If ‘x’ was the last packet in a given QoS queue, the Hq returns to undefined. The test of Bq>Lq applies only to when new frames are being introduced.
Graphically, focusing on the lowest priority queue for the purposes of illustration, the method works as follows. A sequence of time is portrayed. At the start, as shown in FIG. 4, no frames are queued to any priority level.
As shown in FIG. 5, packet A arrives destined for the lowest priority queue, and becomes head of queue. Since Hq=Pqx=C, there is no change to Lq, but Bq increases by the size of the frame. Lq is at its maximum and will remain there in the absence of higher priority traffic, regardless of the number of lowest priority packets that are queued (following of course the limitation that Bq<=Lq). This is shown in FIG. 6, where packet B arrives after A has been transmitted.
Before Packet B can be transmitted, packets C and D arrive (FIG. 7) on a higher priority queue.
While packet C is being transmitted in preference to packet B due to its higher priority, packet E arrives destined for the lowest priority queue (FIG. 8). Lq for the lowest priority queue is recalculated using the head of queue timestamp of packet B and the timestamp of newly arrived packet E, with the result being a lower Lq. However the new Bq does not exceed Lq so packet E is admitted.
Packet F (FIG. 9) arrives destined for the lowest priority queue while packet D is being transmitted. It results in a Bq that is larger than Lq and thus the packet is dropped. Packets B and E are not affected by the changes to either Bq and Lq.
Once packet D finishes transmitting, packets B and E will be transmitted (FIG. 10) presuming that their timestamp shows that they have not exceeded age limits.
An optimization which may be added to the described method, depending on the cost to real time processing capabilities, is: when a frame arrives destined for a given queue, before calculating Lq first check if the frame at the head of the queue has exceeded its aging limit and thus can be dropped (optionally this could be extended to the ‘new’ head of queue frame as well, again depending on the real time processing impact). Dropping aged frames as soon as possible creates buffering space for newly arriving frames, which may have a chance to be transmitted.
The described method has the advantages that it provides buffering for all packets which have the potential to be transmitted within desired aging limits, it adjusts buffering levels dynamically based on current conditions, it reduces buffering requirements by not buffering packets that will end up being dropped due to exceeding aging limits, it is not dependent on priority scheduling algorithm employed for quality of service, the method has low storage requirements and computational load, will easily work in real time systems, and the same method can be used regardless of level of quality of service implemented on.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US6115390 *||May 22, 1998||Sep 5, 2000||Lucent Technologies, Inc.||Bandwidth reservation and collision resolution method for multiple access communication networks where remote hosts send reservation requests to a base station for randomly chosen minislots|
|US6167445 *||Oct 26, 1998||Dec 26, 2000||Cisco Technology, Inc.||Method and apparatus for defining and implementing high-level quality of service policies in computer networks|
|US6226277 *||May 22, 1998||May 1, 2001||Lucent Technologies Inc.||Method for admitting new connections based on usage priorities in a multiple access system for communications networks|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US6535514 *||Jun 11, 1999||Mar 18, 2003||Netsilicon, Inc.||Method and apparatus for fast processing of selected packetized data requests|
|US6560230 *||May 21, 1999||May 6, 2003||Redback Networks Inc.||Packet scheduling methods and apparatus|
|US6680906 *||Mar 31, 1999||Jan 20, 2004||Cisco Technology, Inc.||Regulating packet traffic in an integrated services network|
|US6795441||Oct 28, 2002||Sep 21, 2004||Redback Networks, Inc.||Hierarchy tree-based quality of service classification for packet processing|
|US6904045 *||Jun 2, 2000||Jun 7, 2005||Agere Systems Inc.||Method and apparatus for guaranteeing data transfer rates and delays in asynchronous transfer mode networks using pivot sessions|
|US6912225 *||May 18, 2000||Jun 28, 2005||Hitachi, Ltd.||Packet forwarding device and packet priority setting method|
|US6947450 *||Dec 28, 2000||Sep 20, 2005||Mitsubishi Denki Kabushiki Kaisha||Method and a device for determining packet transmission priority between a plurality of data streams|
|US7016302 *||Aug 16, 2000||Mar 21, 2006||Ironbridge Networks, Inc.||Apparatus and method for controlling queuing of data at a node on a network|
|US7027394||Sep 13, 2001||Apr 11, 2006||Narad Networks, Inc.||Broadband system with traffic policing and transmission scheduling|
|US7054267 *||Sep 10, 1999||May 30, 2006||Lucent Technologies Inc.||Method and apparatus for scheduling traffic to meet quality of service requirements in a communication network|
|US7072360||Sep 13, 2001||Jul 4, 2006||Narad Networks, Inc.||Network architecture for intelligent network elements|
|US7113510 *||Mar 1, 2002||Sep 26, 2006||Xinming Allen Lin||Hardware self-sorting scheduling queue|
|US7139247||Sep 13, 2001||Nov 21, 2006||Narad Networks, Inc.||Broadband system with topology discovery|
|US7146630||Sep 13, 2001||Dec 5, 2006||Narad Networks, Inc.||Broadband system with intelligent network devices|
|US7170900 *||Jul 13, 2001||Jan 30, 2007||Telefonaktiebolaget Lm Ericsson (Publ)||Method and apparatus for scheduling message processing|
|US7212539 *||Dec 29, 2004||May 1, 2007||Nec Corporation||Multi-service-class definition type ATM switch|
|US7283472||Oct 28, 2002||Oct 16, 2007||Redback Networks Inc.||Priority-based efficient fair queuing for quality of service classification for packet processing|
|US7339953||Oct 28, 2002||Mar 4, 2008||Redback Networks, Inc.||Surplus redistribution for quality of service classification for packet processing|
|US7414975||Mar 24, 2005||Aug 19, 2008||Ixia||Protocol stack|
|US7450949 *||Jul 18, 2005||Nov 11, 2008||Cisco Technology, Inc.||Method and system for managing real-time bandwidth request in a wireless network|
|US7558197||Jul 30, 2002||Jul 7, 2009||Juniper Networks, Inc.||Dequeuing and congestion control systems and methods|
|US7593948 *||Jun 23, 2005||Sep 22, 2009||Network Appliance, Inc.||Control of service workload management|
|US7619969 *||Nov 17, 2009||Xinming Allen Lin||Hardware self-sorting scheduling queue|
|US7684422 *||Mar 23, 2010||Juniper Networks, Inc.||Systems and methods for congestion control using random early drop at head of buffer|
|US7694166||Apr 6, 2010||Network Appliance, Inc.||Integrating control of service during cluster failover|
|US7835379||Nov 16, 2010||Ciena Corporation||Network architecture for intelligent network elements|
|US8036117||Oct 11, 2011||Juniper Networks, Inc.||Dequeuing and congestion control systems and methods|
|US8072998||Dec 10, 2009||Dec 6, 2011||Juniper Networks, Inc.||Systems and methods for congestion control using random early drop at head of buffer|
|US8165123 *||Apr 24, 2012||Hitachi, Ltd.||Packet forwarding device and packet priority setting method|
|US8184643||Sep 14, 2005||May 22, 2012||Ciena Corporation||Device, system, and method for transporting data using combined broadband and legacy network infrastructures|
|US8213449 *||Aug 29, 2008||Jul 3, 2012||Sprint Spectrum L.P.||Aging EV-DO pages in a queue based on latency-sensitivity|
|US8599868||Dec 29, 2010||Dec 3, 2013||Juniper Networks, Inc.||Systems and methods for determining the bandwidth used by a queue|
|US8682996||Nov 18, 2010||Mar 25, 2014||Fujitsu Semiconductor Limited||Apparatus for handling message reception|
|US20010007570 *||Dec 28, 2000||Jul 12, 2001||Mitsubishi Denki Kabushiki Kaisha||Method and a device for determining packet transmission priority between a plurality of data streams|
|US20020075805 *||Sep 13, 2001||Jun 20, 2002||Narad Networks, Inc.||Broadband system with QOS based packet handling|
|US20020075814 *||Sep 13, 2001||Jun 20, 2002||Narad Networks, Inc.||Broadband system with topology discovery|
|US20020075875 *||Sep 13, 2001||Jun 20, 2002||Narad Networks, Inc.||Broadband system with transmission scheduling and flow control|
|US20020078464 *||Sep 13, 2001||Jun 20, 2002||Narad Networks, Inc.||Broadband system with intelligent network devices|
|US20020085552 *||Sep 13, 2001||Jul 4, 2002||Narad Networks, Inc., Westford Massachusetts||Broadband system having routing identification assignment|
|US20020085589 *||Sep 13, 2001||Jul 4, 2002||Narad Networks, Inc.||System and method for assigning network data packet header|
|US20020097674 *||Sep 13, 2001||Jul 25, 2002||Narad Networks, Inc.||System and method for call admission control|
|US20020101820 *||Sep 13, 2001||Aug 1, 2002||Narad Networks, Inc.||Broadband system with traffic policing and transmission scheduling|
|US20020105965 *||Sep 13, 2001||Aug 8, 2002||Narad Networks, Inc.||Broadband system having routing identification based switching|
|US20020138854 *||Sep 13, 2001||Sep 26, 2002||Narad Networks, Inc.||System and method for mapping end user identifiers to access device identifiers|
|US20030016685 *||Jul 13, 2001||Jan 23, 2003||Arthur Berggreen||Method and apparatus for scheduling message processing|
|US20030165149 *||Mar 1, 2002||Sep 4, 2003||Lin Xinming Allen||Hardware self-sorting scheduling queue|
|US20040019876 *||Sep 13, 2001||Jan 29, 2004||Narad Networks, Inc.||Network architecture for intelligent network elements|
|US20040042484 *||Oct 28, 2002||Mar 4, 2004||Widmer Robert F.||Surplus redistribution for quality of service classification for packet processing|
|US20040081091 *||Oct 28, 2002||Apr 29, 2004||Widmer Robert F.||Priority-based efficient fair queuing for quality of service classificatin for packet processing|
|US20050111458 *||Dec 29, 2004||May 26, 2005||Nec Corporation||Multi-service-class definition type ATM switch|
|US20050207419 *||May 11, 2005||Sep 22, 2005||Hitachi, Ltd.||Packet forwarding device and packet priority setting method|
|US20050246754 *||Jul 11, 2005||Nov 3, 2005||Narad Networks, Inc.||System and method for mapping end user identififiers to access device identifiers|
|US20050250509 *||Jul 18, 2005||Nov 10, 2005||Cisco Technology, Inc., A California Corporation||Method and system for managing real-time bandwidth request in a wireless network|
|US20050251846 *||Jul 12, 2005||Nov 10, 2005||Narad Networks, Inc.||Network architecture for intelligent network elements|
|US20060294045 *||Jun 23, 2005||Dec 28, 2006||Darrell Suggs||Control of service workload management|
|US20070053349 *||Sep 2, 2005||Mar 8, 2007||Bryan Rittmeyer||Network interface accessing multiple sized memory segments|
|US20070076746 *||Sep 14, 2005||Apr 5, 2007||Faska Thomas S||Device, system, and method for transporting data using combined broadband and legacy network infrastructures|
|US20090252219 *||Dec 12, 2005||Oct 8, 2009||Koninklijke Philips Electronics, N.V.||Method and system for the transmission of digital video over a wireless network|
|US20100172363 *||Jul 8, 2010||Juniper Networks, Inc.||Systems and methods for congestion control using random early drop at head of buffer|
|US20100238801 *||Sep 23, 2010||Smith Donald L||Method and system for stale data detection based quality of service|
|US20100274897 *||Oct 28, 2010||Yean Kee Yong||Random Early Detect and Differential Packet Aging Flow Control in Switch Queues|
|US20110096689 *||Apr 28, 2011||Juniper Networks, Inc.||Systems and methods for determining the bandwidth used by a queue|
|US20110131284 *||Nov 18, 2010||Jun 2, 2011||Fujitsu Semiconductor Limited||Message reception|
|US20130170349 *||Jul 2, 2012||Jul 4, 2013||Ren Wang||Generating, at least in part, and/or receiving, at least in part, at least one request|
|US20140105012 *||Dec 19, 2013||Apr 17, 2014||Rockstar Consortium Us Lp||Dynamic Assignment of Traffic Classes to a Priority Queue in a Packet Forwarding Device|
|US20140105025 *||Dec 19, 2013||Apr 17, 2014||Rockstar Consortium Us Lp||Dynamic Assignment of Traffic Classes to a Priority Queue in a Packet Forwarding Device|
|USRE42047||Jan 18, 2011||Lg Electronics Inc.||Method for providing multiple services and apparatus for the same|
|USRE42956||Nov 11, 2010||Nov 22, 2011||Lg Electronics Inc.||Method for providing multiple services and apparatus for the same|
|USRE45600||May 8, 2013||Jul 7, 2015||Intel Corporation||Techniques for collaborative power management for heterogeneous networks|
|EP2330791A1||Nov 30, 2009||Jun 8, 2011||Fujitsu Semiconductor Limited||Message reception|
|U.S. Classification||370/412, 370/395.42|
|International Classification||H04L12/823, H04L12/927|
|Cooperative Classification||H04L45/00, H04L47/2491, H04L47/2408|
|European Classification||H04L45/00, H04L47/24K, H04L47/24A|
|Feb 1, 1999||AS||Assignment|
Owner name: NEWBRIDGE NETWORKS CORPORATION, CANADA
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:WOLFF, MARK S.;REEL/FRAME:009760/0786
Effective date: 19981216
|Oct 17, 2000||AS||Assignment|
Owner name: ALCATEL NETWORKS CORPORATION, ONTARIO
Free format text: CHANGE OF NAME;ASSIGNOR:NEWBRIDGE NETWORKS CORPORATION;REEL/FRAME:011241/0620
Effective date: 20000525
|Feb 8, 2001||AS||Assignment|
Owner name: ALCATEL CANADA INC., ONTARIO
Free format text: CHANGE OF NAME;ASSIGNOR:ALCATEL NETWORKS CORPORATION;REEL/FRAME:011534/0458
Effective date: 20000929
|Dec 9, 2005||FPAY||Fee payment|
Year of fee payment: 4
|Dec 11, 2009||FPAY||Fee payment|
Year of fee payment: 8
|Jan 30, 2013||AS||Assignment|
Owner name: CREDIT SUISSE AG, NEW YORK
Free format text: SECURITY INTEREST;ASSIGNOR:ALCATEL-LUCENT CANADA INC.;REEL/FRAME:029826/0927
Effective date: 20130130
|Dec 12, 2013||FPAY||Fee payment|
Year of fee payment: 12
|Sep 4, 2014||AS||Assignment|
Owner name: ALCATEL-LUCENT CANADA INC., CANADA
Free format text: RELEASE OF SECURITY INTEREST;ASSIGNOR:CREDIT SUISSE AG;REEL/FRAME:033686/0798
Effective date: 20140819