Home
Patent Search
IMT Blog
REGISTER
|
SIGN IN
United States Patent Application
20030210686
Kind Code
A1
Terrell , William ; et al.
November 13, 2003
ROUTER AND METHODS USING NETWORK ADDRESSES FOR VIRTUALIZATION
Abstract
A router for use in a network includes a scalable architecture and performs methods for implementing quality of service on a logical unit behind a network port; and for implementing storage virtualization. The architecture includes a managing processor, a supervising processor; and a plurality of routing processors coupled to a fabric. The managing processor has an in-band link to a routing processor. A routing processor receives a frame from the network, determines by parsing the frame, the protocol and logical unit number, and routes the frame to a queue according to a traffic class associated with the logical unit number in routing information prepared for the processors. An arbitration scheme empties the queue in accordance with a deficit round robin technique. If a routing processor detects the frame"s destination is a virtual entity, and so is part of a virtual transaction, the router conducts a nonvirtual transaction in concert with the virtual transaction. The nonvirtual transaction accomplishes the intent of the virtual transaction but operates on an actual network port, for example, a storage device.
Inventors:
Terrell; William
(Thousand Oaks, CA)
, Jeong; Wayland
(Agoura Hills, CA
)
, Muliadi; Haun
(Thousand Oaks, CA
)
, Chan; Norman
(Diamond Bar, CA
)
, Hill; Rexford
(San Diego, CA
)
, Nishimura; Michael
(San Diego, CA
)
, How; Stephen
(San Diego, CA
)
, Peterson; Eric
(Camarillo, CA
)
, Edmond; Tracy
(Morgan Hills, CA
)
Correspondence Name and Address:
William R. Bachand Two Renaissance Square 40 N. Central Avenue Phoenix Arizona
SQUIRE, SANDERS & DEMPSEY, L.L.P.
85004
(602) 528-4000
(602) 859-8129
Series Code:
120266
Filed:
October 18, 2001
U.S. Current Class:
370/389;
370/400
U.S. Class at Publication:
370/389;
370/400
Intern'l Class:
H04L 012/28
Claims
Claims
1. A method performed by a router for routing frames in a network, the router comprising a managing processor and a plurality of routing processors, each routing processor for receiving frames from the network, for routing frames to the network, and for routing frames to the managing processor, the method performed by the managing processor comprising:a step for executing a plurality of proxy processes, each proxy process associated with a destination of a set of destinations used for communication between the managing processor and the plurality of routing processors;a step for receiving from a routing processor of the plurality a frame comprising indicia of a particular destination of the set;a step for parsing the frame to determine a particular proxy process of the plurality in accordance with the indicia of the particular destination by parsing the frame; anda step for providing at least a payload of the frame to the particular proxy process.
2. The method of claim 1 wherein the indicia of the destination comprises a network address reserved to the router.
3. The method of claim 1 wherein the managing processor comprises a port for receiving frames, each frame comprising indicia of any destination of the set.
4. The method of claim 1 wherein each proxy process performs a step for receiving exclusively frames comprising indicia of one destination of the set.
5. The method of claim 1 wherein each indicia of destination comprises a network address.
6. The method of claim 5 wherein:the network address comprises a domain value, an area value, and a port value and each member of the set comprises the same domain value and the same area value.
7. The method of claim 1 wherein each indicia of destination comprises a virtual address.
8. The method of claim 1 wherein each proxy process performs a step for maintaining a state of a respective virtual target.
9. A method performed by a router for routing frames in a network, the router comprising a managing processor and a plurality of routing processors, each routing processor for receiving frames from the network, for routing frames to the network, and for routing frames to the managing processor, the method performed by the managing processor comprising:a step for maintaining an association between indicia of a first destination and indicia of a second destination, the indicia of first destination being a member of a set used for communication between the managing processor and a routing processor of the plurality;a step for receiving from the routing processor a first frame of a first transaction comprising indicia of the first destination;a step for determining the indicia of the second destination by parsing the first frame and referring to the association;a step for passing a second frame to the routing processor for routing to the network to initiate a second transaction, the second frame comprising indicia of the second destination.
10. The method of claim 9 wherein the indicia of the first destination comprises a virtual address.
11. The method of claim 9 wherein the indicia of the second destination is a nonvirtual address.
12. The method of claim 9 wherein the set consists of network addresses identified to the router.
13. The method of claim 9 wherein:the router includes a distributing circuit; and the first frame and the second frame are communicated between the routing processor and the managing processor via the distributing circuit.
14. The method of claim 9 wherein the first frame conveys a payload for a SCSI non-I/O R/W virtual transaction.
15. The method of claim 14 wherein the second frame conveys the payload for a nonvirtual transaction.
16. A method performed by a router for routing frames in a network, the router comprising a managing processor and a plurality of routing processors, each routing processor for receiving frames from the network, for routing frames to the network, and for routing frames to the managing processor, the network comprising a nonvirtual target having a first page and a second page, the method performed by the managing processor comprising:a step for receiving from the routing processor a first frame for accessing a page of a virtual target;a step for passing to the routing processor a second frame identified to the nonvirtual target for accessing a portion of the first page; anda step for passing to the routing processor a third frame identified to the nonvirtual target for accessing a portion of the second page.
17. The method of claim 16 wherein the first frame comprises a virtual address.
18. The method of claim 16 wherein the first frame is part of a virtual transaction to read data from the virtual target.
19. The method of claim 16 wherein the first frame is part of a virtual transaction to write data to the virtual target.
20. The method of claim 16 wherein the first page consecutively precedes the second page to define a page boundary between the first page and the second page.
21. A router for routing frames in a network, the router comprising:a managing processor;a plurality of routing processors, each routing processor for receiving frames from the network, for routing frames to the network, and for routing frames to the managing processor;means for executing a plurality of proxy processes, each proxy process associated with a destination of a set of destinations used for communication between the managing processor and the plurality of routing processors;means for receiving from a routing processor of the plurality a frame comprising indicia of a particular destination of the set;means for determining a particular proxy process of the plurality in accordance with the indicia of the particular destination; andmeans for providing at least a payload of the frame to the particular proxy process.
22. The router of claim 21 wherein the indicia of the destination comprises a network address reserved to the router.
23. The router of claim 21 wherein the managing processor comprises a port for receiving frames, each frame comprising indicia of any destination of the set.
24. The router of claim 21 wherein each proxy process performs a step for receiving exclusively frames comprising indicia of one destination of the set.
25. The router of claim 21 wherein each indicia of destination comprises a network address.
26. The router of claim 25 wherein network address comprises a domain value, an area value, and a port value and each member of the set comprises the same domain value and the same area value.
27. The router of claim 21 wherein each indicia of destination comprises a virtual address.
28. The router of claim 21 wherein each proxy process performs a step for maintaining a state of a respective virtual target.
29. A router for routing frames in a network, the router comprising:a managing processor;a plurality of routing processors, each routing processor for receiving frames from the network, for routing frames to the network, and for routing frames to the managing processor;means for maintaining an association between indicia of a first destination and indicia of a second destination, the indicia of first destination being a member of a set used for communication between the managing processor and a routing processor of the plurality;means for receiving from the routing processor a first frame of a first transaction comprising indicia of the first destination;means for determining the indicia of the second destination by parsing the first frame and referring to the association; andmeans for initiating a second transaction by passing a second frame to the routing processor for routing to the network, the second frame comprising indicia of the second destination.
30. The router of claim 29 wherein the indicia of the first destination comprises a virtual address.
31. The router of claim 29 wherein the indicia of the second destination is a nonvirtual address.
32. The router of claim 29 wherein the set consists of network addresses identified to the router
33. The router of claim 29 wherein:the router further includes a distributing circuit; and the first frame and the second frame are communicated between the routing processor and the managing processor via the distributing circuit.
34. The router of claim 29 wherein the first frame conveys a payload for a virtual data transaction.
35. The router of claim 34 wherein the second frame conveys the payload for a nonvirtual data transaction.
36. A router for routing frames in a network, the network including a nonvirtual target having a first page and a second page, the router comprising:a managing processor;a plurality of routing processors, each routing processor for receiving frames from the network, for routing frames to the network, and for routing frames to the managing processor;means for receiving from a particular routing processor of the plurality a first frame for accessing a page of a virtual target;means for passing to the particular routing processor a second frame identified to the nonvirtual target for accessing a portion of the first page; andmeans for passing to the particular routing processor a third frame identified to the nonvirtual target for accessing a portion of the second page.
37. The router of claim 36 wherein the first frame comprises a virtual address.
38. The router of claim 36 wherein the first frame is part of a virtual transaction to read data from the virtual target
39. The router of claim 36 wherein the first frame is part of a virtual transaction to write data to the virtual target.
40. The router of claim 36 wherein the first page consecutively precedes the second page to define a page boundary between the first page and the second page.
41. The method of claim 8 wherein the particular proxy process further performs a step for completing a function of a SCSI protocol with respect to the target.
42. The method of claim 1 wherein the payload is part of a virtual transaction.
43. The method of claim 42 wherein the payload is part of a control frame.
44. The method of claim 1 wherein the payload is part of a SCSI non-I/O R/W transaction affecting the state of a virtual target.
45. The method of claim 9 wherein the first transaction comprises a control frame.
46. The method of claim 45 wherein the first transaction comprises a virtual control transaction.
47. The method of claim 9 wherein the first transaction comprises a SCSI non-I/O R/W transaction affecting the state of a virtual target.
48. The method of claim 9 wherein the second transaction comprises a control frame.
49. The method of claim 48 wherein the second transaction comprises a nonvirtual control transaction.
50. The method of claim 9 wherein the second transaction comprises a SCSI non-I/O R/W transaction affecting the state of a virtual target.
51. The method of claim 20 performed by a router further including a method, performed by a routing processor of the plurality, comprising:a step for referring to a predetermined page size to determine that the page of the virtual target crosses the page boundary of the nonvirtual target.
52. The method of claim 51 wherein the method performed by the managing processor further comprises:a step for transferring indicia of the page size to at least one routing processor of the plurality.
53. The method of claim 16 wherein the first frame comprises a data frame.
54. The method of claim 53 wherein the first frame is part of a virtual data transaction.
55. The method of claim 54 wherein the data transaction comprises a SCSI I/O R/W transaction.
56. The method of claim 16 wherein the second frame comprises a data frame.
57. The method of claim 56 wherein the second frame is part of a nonvirtual data transaction.
58. The method of claim 57 wherein the data transaction comprises a SCSI I/O R/W transaction.
59. The router of claim 28 wherein the particular proxy process further performs a step for completing a function of a SCSI protocol with respect to the target.
60. The router of claim 21 wherein the payload is part of a virtual transaction.
61. The router of claim 60 wherein the payload is part of a control frame.
62. The router of claim 21 wherein the payload is part of a SCSI non-I/O R/W transaction affecting the state of a virtual target
63. The router of claim 29 wherein the first transaction comprises a control frame.
64. The router of claim 63 wherein the first transaction comprises a virtual control transaction.
65. The router of claim 29 wherein the first transaction comprises a SCSI non-I/O R/W transaction affecting the state of a virtual target.
66. The router of claim 29 wherein the second transaction comprises a control frame.
67. The router of claim 66 wherein the second transaction comprises a nonvirtual control transaction.
68. The router of claim 29 wherein the second transaction comprises a SCSI non-I/O R/W transaction affecting the state of a virtual target.
69. The router of claim 40 further comprising means in the particular routing processor for determining that the page of the virtual target crosses the page boundary of the nonvirtual target by referring to a predetermined page size.
70. The router of claim 61 further comprising means for transferring indicia of the page size to at least one routing processor of the plurality.
71. A router for routing frames in a network, the router comprising:a first means for processing processor that performs a proxy process for a virtual member of the network, the proxy process responsive to a frame addressed in accordance with a first network address, the virtual member corresponding to at least a portion of a nonvirtual member of the network, the nonvirtual member responsive to frames addressed in accordance with a second network address; anda second means for processing that performs a routing process that routes frames addressed in accordance with the first network address and of a first type to the proxy process; that in response to receiving frames in accordance with the first network address and of a second type, provides a corresponding frame revised in accordance with the second network address; and that routes the revised frames to the nonvirtual member in accordance with the second network address.
72. The router of claim 71 wherein the second means for processing comprises:means for storing;means for parsing;a plurality of queues that receive data enqueued by the means for parsing; andmeans for submitting that submits data from a queue of the plurality to the means for storing to provide a result; whereinthe second means for processing determines the second network address in accordance with the result.
73. The router of claim 72 wherein:the means for storing provides a first result in accordance with first data submitted from the means for parsing, the first data comprising a flag; andthe means for storing provides the result in accordance with second data, the second data submitted from the second means for processing, in accordance with the flag.
74. The router of claim 73 wherein the result comprises at least one of a resource identifier, a network address of a proxy, and logical unit number.
75. The router of claim 72 wherein the first means for processing stores indicia of the second network address in a routing table in the means for storing.
76. The router of claim 75 wherein the first means for processing further stores indicia of a resource identifier in association with the indicia of the second network address, and the second means for processing routes frames in accordance with the indicia of the resource identifier.
77. The router of claim 76 wherein the resource identifier comprises a logical unit number
78. The router of claim 71 further comprising:a third means for processing that provides first data to the first means for processing; wherein:the first means for processing further provides the first data to the second means for processing; andthe second means for processing distinguishes the first type from the second type in accordance with the first data.
79. The router of claim 78 wherein the first data further comprises routing information.
80. A router for use in a network, the router comprising:means for participating as a virtual target in a virtual transaction initiated by an initiator of the network; andmeans for initiating a nonvirtual transaction with a target of the network to accomplish an intent of the virtual transaction.
81. The router of claim 80 wherein the target is at least one of a member and a resource.
82. The router of claim 80 wherein the virtual target is at least one of a virtual member and a virtual resource.
83. The router of claim 80 wherein the initiator is at least one of a nonvirtual member a nonvirtual resource, a virtual member, and a virtual resource.
84. A router for use in a network including a virtual member associated with first network port identifier and including a nonvirtual member corresponding to the virtual member, the nonvirtual member associated with a second network port identifier, the router comprising:a first means for processing that performs a proxy process for the virtual member, wherein the proxy process responds to a control frame; anda second means for processing that performs a routing process that routes the control frame to the proxy process in accordance with the first network port identifier; and, on receiving a data frame comprising the first network port identifier, routes to the nonvirtual member a substitute data frame in accordance with the second network port identifier.
85. _The router of claim 84 wherein:the control frame comprises the first network port identifier as a destination address;the data frame comprises the first network port identifier as a destination address; andthe substitute frame comprises the second network port identifier as a destination address.
Description
Background of Invention
[0001] FIELD OF THE INVENTION
[0002] Embodiments of the present invention relate to improved networks having routers that perform routing functions and to methods for routing network traffic.
[0003] In a conventional network, data is transferred between computers and peripherals to accomplish the data processing demands of the computers and peripherals. Demands for data to be transferred via the network may arise in any particular computer or peripheral in a manner unsynchronized with demands that arise on other computers and peripherals of the network. Data transfer to accomplish delivery is generally between respective ports of the computers and peripherals and may pass through switches having ports as well. Such switches have numerous ports and generally retransmit data (also called routing network traffic) from one port to another according to address information associated with the data to be transferred. A pair of ports communicate via a link between the ports.
[0004] Demands generally vary widely in the amount of data to be delivered over the network and the manner in which the delivery is to be made. For example, some demands may be made for a relatively large amount of data without regard to the order in which the data is delivered via the network. Other demands may require that the data be delivered in a particular order. Some demands may have no use for data that is presented outside of an expected time for delivery. Other demands may be met at any time, though system efficiency may suffer if delivery is made outside of an expected time for delivery.
[0005] With a large number of network links, use of the network may be regulated to some extent by establishing a priority for each link. In particular, when attempts to meet demands result in delivery of data in bursts between pairs of computers and/or peripherals, network performance may exhibit several undesirable results. Network capacity (sometimes colloquially referred to as bandwidth) for servicing lower priority links may be unavailable. Delivery of data may be noticeably delayed. More out of order deliveries may be made. And, service between ports on particular links may be denied intermittently, causing queues to fill and network capacity to be used for overhead messages regarding the control of network traffic as opposed to actually routing the traffic.
[0006] Traditional approaches to improving a network's ability to deliver data which would otherwise be delivered in bursts and to decreasing the likelihood of the undesirable results described above have focused on increasing network data transfer speed, increasing the depth of queues for data awaiting processing before or after transfer via the network, and increasing the instruction processing speed for processors (e.g., per-port processors) that accomplish delivery over the network. In a conventional architecture, each port may be implemented with a processor and memory dedicated to servicing all forms of traffic for that port.
[0007] In another known approach to solving some of the problems discussed above, a traffic stream having a traffic profile is affected by provisioning a facility for traffic conditioning as described in Request For Comment "An Architecture for Differentiated Services,"RFC2475 by S. Blake of Torrent Networking Technologies. A traffic profile is a set of desired temporal properties for a traffic stream (i.e., packet rate and burst size). A traffic stream is an administratively significant set of microflows that traverse a path segment as selected by a particular classifier. Provisioning includes mapping traffic streams to per hop behaviors, and specifying methods of traffic conditioning. Per hop behaviors are effected by shaping. Traffic conditioning is defined as classifying, metering, marking, shaping, and dropping packets. A microflow classifier selects packets (e.g., for marking) based on an arbitrary number of header fields including source address, destination address, protocol (e.g., IP), fields (e.g., DS field in IP header), source port, and destination port. Marking is defined (for IP) as setting the value of the DS field. Metering is defined as measuring temporal properties of a traffic stream. Shaping is delaying packets to conform a traffic stream to a desired traffic profile. Shaping includes enqueueing a marked packet and holding the packet in queue until transmitting the packet would not exceed a desired traffic profile. The basic architecture assumes that traffic conditioning functions are accomplished at each ingress and egress node (i.e., at each port of an edge node) of the network. According to a first conventional hardware architecture, all traffic conditioning functions would be accomplished by a central processing unit (CPU) serving a group of ports at an ingress and egress node. Such a CPU would not be capable of significant bandwidth. According to a second conventional hardware architecture, each port of an edge node would be implemented with a processor and memory dedicated to performing traffic conditioning functions by servicing all forms of traffic for that port.
[0008] A large portion of network traffic is associated with reading or writing data storage media. The data delivery problems described above are evident in networks that provide shared access to data storage devices. Managing data for improved access according to traditional approaches has included introducing servers between data storage devices and the network. Such server technology impedes network traffic flow, and may facilitate unexpected denial of access or damage to data due to failure mechanisms with a single point of failure.
[0009] Without the present invention, data delivery cannot be further improved without unreasonably increasing the cost per port of the network and the computers and peripherals that use the network. Increased costs stem from increased memory for queues and sophisticated processing instructions to be executed by the port processors, from increased processing speed, and from circuits that operate at higher frequencies to provide increased network data transfer speed. The comparatively high cost of circuits that operate at increased frequency stems from difficulties in designing such circuits and difficulties in fabrication.
Summary of Invention
[0010] A router, in one embodiment of the present invention, routes frames in a network. The router includes means for participating as a virtual target in a virtual transaction initiated by an initiator of the network and means for initiating a nonvirtual transaction with a target of the network to accomplish an intent of the virtual transaction.
[0011] By analyzing at least a portion of a received frame, and preparing an outbound frame back to the requester, a router operating according to various aspects of the present invention provides a logical interface between the requester and resources. An additional outbound frame to a resource may be prepared by the router to fulfill the request. A logical interface facilitates management of the resources for improved efficiency and reliability of data transfers; and, supports demanding levels of quality of service as to order and timeliness of deliveries.
[0012] In another embodiment a router includes a processor that stores a virtual resource identifier and routes a frame that includes indicia of a nonvirtual resource identifier. The nonvirtual resource identifier may be determined by the processor with reference to an association between the nonvirtual resource identifier and the virtual resource identifier. The association may be made by an administrating process and communicated to the processor as routing information.
[0013] A router, in another embodiment of the present invention, includes a processor that stores a resource identifier determined from a first frame and routes a second frame in accordance with the resource identifier. For example, the second frame may be received without indication of the resource identifier and received after the first frame is received.
[0014] In another embodiment of the present invention, a router includes a processor that routes a frame in accordance with a policy value to implement a quality of service. The policy value is determined at least in part by parsing the frame to determine a resource identifier and recalling an association of the policy value and indicia of the resource identifier. The association may be made by an administrating process and communicated to the processor as routing information.
[0015] By analyzing at least a portion of a received frame, and identifying more than one field value, a router operating according to various aspects of the present invention selectively controls the quality of service as applied to particular data transfers and frames having particular sets of field values. Quality of service may effectively be controlled for a predetermined protocol and/or predetermined group of resources. Quality of service may include specifications regarding order and timeliness of deliveries, or in other words, bandwidth allocation, maximum delays, and reduction in network congestion. Statistics may be collected and analyzed for a subflow.
[0016] A router, in another embodiment of the present invention, includes a managing processor, a supervising processor, and a routing processor. The managing processor performs a proxy process that responds to a control frame directed to a virtual entity. The supervising processor performs a control process that responds to a control frame directed to the router. The routing processor routes data frames directed respectively to virtual and to nonvirtual entities via the network.
[0017] A router, in another embodiment of the present invention, includes two processors. The first processor performs a proxy process for a virtual member of the network. The proxy process responds to a control frame having a first network port identifier. The virtual member corresponds to at least one nonvirtual member or resource of the network. The nonvirtual member responds to a data frame having a second (i.e., different) network port identifier. The second processor performs a routing process that routes frames having the first network port identifier to the proxy process, routes frames having the second network port identifier to the nonvirtual member, and on receiving a data frame having the first network port identifier, routes a substitute data frame having the second network port identifier. For example, data frames originally addressed to the virtual member are readdressed and routed to a corresponding nonvirtual (e.g., actual) member.
[0018] The modular architecture provided according to various aspects of the present invention permits scaling of the router design and scaling of the network, lowering the cost for competitive router products and improving network maintenance.
[0019] A router, in another embodiment of the present invention, routes a frame received from a network. The router includes a routing processor. The routing processor includes: a frame processor, a parser, a plurality of queues, a submitter, and a memory circuit. The parser prepares a flow lookup in response to the frame received from the network. The memory circuit performs a flow lookup and provides a result as directed by the submitter and a first entry in a first queue, the first entry having been enqueued by the parser. The memory circuit also performs a subflow lookup and provides a result as directed by the submitter and a second entry, the second entry having been enqueued in a second queue by the frame processor in accordance with the result of the flow lookup. The frame processor routes the frame in accordance with the result of the subflow lookup.
[0020] A router, in another embodiment of the present invention, includes: a plurality of physical ports, a managing processor, and at least one routing circuit coupled to the manager by a first bus. Each routing circuit includes: a supervising processor, a memory, a second bus, and a plurality of port logic circuits. The memory includes indicia of a routing table. The memory is coupled to the supervising processor by the second bus. The plurality of port logic circuits is coupled to the supervising processor by a third bus. Each port logic circuit provides a multiplicity of the physical ports. Each port logic circuit is coupled to other port logic circuits for data transfer between physical ports. At least one physical port of the plurality is coupled to the managing processor.
[0021] By providing in-band access to the managing processor, virtualization functions are less complex and more efficient. Wire speed virtualization is facilitated.
[0022] A router, in another embodiment of the present invention, includes: a plurality of physical ports; a managing processor having a first memory; and at least one routing circuit coupled to the managing processor by a first bus. Each routing circuit includes a supervising processor and a plurality of port logic circuits. The supervising processor has a second memory. Each routing circuit further includes a third memory. The third memory includes indicia of a routing table. The third memory is coupled to the supervising processor by a second bus. The plurality of port logic circuits are coupled to the supervising processor by a third bus. Each port logic circuit provides a multiplicity of the physical ports. Each port logic circuit is coupled to other port logic circuits for data transfer between physical ports. Each port logic circuit has a frame processor that includes a respective fourth memory. The managing processor updates the second memory via the first bus. The supervising processor updates the third memory and the fourth memory via the second bus.
[0023] By loading and updating routing information tailored to particular frame processors and tailored to particular routing processors, the computational burden of performing virtualization functions may be distributed among routers of a network.
[0024] A method, in another embodiment of the present invention, is performed by a router for routing frames in a network. The router includes a plurality of network ports, a fabric, and a plurality of routing processors coupled between the fabric and the network ports. Each routing processor includes an ingress buffer for receiving frames from a network port and for transmitting frames to the fabric; an egress buffer for receiving frames from the fabric and for transmitting frames to the network port; and a frame processor. On receiving from a requester a data frame directed to a virtual participant, the frame processor modifies the data frame in the ingress buffer for routing to a nonvirtual participant. On receiving from the fabric a data frame not directed to a nonvirtual requester, the frame processor modifies the data frame in the egress buffer for routing to a nonvirtual requester. Further, the frame processor may, on receiving from the fabric a data frame not directed to a nonvirtual requester for which the frame processor does not have sufficient modification information, route the data frame via the fabric to another routing processor of the plurality.
[0025] A router, in another embodiment of the present invention, includes a first routing processor and a second routing processor and a fabric. Each routing processor includes: an ingress buffer coupled to an input port, an egress buffer coupled to an output port, a parser, and a memory that stores routing information. The ingress buffer is coupled between the input port and the fabric to transfer frames from the ingress buffer to the fabric. The egress buffer is coupled between the fabric and the output port to transfer frames from the fabric to the output port. The first routing processor parses a frame received from its input port to determine a virtual destination identifier, determines a nonvirtual transaction identifier in response to the virtual destination identifier, prepares a second frame having the nonvirtual transaction identifier, and transmits the second frame to the fabric. The second routing processor receives the transmitted second frame from the fabric and transmits the second frame to its output port. The second processor, on receiving a third frame on its input port parses the third frame to determine a nonvirtual transaction identifier, marks the third frame for modification, and transmits the third frame to the fabric. The first processor receives the transmitted third frame from the fabric, parses the third frame to access the routing information from its memory, modifies the third frame in accordance with the accessed routing information, and transmits the modified frame from its output port.
[0026] By operating on frames in the ingress buffer and egress buffer, a lower complexity router design results. For example, less memory is needed for maintaining virtual context tables.
[0027] A router, in another embodiment of the present invention, includes a plurality of ports and a routing processor. The routing processor includes: at least a portion of a fabric, an ingress buffer, an egress buffer, The ingress buffer is coupled between the fabric and a first port of the plurality to transfer frames from the fabric to the first port. The egress buffer includes a plurality of queues, an arbitrating circuit coupled between the egress buffer and the first port, and a counter associated with each queue. Each counter has a respective current count. The arbitrating circuit (a) adds received grants to a grant pool for the plurality of queues; (b) transfers a frame from a selected queue to the fabric when sufficient grants exist in the grant pool; (c) decrements the grant pool in response to the transfer; (d) adds transmitted frame size to the counter associated with the selected queue; and tests whether the counter associated with the selected queue is greater than a threshold. If so, the arbitrating circuit: (a) sets an overrun amount to the current count of the counter associated with the selected queue; (b) resets the counter associated with the selected queue; (c) subtracts the overrun amount from a current count of each other counter; (d) clears all asserted stalled flags; and (e) stalls the selected queue.
[0028] A router, in another embodiment of the present invention, includes a plurality of routing processors each having at least a portion of a distributing circuit. Each distributing circuit portion has a crossbar switch that completes a plurality of point-to-point connections between routing processors. The crossbar switch operates in response to at least one of: an input that indicates a number of routing processors that have been installed, and an input that indicates a position of the routing processor among the number of routing processors. A second crossbar switch may provide a termination for a point-to-point connection according to at least one of: an input that indicates a number of routing processors that have been installed, and an input that indicates a position of the routing processor among the number of routing processors.
[0029] Combinations of the various aspects of the present invention provide solutions to the problems described in the background section and mitigate other problems. For example, stall and continue capabilities on a subflow basis accommodate bursty network traffic from various applications sharing a network link. Further, accommodating quality of service differences (e.g., in the time or ordering of data) on a subflow basis better accommodates performance variations among processes and storage functions in any member or within the network (e.g., an interswitch link). A router operating according to various aspects of the present invention efficiently allocates bandwidth without completely stalling a low priority flow or unreasonably fragmenting a high priority flow. Routers that provide a logical resource interface provide more efficient and more reliable networks for application service providers and storage service providers, thereby lowering the cost of operating and lowering the cost of these services to the consumer.
[0030] According to various aspects of the present invention, sophisticated network functions are accomplished without a general purpose processor per port. Such functions include, inter alia, mirroring, third party copy, arbitration based on subflows, subflow stalls, statistics gathering, provision of a logical resource interface, and maintaining caches in the router for read and write operations.
[0031] By maintaining one or more pointers to the original copy of a snapshot and possibly to revised portions of the snapshot, the time to initially support use of a snapshot may be reduced and the interruption due to taking time to prepare a full copy of the snapshot may be avoided.
[0032] By maintaining a cache in the router, more efficient data transfer to a member of the network results. Egress from the cache is provided to meet the needs of the resource as opposed to the resource being forced to accommodate operation of the network or operation of another network member.
[0033] By maintaining a cache in the router, a multicast write is accomplished with fewer data transfers. More efficient network operation results.
[0034] In a router architecture according to various aspects of the present invention, memory is provided where it can be effectively used and the cost of router circuits can be decreased by avoiding large amounts of memory that are infrequently accessed. Operations limit the need to synchronize redundant copies of information in separate parallel processors within the router. Such an architecture supports frame disposition at the maximum rate on all ports and full mesh connectivity at wire speed. Routers based on scaling and reusable design (e.g., a reconfigurable full mesh circuit) help control the overall router cost and reduce dependency on higher cost processors and memory. Furthermore, routers with different quantities of ports may be economically assembled with a greater reliance on common designs and subassemblies, lowering the cost of manufacturing.
[0035] According to various aspects of the present invention, processors that are in the data path execute frame preparation functions with reference to commands and information prepared by processors that are not in the data path. Such functions include, for example, access control from a centralized administration processor; providing security from rogue processes (e.g., identifier translation tables (e.g., used for resource mapping or frame routing) are not directly accessible from the port interface); or gathering statistics on a subflow so that control decisions may be based on use of the network by a particular type of process (e.g., Virtual Interface (VI) communication having priority over SCSI communication from the same port of the network) or a particular type of storage device (e.g., streaming audio access having priority over data processing transactional file access). A managing processor in a router may filter statistics and more efficiently report to an administrating processor for management of virtual resources.
[0036] An administrating processor updates the configuration (e.g., routing tables) of several routers uniformly. An administrating process may assign network port identifiers to be used for virtual members, virtual resources, and proxy processes. Proxy processes may receive control frames for a virtual member or virtual resource.
[0037] As router products are developed with varying need for processing, the ratio of the various processors to the number of ports may be economically scaled while continuing to benefit from the investment in circuit and firmware design. The following are but a few examples. The number of buses made active in the mesh may scale with the quantity and bandwidth of the ports. Port protocol support may be downloaded to the processor(s) responsible for particular ports. Supervisory processing may scale with the quantity of ports in part due to the bus interface between a plurality of port processing slices and the supervisory processor(s). Managing processor(s) scale with the number of ports in part due to use of one or more in-band links to the supervising processor(s). Processing responsibility scales with the amount of available memory due in part to the shared nature of memory between port logic circuits. RAID device control may be implemented at the device cluster level from processor(s) in one or more routers or from processor(s) that is(are) part of a member. Multiple protocol capability scales with different demands for different protocols. Multiple zone capability for load balancing scales by performance and extent of physical, logical, and virtual resources.
[0038] A router according to various aspects of the present invention detects in a virtual data frame a page boundary crossing, initiates nonvirtual data frames to accomplish the operation intended, and routes the nonvirtual data frames to corresponding nonvirtual storage. A page boundary crossing occurs, for example, when reference is made in a data frame to a portion of a virtual storage device, and the reference when mapped to nonvirtual storage would include more than one page of one or more nonvirtual storage devices.
[0039] By detecting page boundary crossings and initiating data frames, a requester may operate on a virtual resource without knowledge of the structure and organization of the corresponding nonvirtual resource, simplifying such operations from the point of view of the requester.
[0040]
Brief Description of Drawings
[0041] Embodiments of the present invention will now be further described with reference to the drawing, wherein like designations denote like elements, and:
[0042] FIG. 1 is a functional block diagram of a system according to various aspects of the present invention;
[0043] FIG. 2 is a data flow diagram of processes in the system of FIG. 1;
[0044] FIG. 3 is a data flow diagram of the administrating process of FIG. 2;
[0045] FIG. 4 is a data flow diagram of the managing process of FIG. 2;
[0046] FIG. 5 is a data flow diagram of the supervising process of FIG. 2;
[0047] FIG. 6 is a data flow diagram of the routing process of FIG. 2;
[0048] FIGs. 7, 8, 9, and 10 form a flow chart of a method for routing according to various aspects of the present invention;
[0049] FIG. 11 is a functional block diagram of a router of the system of FIG. 1;
[0050] FIG. 12 is a functional block diagram of the supervising processor of FIG. 11;
[0051] FIG. 13 is a functional block diagram of the memory circuit of FIG. 11;
[0052] FIG. 14 is a functional block diagram of the port logic circuit of FIG. 11;
[0053] FIG. 15 is a functional block diagram of the descriptors of FIG. 14;
[0054] FIG. 16 is a message sequence diagram for operations performed by the system of FIG. 1;
[0055] FIGs. 17-20 form a flow chart of methods performed by router 102 of FIG. 2;
[0056] FIG. 21 is a functional block diagram of a fabric having five fabric nodes according to various aspects of the present invention;
[0057] FIG. 22 is a functional block diagram of the fabric of FIG. 20
implemented with three fabric nodes;
[0058] FIG. 23 is a functional block diagram of the distributing circuit of FIG. 11; and
[0059] FIG. 24 is a message sequence diagram for operations performed by the system of FIG. 1.
Detailed Description
[0060] A system according to various aspects of the present invention may include any computing environment supporting transfer of data among computer systems via a communication network. Such a system, in one implementation, provides more efficient non-blocking delivery of data, improved utilization of bandwidth, a facility for managing network traffic flows, subflows, and virtual flows, and higher quality of service. Data may be transferred between application programs being executed by one or more of the computer systems, between an application program and a data storage device, or between one or more data storage devices.
[0061] The network may be understood as a graph or a tree having network nodes. A communication network of the present invention includes at least one computer system at each of several network nodes. Each network node is coupled by a link from time to time for communication with other network nodes. Each link includes conventional computer communication technology at the physical layer and primitive layers of the type including, for example, local area, wide area, dedicated telephone, wireless, and satellite services and including conventional data communication hardware and software at each network node. The popular computer networks known as storage area networks, intranets, the Internet, the World Wide Web, and the National Information Infrastructure are examples of communication networks in which various aspects of the present invention may be practiced. Network nodes are generally at physically separate locations and are generally suitably identified, for example, by a node name, node identifier, node address, a world wide identifier (WWPN), a uniform resource locator (URL), a name from a domain name system (DNS), or an Internet Protocol address (IP).
[0062] Data transfer at the lowest level occurs via a link between ports, nominally a requesting port and a participating port, where a requesting port requests a data transfer and the participating port either supplies the data (e.g., a read) or receives the data being transferred (e.g., a write). A port includes a physical implementation for common signaling between ports (e.g., any circuitry suitable for the transfer media); and a logical implementation (e.g., any combination of firmware and software). Cooperation between ports occurs in accordance with a physical protocol (e.g., signals and their characteristics) and a logical protocol (e.g., one or more layers of application program interfaces). The physical and logical implementations and protocols together constitute a port by which other software can manage, among other things, how to obtain the data to be supplied to the port and what to do with the data obtained from the port. A port may communicate using several protocols. Frames according to a first protocol may be encapsulated (i.e., become the payload) in frames according to another protocol. Ports of a router according to various aspects of the present invention may support, for example, combinations of Fibre Channel (FC) Protocol (FCP), Internet Protocol (IP), based IEEE 802.3 Ethernet protocol, Small Computer Systems Interface (SCSI) Parallel Interface, Serial Bus Protocol, IEEE 1384 (Fire wire), SSA SCSI-3 Protocol, Scheduled Transfer, and Virtual Interface (VI).
[0063] A network node may include one or more ports. Multiple ports at a network node may be serviced as a group (e.g., a hunt group) to serve an upper level process with higher band width, to provide fail-over capability, or to serve multiple parallel processes. Network node identifiers (e.g., port identifiers) facilitate requesting (e.g., initiating) and participating in a data transfer (e.g., performing as a target or as a virtual target).
[0064] A group of ports may provide data transfer functions transparently. For example, a bridge, located between a requester and a participant, may receive requests in a first protocol (e.g., not understood by the participant) and provide a corresponding request to the participant in a second protocol (e.g., not understood by the requester). Further, a router, located anywhere in the network, may serve as a hub for several links; each link being served by one or more ports. Such a router routes network traffic between a requester (having a requesting port) and a participant (having a participating port) without either the requester or the participant having knowledge of the port identifiers of the ports of the router. The router transfers traffic between its ports in accordance with a routing table that defines communication paths through the router. The routing table may be specified by a network technician, by an administrator as discussed below, or may be determined by the router as a result of communication with other routers to which it is linked. A router according to various aspects of the present invention may function as a gateway receiving frames at an input port according to a first protocol and forwarding frames to an output port that (a) encapsulate the input payload; (b) strip the encapsulation of an input frame and forward the payload to the output port; or (c) use frames of the second protocol to conduct the function intended by the first protocol (e.g., data transfer with a virtual destination or with a logical destination; or a Virtual Interface transaction to a SCSI transaction).
[0065] A system according to various aspects of the present invention includes a communication network and numerous computer systems. Any of the computer systems that are currently members of the communication network, may transfer data to any other computer systems that are members of the communication network (or will be at a suitable future time) via links through routers. For example, system 100 of Figs. 1-6, 11-14, and 23 includes communication network 101 (i.e., network 101) and members 110-117. Network 101 includes a link to each member: respectively links 150-158 to members 110-117. Network 101 also includes routers 102-105. The quantity, configuration, and arrangement of members, links, and routers in system 100 is merely illustrative and any number, configuration, and arrangement may be used in practice of the various aspects of the present invention.
[0066] Practice of variations of the present invention is independent of whether any particular link is maintained continuously, as in a dedicated line, or is maintained for a suitable duration. Members, links, and routers may each incorporate multiple units and be organized to provide redundancy or fail-over capacity to avoid a single failure from disrupting communication.
[0067] System administration includes establishing and maintaining router configuration for some or all routers of a network as the utilization of the network changes, as link reliability changes, and as the network grows or shrinks in number of links, routers, and members. Information for manual or dynamic network administration may be collected and reported by routers of the network. Administration may be accomplished by use of one or more workstations (e.g., for a human operator) or servers (e.g., for administration directed by process(es) running on the servers). Network 101 includes administration subsystem 109 having port 106. Link 107 supports communication between administration subsystem 109 and any router of network 101, particularly connecting port 106 to port 108 of router 102. Ports 106 and 108 and link 107 may be identical in structure and function to links and ports described above with reference to routers and members. In an alternate implementation, administration may be accomplished by any suitable member.
[0068] System administration may include management of network topology and may include management of virtualization. Virtualization includes the designation (e.g., mapping) of a nonvirtual member or nonvirtual resource (e.g., a nonvirtual entity) to be used in place of any reference to a virtual member or to a virtual resource (e.g., a virtual entity), the communication of that designation to suitable routers, and the use of that designation in routing packets. Routers according to various aspects of the present invention may perform the communication and use of designations that are defined by system administration.
[0069] A member of a network is a computer system that communicates via a link as described above and either operates, inter alia, to request data transfer or to participate in data transfer via the link. Some members may provide a resource to network 101 so that all members of the network may share the capability of the resource. For example, members 110-119
may include all or any part of the structure and functions described below with reference to members 115-116. Members 115-116 are capable of requesting data from any other member 110-117 or participating in data transfer with any other member 110-117 and vice versa.
[0070] Any member may include a subnetwork. A subnetwork includes any subsystem that employs ports connected to network 101 for communication generally between any member of network 101 and any subnetwork member (e.g., a resource) that is not directly connected to network 101. The interface between network 101 and such a subsystem may provide redundancy, fail-over, multiple or expanded use of network ports, access controls, security (e.g., functions of a conventional firewall), protocol conversion (e.g., functions of a bridge), and/or priority flow controls (e.g., functions of a router as discussed herein). For example, member 115 includes subnetwork 170 having ports 165 and 166 connected to network 101 via links 155 and 156 respectively, a port interface 171, a resource interface 172, a controller 173 servicing interfaces 171 and 172, a plurality of resources 174 that includes a processing resource 175, and a storage resource 177. Port interface 171, resource interface 172, and controller 173 may cooperate as a server 178. The plurality of resources 174 may include zero or more processing devices (e.g., computers, servers, or workstations) and zero or more storage devices (e.g., disks, tapes, media handlers, or RAID systems).
[0071] Port interface 171 performs suitable port interface functions (e.g., signaling protocols) as described herein and is exemplary of ports 160-168 respectively of members 110-117. Port interface 171 is configured, directed, and controlled by controller 173. Any conventional status and command interface signaling may couple port interface 171 and controller 173. Resource interface 172 performs suitable interface functions (e.g., signaling protocols) to accomplish any conventional network functions for and among resources via subnetwork 170. In alternate implementations, interfaces 171 and 172 may be integrated as one interface, may operate in the absence of a controller 173, and/or may be integrated with one or more resources. Port interface 171 and resource interface 172 communicate over line 176 (e.g., a bus or a link).
[0072] Controller 173 accomplishes all conventional protocol functions not already implemented in port interface 171 and resource interface 172. Controller 173 may include memory used, for example, for programmable operations of controller 173, data buffering, stateful control of interfaces 171 and 172, and subnetwork communication. Controller 173
communicates with the plurality of resources 174 via subnetwork 170. Subnetwork 170 may include any conventional logical and physical organization. As shown, each resource 175 and 177 communicates with resource interface 172 via a dedicated link. Communication between resources and from resources to ports 165 and 166 is accomplished by the cooperation of interfaces 171, 172, and controller 173. Controller 173
may perform processing of the type known as Random Array of Independent Disks (RAID) for one or more storage devices 177. Controller 173 may perform routing and priority functions for fail-over and load sharing among processing devices (e.g., functioning as an application service provider) and/or analogous functions among storage devices (e.g., functioning as a storage service provider).
[0073] A resource provides any capability used with data communication. For example, processing device 175 and storage device 177 may include conventional computers, array processors, peripherals, personal computers, workstations, telecommunications equipment, disk drives, disk drive arrays, tape drives, tape drive arrays, printers, scanners, video displays and cameras, audio equipment, and measurement instrumentation. Generally, devices 175 and 177 and to some extent controller 173 provide functions described above as a resource to network 101.
[0074] Subnetwork 170 may be any conventional network (e.g., a LAN, SCSI network, Fibre Channel network, Integrated Drive Electronics (IDE) network, or a star interface to just a bunch of disks (JBOD)). Communication to and from resources 174 may refer to any suitable device identifiers (e.g., World-Wide Identifiers (WWPNs), logical unit numbers (LUNs), or device addresses).
[0075] A router includes any mechanism that provides a logical communication facility between a requester and one or more participants. The facility may be dedicated (e.g., independent of all other communication through the router) or shared (e.g., time multiplexed). When communication is accomplished by separating data into frames (also called packets), frames may be passed through the facility in order, out of order, with or without regard to a time period specified for transfer, repeated, or dropped. If the facility is of the type conventionally known as non-blocking, no frame that properly enters the router will be dropped. A facility that is non-blocking at full capacity will drop no frames while all its ports operate indefinitely at maximum continuous communication link capacity. A router according to various aspects of the present invention provides a virtual communication facility alone or in cooperation with other routers.
[0076] A frame that enters a router at a given port may exit the router at any one or more ports including the port from which it entered, as directed by router configuration (e.g., static paths and/or dynamic routing tables). For example, router 102 provides non-blocking communication among ports 108, 130-133 respectively supporting links 107, 150 and 121-123; router 103 provides non-blocking communication among ports 134-137 respectively supporting links 121, 151, 152, and 124; router 104 provides non-blocking communication among ports 138-143
respectively supporting links 122, 124, 153-155, and 125; and router 105
provides non-blocking communication among ports 144-148 respectively supporting links 123, 125, and 156-158.
[0077] A router may serve as a core router or as an edge router. Routers 102-103 are illustrated as edge routers because they serve links to members 110-112 that are outside of boundary 102. Boundary 102 may be designated for security purposes or represent a physical or political divide. Routers 104-105 are illustrated as core routers because all ports serve links to members within boundary 102 or serve other routers of network 101. In an alternate network, the ports of a core router serve no members, only other routers. Analysis of frames for purposes of determining a classification and consequently designating the effectivity of a suitable policy value may occur at an edge router as opposed to a core router. Effectivity may be implemented, for example, by marking a frame, setting a preference bit, specifying a priority value, setting a preemption bit, or identifying a suitable output queue that is serviced in a manner that is consistent with a desired quality of service. Core routers may be programmed to pass traffic without such analysis. A router may act as an edge router as to some ports (e.g., ports 140 and 141 of router 104) and as a core router as to other ports (e.g., 138-139
and 142-143 of router 104).
[0078] Interswitch links 121-125 may employ any conventional protocol, including a protocol different from the protocol used between a router and a member. For example, frames leaving a router's port onto an interswitch link may include additional information that may be removed before the frame is passed on a non-interswitch link. Further, the quality of service (QoS) provided by a router's port to an interswitch link may be better than the quality of service provided by a router's port to a non-interswitch link.
[0079] Preferably, all of a router's ports have identical port physical implementations, for example, for convenience of installation and maintenance of network 101 as other routers and links are added to network 101. A frame routed on an interswitch link may be marked (at an ingress edge router) as discussed above for effecting a policy and such marking may be removed (at an egress edge router).
[0080] Communication among members and resources, according to various aspects of the present invention, is supported by a system architecture that facilitates expansion and reliability. Expansion includes, inter alia, adding physical assemblies to the system to support additional ports, links, and/or processing capacity as well as to support redundant and fail-over capabilities. Reliability is further enhanced by, inter alia, dividing processing responsibility to avoid processing and communication bottlenecks, and by modular and reusable procedural, data, and hardware structures.
[0081] A system architecture is a plan by which system functions are made the responsibility of particular processes for efficient performance of system functions and for efficient communication among processes. The system architecture is systematically applied as implementations of the system are developed and expanded. For example, system architecture 200
of FIGs. 2-6 includes administration subsystem 109, network 101, and router 102 comprising managing, supervising, and routing processes. Implementations of router 102 provide one or more processors for executing these processes. Systems employing architecture 200 solve the problems discussed above (e.g., provide qualities of service), expand and contract without disruption of services, and exhibit extraordinary reliability.
[0082] An administration subsystem includes any computer having a port for communication via a link to network 101 and a processor that performs an administrating process. For example, administration subsystem 106 of FIGs. 1 and 2 may include one or more servers and/or workstations that provide a user interface and a port 106, coupled by link 107 to port 108
of router 102.
[0083] An administrating process provides routing information to any router of a network and receives reports from any router of the network. The routing information provided to a router from an administrating process may define alternative paths through the network that a router may choose on a frame by frame basis. When a frame identifies a destination to which it is to be routed, each router of the network, as a consequence of receiving routing information from an administrating process or from another router, may have one or more alternative paths that it may use to route the frame successfully. The router is generally free to make the choice of a particular path in accordance with routing information and other information, including current traffic conditions. Administrating includes assisting a human operator to develop suitable routing information for any number of routers of the network. To that end, the administrating process may also obtain or be automatically provided with information describing current conditions of the network. For example, administrating process 202 receives reports from router 102
via network links 107 and provides routing information (e.g., paths) via network link 107 to router 102. By coupling the administrating process to the router via network links, any suitable number and locations of administrating processes and administration subsystems may be used to accomplish reliable access to any or all routers of network 101. Consequently, all administrating functions are scalable to the complexity of network 101 as network 101 may expand (e.g., as the quantity of routers to be administered by a particular administration subsystem may increase).
[0084] In one implementation system administration includes network management and virtualization management functions that may be performed independently by different operators. In addition, routers of the network may include conventional routers (e.g., that do not recognize virtual members and virtual resources) and routers according to various aspects of the present invention (e.g., that recognize a packet that is destined for a virtual member or a virtual resource). Virtualization management includes communicating the designation of a nonvirtual member or resource to each router that is responsible for implementing a nonvirtual transaction corresponding to (e.g., in place of) a virtual transaction. The router receiving such communication is responsible for routing packets of the virtual transaction and of the nonvirtual transaction in accordance with routing information as discussed above.
[0085] A router, according to various aspects of the present invention includes scalable processes and scalable interfaces for communication between processes. Consequently, routers of any suitable complexity (e.g., number and speed of ports, number of protocols supported, and extent of frame analysis) may be implemented in accordance with architecture 200. For example, router 102 includes managing process 204, supervising process 206, and routing process 208. In operation, managing process 204 receives routing information via network 101 and provides routing information to supervising process 206 via bus 210; supervising process 206 stores routing information in memory 211 from which routing process 208 retrieves it; and, routing process 208, routes frames through links 214 and 216 to network 101 with reference to routing information recalled from memory 211. Frames received from network 101 are generally handled by routing process 208 in one of three ways: routing at least the payload of the same or corresponding frames to network links via fabric 213 and ports 216, routing at least the payload of the same or corresponding frames to managing process 204 via ports 201 and 214, and passing at least the payload of the same or corresponding frames to supervising process 206 via bus 212.
[0086] Particular advantages are realized in a system according to various aspects of system architecture 200. For example, by providing buses 210
and 212 as physical entities, processes 204, 206, and 208 may be hosted by independent processors (e.g., processors having access privileges over particular resources or separately packaged microprocessors). Consequently, each process 204, 206, and 208 may be hosted (e.g., provided with suitable resources) in scale with the complexity of functions performed by router 102. Significant economies result including economies related to modular circuit, firmware, and software design techniques. For example, one or more managing processes 204 may communicate on bus 210 with any number of supervising processes 206. One or more supervising processes 206 may communicate on bus 212 and by virtue of shared access to memory 211 with any number of routing processes 208. One or more routing processes 208 may communicate with network 101 via any number of ports 216 (e.g., conveying frames to any administrating subsystem and any network member) and communicate with any number of managing processes 204 via ports 201-214. In an alternate implementation, buses 210 and 212 may be a common entity. In yet another alternate implementation, processes 204, 206, and 208 may be performed by fewer than three processors (e.g., one processor or one array processor) and bus communication may be replaced with conventional interprocess communication (e.g., software interrupts, semaphores, common buffers, and multithreading).
[0087] An administrating process includes any process that provides a user interface to a human operator for the purpose of determining routing information (e.g., for virtualization management, or network management) to be used in routers of a network and that provides information regarding network utilization. For example, administrating process 202
of FIG. 3 includes edit paths process 302, obtain reports process 304, manage link loads process 310, display link utilization process 312, and port I/O (i.e., input/output) process 306. Routing information may be presented, stored, and communicated in any suitable form.
[0088] Edit paths process 302 creates and revises routing information, automatically and in response to input by a human system operator. Routing information, according to various aspects of the present invention, may include any combination of descriptions including: a set of alternate paths through the network, an association of a virtual member and at least one of a nonvirtual member and a nonvirtual resource, and an association of a virtual resource and at least one nonvirtual resource. Each path and association may be defined to include several links and policy values. Each link may be identified as a logical entity or as a physical entity.
[0089] A logical entity (e.g., a logical link, a logical resource, or a logical member) may correspond from time to time with one or more physical entities. By referring to a logical entity, the correspondence between the logical entity and any particular physical entity (or entities) may be determined dynamically, or in accordance with information that is not available at the time that the reference to the logical entity is made. For example, a reference to a logical entity need not be revised in light of the addition or removal of redundant physical entities. Consequently, system 100 may expand or portions of system 100 may fail and the reference to the logical entity remains valid (e.g., does not require amendment to continue particular network functions). At the physical level, a router that has received a frame on one of its ports either routes or drops the frame. Routing includes determining (not necessarily unambiguously) at least one physical output port to which the frame may be directed. If no such output port can be determined (or the only such output port is not available), the frame is said to be dropped. Dropping a frame (e.g., for lack of information sufficient to route the frame) accomplishes a denial of access to the member or resource intended to receive the frame.
[0090] Routing of frames between members (and resources) is somewhat analogous to sending a letter through the postal system. The letter originally bears the address that the sender believes is the current address of the intended recipient. For example, the sender may live in Ohio and may address a letter to a corporate headquarters in Georgia requesting a copy of the latest specialty catalog. The sender need not have any knowledge of the street addresses of the post offices or the names of their internal departments that may be involved. Suppose that the corporation has moved its headquarters to Florida and has filed with the Georgia post office a notice of change of address. When the letter is routed from the point of deposit into the postal system in Ohio to the post office in Georgia, the postal workers in Georgia may place the original letter in a surrounding envelope and address the outer envelope to Florida. The corporation may recognize from the outer envelope or otherwise that the letter is requesting a specialty catalog. The corporation may then enclose the outer envelope and its entire contents in a further enclosing envelope and apply the address of a particular catalog fulfillment center in Indiana; then redeposit it in the postal system. At the fulfillment center in Indiana, all envelopes may be discarded and the catalog shipped to the requester's Ohio address given in the letter.
[0091] When a router operating according to various aspects of the present invention determines that the frame that entered the router must have an additional address, a frame that encloses (and thereby includes) the received frame may be prepared and routed. This is analogous to enclosing the letter in an outer envelope as discussed above. For example an in-bound edge router may enclose the frame and an out-bound edge router may discard the outer frame and pass merely the inner frame. Alternately, a router operating according to various aspects of the present invention may prepare a frame that contains the payload of the original frame and a different address than originally received. This is analogous to covering an address on a letter with a sticker that bears a forwarding address. Generally, a frame bears at least one destination identifier; an address being one form of an identifier as discussed above. The several identifiers that may be encountered in operation of system 100 are outlined briefly in Table 1 tracing the routing of a request for data to be supplied by a resource (e.g., a request from SORT process 181 of member 116 for file CITIES from member 115.
1TABLE 1
Source or Network Destination Entity Context Services Identifiers Role Member User's process Upper level User's handle, Re- (e.g., 116) (e.g., SORT protocol API record number quester 181) Member Operating Operating Operating system (e.g., system API system handle, 182) filename Member Logical unit Device driver Path, logical unit abstraction manager API number, block address range Member Logical unit Device driver Logical unit (e.g., as API number, page supported by number, sector
[0092]
2
Source or Network Destination Entity Context Services Identifiers Role device driver number 183) Member Logical port Port API Logical port Requester, (e.g., 167) identifier Source Member Physical port Signals Physical port identifier Router Physical port Signals Physical port Ingress identifier Router Logical port Router input Logical port (e.g., 147) port logic identifier Router Virtual unit Router Virtual unit virtualization number, page API number, sector number Router Logical unit Router Path, logical unit abstraction management number, block logic address range Router Logical unit Router routing Logical unit logic number, page number, sector number Router Logical port Router output Logical port (e.g., 146) port logic identifier Router Physical port Signals Physical port Egress identifier Member Physical port Signals Physical port (e.g., 115) identifier
[0093]
3
Source or Network Destination Entity Context Services Identifiers Role Member Logical port Port API Logical port Participant, (e.g., 166) identifier Destination Member Logical unit Device driver Logical unit API number, page number, sector number Member Logical unit Device driver Path, logical abstraction manager API unit (e.g., controller number, block 173) address range Member Operating Operating Operating system system API system handle, filename (e.g., CITIES file on disk 177) Member Data entity Upper level User's handle, protocol API record number
[0094] Routing information may be stored in any conventional database such as paths database 303. In one implementation, paths database 303
includes for each router a data structure (e.g., one or more files) having records, each record comprising a data structure having fields for a destination identifier as specified in a received frame and one or more of a list of alternate logical or physical ports of the router to which the frame may be routed. Table 2 lists several records of paths database 303 describing sets of paths for routers 102, 104, and 105. The reference numbers in Table 2 identify ports shown in FIG. 1. In records corresponding to rows of Table 2, reference numbers would be replaced with logical port identifiers.
4TABLE 2
Output port Destination Router of the port as at which router at indicated in the which the the frame frame was frame was received received received Comment 165 105 145 Use link 125 to router 104. 165 105 144 If link 125 is busy or down, use link 123 to router 102. 166 105 146 Use link 156 to destination. 166
105 145 If link 156 is busy or down, use link 125 to router 104. 165 104 142 Use link 155 to destination. 165 102 132
Use link 122 to router 104.
[0095] Routing information may include associations of policy values and port identifiers (e.g., for network ports such as 165 and/or for router ports such as 145). Generally a policy value includes any value that specifies (e.g., directly, or indirectly by identifying another specification) access permissions, desired quality of service, priority, connection type (e.g., connection oriented, connectionless) class of service, traffic class, or other transaction controls to be implemented before or during routing. Policy values include any control values defined by a protocol including the identification of the protocol (e.g., SCSI and version number). For example, a Fibre Channel header includes a CS_CTL field that describes a class of service having functional specifications that assure a particular quality of service.
[0096] When routing information is being prepared by edit maps process 302, any representation of a port may be used (e.g., a name from a name server, an index or pointer into a list of names, a mnemonic, an icon, a world wide port name WWPN). Routing information may be entered by a system operator in any conventional manner including "point-and-click", "drag-and-drop", identification of a group of ports as equivalent (e.g., ports 165 and 166 may be identified as functionally equivalent as to member 115), or identification of a group of members of the network or of one or more subnetworks that are to be considered as a zone for a common purpose such as specifying policy values.
[0097] In one implementation according to various aspects of the present invention, routing information between physical entities is developed by routers 102-105 without user intervention according to methods performed by routers 102-105 that (a) identify port capabilities of all ports coupled to each port of a router; (b) advertise port identifiers to other routers via interswitch links; and (c) maintain routing information (e.g., further identification and advertising) when changes in port connections are detected. In such an implementation, the virtual ports, virtual members, and virtual resources (with suitable policy values) might not be discovered by routers 102-105 and are developed by an administrating process with user input.
[0098] Routing information (e.g., paths database 303) may be stored and maintained in a relational database. In one implementation, policy values are associated with group names, group names are associated with identifiers of members, and zone names are associated with identifiers of resources (processes and devices). A zone name may be used to describe a virtual member or a virtual resource. Further, group/zone tuples of group name, zone name, and policy values may be derived or maintained in such a database. Still further, member/resource tuples of member identifier, resource identifier, and policy values may be derived or maintained. In an alternate implementation, the derivation of member/resource tuples is accomplished by managing process 204 based on maps received from administrating process 202.
[0099] Routers of network 101 may gather information useful for any portion of administrating process 202. Obtain reports process 304 may use any suitable technique to obtain such information from routers 102-105. For example, obtain reports process 304 may poll routers of network 101 by sending a frame containing a command (e.g., a fabric control command or link service request). Routers 102-105 may provide such information in any suitable form from which obtain reports process 304 formats one or more entries in reports database 308. Obtain reports process 304 may, for example, use commands of the Simple Network Management Protocol (SNMP) to read any register or region of memory in a router 102-105. For example, routers 102-105 may provide lists of ports, image data, maps, and current configuration information from which administrating process 202 may develop new, expanded, or revised paths.
[0100] A method for preparing a map according to various aspects of the present invention includes in any order: requesting member identifiers and resource identifiers from routers of the network; associating each member identifier to a group of members; associating each resource identifier to a zone of resources; associating a path and a policy value to at least one of the group and the zone; determining port identifiers associated with the path; and communicating the policy value to each router of the network having at least one port identified to the path. In alternate implementations, the group and/or zone layer of indirection may be omitted so that path and policy values are associated directly with member identifiers and resource identifiers.
[0101] A method for preparing a map that enables routing to virtual entities (e.g., virtual members, virtual resources) includes in any order: (a) providing a name for each of any number of virtual members and/or virtual resources; (b) associating one or more portions of nonvirtual members and/or nonvirtual resources with each of the names; and (c) communicating each association to at least one router on each path used to communicate between a nonvirtual member and either of a virtual member or a virtual resource. The method may also include assigning a network port identifier to each virtual member or virtual resource. When a resource is divisible into fungible units (e.g., identically functioning units of storage or processing such as sector addresses or object references), the method may further include associating a unit of a nonvirtual resource to the name or to a portion of a virtual resource. For example, a sector of a named virtual resource may be associated with a sector of a nonvirtual resource.
[0102] Policy values may include access control values (e.g., identifiers of members or resources permitted to access other members or resources). Access controls may be associated with nonvirtual and virtual members and resources.
[0103] A method for facilitating network traffic may include the steps of: (a) preparing a map to facilitate routing of frames referring to virtual entities; (b) providing policy values associated with virtual entities; and (c) communicating the identifiers of virtual entities only to members or resources permitted to access them. Communication of the identifiers may be by selectively advertising.
[0104] A definition for implementing a database for use by edit paths process 302 and storage of routing information in paths database 303 is described in Table 3.
5TABLE 3
FILE (or list) and associated fields or a record (or entry) Description GROUP/POLICY A named group may serve as a logical construct group_name used to describe a set of policy values. Group policy_value names may be associated with policy values in many-to-many relationships. In other words, several policy values may be associated with a group name and the group name serve as an indication of the combination of policy values. Requesters (e.g., initiators) may be identified to groups as opposed to zones. MEMBER/GROUP A member, as identified by any suitable member_identifier identifier (e.g., IP address, or WWPN), group_name may be associated with a group name to indicate that the policy values of the group are to be associated with all network traffic involving the member. Members may be associated with groups in many-to-many relationships. An administration system operator may designate a member.sub.- identifier (e.g., an unused name) as a designation of one or more actual members. Such a member_identifier is herein called an identifier of a virtual member. ZONE/POLICY A named zone may serve as a logical construct zone_name used to describe a set of policy values to be policy_value applied to resources and members that are part of the zone. Zone names may be associated with policy values in many-to-many relationships. Participants (e.g., targets) may be identified to zones as opposed to groups.
[0105]
6
FILE (or list) and associated fields or a record (or entry) Description RESOURCE/ZONE A resource, as identified by any suitable resource_identifier identifier (e.g., IP address, WWPN; for a process, zone_name an object reference or reference of the type used if virtual, the with CORBA), may be associated with a zone associated nonvirtual name to indicate that the policy values of the resource identifier, zone are to be associated with all network traffic and (if applicable) involving the resource. Resources may be virtual unit to associated with zones in many-to-many nonvirtual unit relationships. An administration system crossreferences (e.g., operator may designate a resource.sub.- page/sector table or identifier (e.g., an unused name) as a object reference designation of one or more actual resources crossreferences) of on or more actual members. Such a member_identifier is herein called an identifier of a virtual resource. A virtual resource may be associated with an actual or virtual member. A virtual member may have virtual resources. MEMBER/ZONE A member and all resources of the member_identifier member (if any) may be associated zone_name with a zone as discussed above. if virtual, associated nonvirtual member identifier
[0106]
7
FILE (or list) and associated fields or a record (or entry) Description PATH/PORT A named path may serve as a logical path_name construct for developing routing information. source_port_identifier Typically two ports define the extremes of a destination_port_identifier path: the port of a requester (e.g., a source) router_identifier that is associated with a member or resource, output_port_identifier and the port of a participant (e.g., a destination) member or resource. An administrating process may have no knowledge of the routers and their output ports that may be involved in alternate paths -- leaving a managing process in a router to obtain, integrate, and dynamically maintain such information, supplementing the definition of a path. Nevertheless, identifiers and ports may be associated to a path name to the extent that an administrating process may suitably designate alternate routes or paths between groups, zones, members, and resources. MEMBER/PORT A port identifier may be a logical or physical member_identifier reference to a particular port. During router_identifier member login to a port of a particular router, port_identifier the member identifier and port identifier may be associated, for example, in a name server. A port identifier for a virtual member may be designated by an operator of the administration subsystem. RESOURCE/PORT The identity of a resource may be associated resource_identifier with a port of a particular router, as router_identifier discussed above. A port identifier for port_identifier a virtual member may be designated by an operator of the administration subsystem.
[0107]
8
FILE (or list) and associated fields or a record (or entry) Description GROUP/ZONE/PATH The association of a path to a group, a zone, group_name or both, provides an association of policy zone_name values to the path. When policy values as path_name defined for the group and the zone conflict, policy_values any suitable negotiation of policy values may occur to result in policy values to be used for the path. Integration of policy values may follow predetermined hierarchical rules maintained by an administrating process (e.g., edit paths process 302) or by a managing process. Negotiation may be accomplished dynamically during a login sequence. Resulting policy values associated with a path form the basis for routing tables. MAP A map may include policy values to be router_identifier implemented for any or every router of source_port_identifier the network. A segmented map or an overall destination_port_identifier map may be derived from the records policy_values discussed above. In an implementation crossreferences for where routers develop routing information implementing routing for without operator input, one or more of the virtual entities fields described here may be omitted.
[0108]
9
FILE (or list) and associated fields or a record (or entry) Description ROUTNG TABLE A routing table for a particular router source_port_identifier may be prepared as an excerpt from a destination_port_identifier MAP. The operator of the system policy_values administration subsystem may crossreferences for determine which routers will use implementing routing for crossreferences for routing virtual virtual entities entities. According to various aspects of the present invention, the readdressing of a frame that originally designated a virtual entity is accomplished at any one router along a path; other routers along that path need not have access to crossreferences implementing routing for that virtual entity. In one implementation, the burden of processing virtual routing is distributed among routers of network 101. In an implementation where routers develop routing information without operator input, one or more of the fields described here may be omitted. IMAGE An image may include information for routing_table_entries routing, supervising, and managing including data (e.g., constants, tables, routing_process_data configuration information), and programs (e.g., downloaded routing_process_programs subroutines for use by a routing processor to perform routing supervising_process_data of a particular type of frame of a particular protocol is supervising_process_programs recognized by a parser). In a network capable of determining managing_process_data paths for actual members and resources, the image may be managing_process_programs limited to information regarding routing of virtual
[0109] A port input/output process provides an application program interface (API) by which an application program may send and receive frames for communication (e.g., command, control, status, and data interchange) with other application programs, resources, and members of the network. For example, port I/O process 306 conducts all lower level protocols to permit administration process 202 to have access to information stored in routers and members of system 100. Port I/O process 306 provides an API to obtain reports process 304 and to manage link loads process 312.
[0110] Links are subject to traffic that consumes available capacity of the link herein called a link load. The link load may be quantified as having frame rate, delays between frames, bursts of immediately succeeding frames, burst length, delays between bursts, and related derived quantities (e.g., maxima, minima, averages, counts, rates, variances) during a suitable duration of measurement or monitoring . Management of a link load at the level of system administration involves transferring routing information from time to time to routers of network 101; and, providing information to assist development of routing information. For example, manage link loads process 310 sends portions of paths database 303 that apply to a particular router (e.g., rows 1-4
of Table 2) as an update to the particular router (e.g., router 105) in any manner suitable for limiting the disruption of ongoing network services. Updates may occur when a router logs into the fabric and at any suitable time thereafter. Manage link loads process 310 may request particular reports from obtain reports process 304 or may make reports from database 308 according to any conventional query.
[0111] The loads on various links over time and/or as related to one or more members constitute link utilization. Link utilization may be displayed in system 100 in any conventional aggregated or sorted manner. For example, display link utilization process 312 reads reports 308 and presents link utilization to the system operator via a graphical user interface. The system operator may analyze displays presented by display link utilization process 312 to determine that improved system performance may result if portions of paths database 303 are edited. Display link utilization process 312 may request particular reports from obtain reports process 304 or may make reports from database 308
according to any conventional query.
[0112] In each router of system 100 (e.g., 102), a managing process accepts paths sent in frames to the router from an administrating process and provides reports in frames to the administrating process. For example, managing process 204 accepts paths as sent by manage link loads process 310 and provides reports from time to time to obtain reports process 304. Due in part to the scalable architecture discussed above, each router may receive updates from any administrating process and provide reports as requested or automatically to any administrating process. A managing process, according to various aspects of the present invention, includes any process that performs one or more of the following operations: providing routing information to one or more supervising processes; obtaining from one or more supervising processes information for reports as discussed above; governing operation of one or more supervising processes to assure policy values are effected on a particular link; serving as a proxy for one or more members in any communication (e.g., for virtualization); operating a cache to provide an up to date redundancy of all or a portion of data stored at a member; and operating a cache to maintain a mirror storage resource as a copy of another storage resource. For example, managing process 204 includes port I/O process 402, LAN I/O process 404, manage configuration process 406, map store 408, image store 409, obtain and supply reports process 410, reports 412, load balance process 414, launch proxy for member process 416, any number of proxy for member processes 418, proxy state 420, cache agent process 422, cache 424, and mirror agent process 426.
[0113] Port I/O process 402 performs functions analogous to port I/O process 306, discussed above.
[0114] LAN I/O process 404 provides an API for processes within router 102 to communicate via bus 210. Bus 210 may be of the type known as a local area network (LAN), for example, including IP over IEEE 802.3
Ethernet for supporting, among other functions, an interprocess protocol of the promulgated by the Object Management Group as Common Object Request Broker Architecture (CORBA).
[0115] Managing configuration for a router, according to various aspects of the present invention, includes establishing initial values and updates of values stored in any memory device of the router. For example, manage configuration process 406 may receive configuration information (not shown) and routing information (e.g., paths) from administrating process 202 by SNMP communication, in frames, or in file transfers (e.g., comprising data in XML). Manage configuration process 406 determines router specific routing and configuration information, and stores received and derived information in map store 408 and in image store 409. Manage configuration process 406 determines configuration values that may be suitably tailored for one or more supervising processes 206. Configuration information may be derived in accordance with the establishment or termination of a proxy for a member, discussed below. Configuration information may also be derived in accordance with a result of load balancing, discussed below.
[0116] Routing information (e.g., paths, and associations implementing virtualization) may be received having references to logical identifiers. Mange configuration process 406 may refer to a name service (e.g., domain name service) to replace logical identifiers with physical identifiers and store results (e.g., maps) in map store 408. Maps in map store 408 may be used to develop routing information for particular routing processes. Routing information particular to a routing process may be combined with other data (configuration information, data, and programs) to form an image for transfer to a routing process.
[0117] Image store 409 is organized for convenient access by manage configuration process 406. Manage configuration process 406 may access image store 409 for reading configuration information, forming a proper message for the protocol on bus 210 (e.g., determining an address of a supervising process 206 for receipt of the configuration information); and for storing configuration information that may be reported by supervising processes from time to time via bus 210. Manage configuration process 406 includes watchdog timers that notice when a configuration of a supervising process has changed, and when such a process is no longer responding. Manage configuration process 406 may execute a reset on any supervising process (or processor) in an attempt to re-establish proper operation of a supervising process (or processor). Image store 409 may contain a description of the state of each supervising process 206 managed by manage configuration process 406.
[0118] Supervising processes are managed to coordinate operation of a router in an initial configuration, a power-on configuration (e.g., persistent from a recent power-off configuration), and an expanded configuration (e.g., additional ports and supervising processes added without disrupting current routing functions). Configuration information to be stored in a memory device of the router includes codes (e.g., flags, identifications, controls, and interrupt settings) for command registers (herein called command/status registers (CSRs)), programs for instruction stores (e.g., microcode for a state machine, native instructions for a processor, or statements for an interpreter), and variables and data for main memory (e.g., semiconductor and/or disk storage for variables, tables, and related data for random access memory or content addressable memory). These memory devices may be volatile or nonvolatile (herein generally called erasable programmable memory (EPM)). Consequently, manage configuration process 406 may conduct a series of download operations via LAN I/O process 404 (in cooperation with LAN I/O process 502) and may receive status and acknowledgements from LAN I/O process 404.
[0119] A managing process may obtain reports from a routing process; and the managing process may provide reports to an administrating process. Reports may be specified as to content and format by the administrating process and/or the managing process. According to various aspects of the present invention, communication of reports between all such processes utilizes network frames. The consuming process for any report may request the report specifically each time it is desired, or specify a subscription for the report to be fulfilled without further intervention by the consuming process. The providing process may produce reports only when requested (e.g., when polled), or may produce reports in response to lapse of a timer or on the occurrence of an event (e.g., an abnormal condition, or a condition requiring information and processing power from a managing or administrating process). Any conventional communication protocol may be used to implement the request/reply or subscription mechanisms. A variety of protocols may be used for a variety of reports. For example, obtain and supply reports process 410 sends requests via ports 214 (FIG. 2) conforming to SNMP to routing process 208 and routing process 208 sends replies via ports 214 to obtain and supply reports process 410 in conformance to SNMP. More particularly, port I/O process 402 parses incoming frames and delivers frames identified as SNMP to obtain and supply reports process 410. Obtain and supply reports process 410 sends reports from managing process 204 via ports 201 to administrating process 202 via ports 107 using network frames as discussed above.
[0120] According to various aspects of the present invention, a managing process may designate in a map a type of frame and an address recognized by the managing process so that a routing process, operating according to the map, that receives a frame of the designated type will route the frame to the managing process. Generally, frames are of two types: those involving data transfer; and otherwise, those involving control and/or status. For example, manage configuration process 406 may specify in a map that control frames of particular protocols (e.g., controls for virtual participants) are to be routed to a network port that is recognized by the parser of port I/O process 402. Such a map, passed to map 211 as discussed above, is used by routing process 208.
[0121] Part of a map may designate nonvirtual members (or resources) that are to be used when reference is made to virtual members (or resources). Each virtual member may accomplish the data processing and data communication functions of a member by obtaining the services of one or more nonvirtual (i.e., actual) members, nonvirtual resources, or portions thereof. The designation of nonvirtual resources to a virtual member may be specified by administrating process 202 and communicated to managing process 204 as part of a map. Operations on a virtual member (e.g., by control frames or data frames) may be accomplished on a physical device (e.g., member 110, or device 175 or 177), may be accomplished on a logical device (e.g., member 115 corresponding to resources on subnetwork 170), or on another virtual device as long as a nonvirtual device can be identified for the operations (e.g., no circular references or undefined virtual identifiers).
[0122] Launch proxy for member process 416 includes any process that analyzes frames and prepares replies to accomplish any of the following: (a) establish a virtual member; (b) identify any or all existing virtual members; and (c) perform for a virtual member any action appropriate for a nonvirtual member (e.g., respond to any control frame). Communication with a virtual member, in accordance with various aspects of the present invention achieves the effect that the requesting member is unaware that the request was accomplished by a proxy as opposed to a nonvirtual member. These determinations and replies may be accomplished using protocol analysis and communication techniques similar in some respects to conventional parsers and port I/O processes suitably modified for launching and cooperating with one or more proxy processes. Launching a proxy includes maintaining a list of operating proxy processes, dedicating resources (e.g., memory in a managing processor) for use by the proxy, determining an identifier for the proxy, updating routing information to enable communication with the proxy, and preparing to accept status and error condition messages that may originate with or be a consequence of the proxy. For example, when port I/O process 402
determines that a frame is a request to identify or to establish a virtual member, port I/O process passes the frame (or related information) to launch proxy process 416. Launch proxy process 416
responds by identifying an existing proxy or launching a new proxy as discussed above.
[0123] A proxy process includes any process that receives frames in a first transaction and that prepares frames directed to a nonvirtual member or resource in a second transaction. The first and second transactions may be in the same protocol or in different protocols. The first and second transactions may be separate in time or may overlap in time. A nonvirtual member or resource has state according to the protocol used to communicate with the nonvirtual member or resource. A proxy process makes virtual state visible to the user of the virtual member or resource (e.g., in response to a control frame). For example, each proxy process 418 operates as if it were a nonvirtual member having state according to the protocol used by the user of the virtual member. The st