Home
Patent Search
IMT Blog
REGISTER
|
SIGN IN
United States Patent
5809543
Byers , ; et al.
September 15, 1998
Title
Fault tolerant extended processing complex for redundant nonvolatile file caching
Abstract
An outboard file cache extended processing complex for use with a host data processing system for providing closely coupled file caching capability is described. Data movers at the host provide the hardware interface to the outboard file cache, provide the formatting of file data and commands, and control the reading and writing of data from the extended processing complex. Host interface adapters receive file access commands sent from the data movers and provide cache access control. Directly coupled fiber optic links couple each of the data movers to the associated one of the host interface adapters and from the nonvolatile memory. A nonvolatile memory to store redundant copies of the cached file data is described. A system interface including bidirectional bus structures and index processors that control the routing of data signals, provides control of storage and retrieval of file cache data derived from host interface adapters and from the nonvolatile memory. Multiple power domains are described together with independent clock distribution within each power domain. The independent clock distribution sources are synchronized with each other. A system for fault tolerant redundant storage of file cache data redundantly in at least two portions of the nonvolatile file cache storage is described.
Inventors:
Byers; Larry L.
(Apple Valley,
MN
)
, Torgerson; James F.
(Andover,
MN
)
, Price, deceased; Ferris T.
(late of Mayer,
MN
)
Assignee:
Unisys Corporation
(Blue Bell,
PA
)
Appl. No.:
745111
Filed:
November 7, 1996
Current U.S. Class:
711/162
711/167
714/14
714/6
711/120
Field of Search:
395/440,446,447,456-458,488,200.08,182.04,182.12,182.01,182.03,180,876 711/162,159,161,119,120,167,170,171,172,173,133
U.S. Patent Documents
4445174
April 1984
Fletcher
4794521
December 1988
Ziegler
5130922
July 1992
Liu
5193162
March 1993
Bordsen et al.
5193166
March 1993
Menasce
5261053
November 1993
Valencia
5313609
May 1994
Baylor et al.
5408651
April 1995
Flocken et al.
5437022
July 1995
Beardsley et al.
Other References
Improved cost, performance, and reliability by simultaneous accesses to pipelined caches with duplicate data protection and enhanced multiprocessor performance, IBM Tehnical Disclosure Bulletin, vol. 33, No. 1A, pp. 264-265, Jun. 1990. .
Renade, Software for mass storage systems, Optical Information Systems, vol. 10, No. 5, p. 256 (14), Sep. 1990..~
Primary Examiner:
Kim; Matthew M.
Attorney, Agent or Firm:
Johnson; Charles A. Starr; Mark T.
Parent Case Text
CROSS REFERENCE TO RELATED PATENT APPLICATIONS
This application is a continuation of application Ser. No. 08/173,459 filed on Dec. 23, 1993 and which is now abandoned. This application is related to the concurrently filed applications listed below, the disclosures of which are incorporated herein by reference. The identified applications are commonly assigned to Unisys Corporation, the assignee of the present invention:
OUTBOARD FILE CACHE SYSTEM, Ser. No. 08/174,750 filed on Dec. 23,1993, invented by Thomas P. Cooper and Robert E. Swenson;
DEDICATED POINT TO POINT FIBER OPTIC INTERFACE, Ser. No. 08/172,652, filed on Dec. 23, 1993, invented by Larry L. Byers, Donald Davies, Joseba M. Desubijana, Michael E. Mayer, Randall L. Piper, and Lloyd Thorsbakken, and which issued as U.S. Pat. No. 5,524,218 on Jun. 04, 1996;
XPC HUB AND STREET ARCHITECTURE, Ser. No. 08/173,429, filed on Dec. 23, 1993, invented by Donald W. Mackenthun, Larry L. Byers, Gregory B. Wiedenman and Ferris T. Price (Deceased), and which issued as U.S. Pat. No. 5,495,589 on Feb. 27, 1996;
ROUTING PRIORITIES WITHIN A HUB AND STREET ARCHITECTURE, Ser. No. 08/172,647, filed on Dec. 23, 1993, invented by Donald W. Mackenthun, and which issued as U.S. Pat. No. 5,450,578 on Sept. 12, 1995;
MICRO SEQUENCER BUS CONTROLLER SYSTEM, Ser. No. 08/172,657, filed on Dec. 23, 1993, invented by Larry L. Byers, Joseba M. Desubijana, and Wayne Michaelson, and which issued as U.S. Pat. No. 5,535,405 on Jul. 09, 1996;
FAULT TOLERANT CLOCK DISTRIBUTION SYSTEM, Ser. No. 08/172,661, filed on Dec. 23, 1993, invented by Larry L. Byers, Thomas T. Kubista, and Gregory B. Wiedenman, and which issued as U.S. Pat. No. 5,422,915 on Jun. 06, 1995;
FOUR PORT RAM CELL, Ser. No. 08/173,379, filed Dec. 23, 1993, invented by Larry L. Byers, Duane Kurth, and Ashgar Malikt, and which issued as U.S. Pat. No. 5,434,818 on Jul. 18, 1995; and
DATA COHERENCY PROTOCOL FOR MULTI-LEVEL CACHE HIGH PERFORMANCE MULTIPROCESSOR SYSTEM, Ser. No. 08/235,196, filed Apr. 29, 1994, invented by Kenichi Tsuchiya, Thomas Adelmeyer, Glen Kregness, Gary Lucas, Heidi Guck and Ferris Price, and which is a continuation of Ser. No. 07/762,276, filed Sept. 19, 1991, for which a File Wrapper Continuance Application was filed on Apr. 29, 1994 and which is now abandoned.
This application is a continuation Ser. No. 08/173,459 filed on Dec. 23, 1993.
Claims
What is claimed:
1. For use with a host data processing system for processing and modifying data files and having an instruction processor, a storage controller, a file mass storage device for storing the data files, main operational memory coupled to the storage controller, a data mover system to control the reading and writing of the data files to and from the host data processing system and being coupled to the instruction processor and to the main operational memory, and a transmission link coupled to the data mover, an outboard file cache external processing complex comprising:
a bidirectional bus structure;
a host interface adapter having host coupling terminals to be coupled to the transmission link to receive the data files and command signals from the host data processor systems and to transmit the data files and said command signals to the host data processing system, and having bus coupling terminals coupled to said bidirectional bus structure;
an index processor coupled to said bidirectional bus structure to control transmission of the data files and said command signals on said bidirectional bus structure;
a file-relative addressable nonvolatile storage system adapted to selectively redundantly cache the data files received from the host data processing system so that multiple copies of each data file are cached substantially simultaneously, said addressable nonvolatile storage system further adapted to selectively read the data files; and
a storage interface control circuit coupled to said addressable nonvolatile storage system and to said bidirectional bus structure to receive said command signals and in response thereto, to control selectively redundantly storing the data files in said file-relative addressable nonvolatile storage system and to further control selectively reading the data files from said file-relative addressable nonvolatile storage system,
whereby file caching and control is closely coupled to the host data processing system and is accomplished in parallel to file data processing and related control functions performed by the host data processing system.
2. The outboard file cache external processing complex as in claim 1 and further including a clock signal distribution system for providing clock signals.
3. The outboard file cache external processing complex as in claim 2 wherein said clock signal distribution system includes redundant oscillators and oscillator control circuits to select one of said redundant oscillators to provide said clock signals.
4. The outboard file cache external processing complex as in claim 1 and further including a power source for providing power.
5. The outboard file cache external processing complex as in claim 4 wherein said power source includes redundant power supplies and power supply control circuits to select one of said redundant power supplies to provide power.
6. The outboard file cache external processing complex as in claim 1 wherein said index processor includes:
first and second microsequencer devices, each having a micro processor and a local storage device;
a control storage device for storing instructions to be executed, said control storage device couple to said first and second microsequencer devices to provide the same instructions stored in said control store device to both said first and second microsequencer devices;
input circuits coupled to said first and second microsequencer devices to receive control signals from said bidirectional bus structure;
an output circuit from said first microsequencer device coupled to said bidirectional bus structure; and
an intercoupling circuit coupling said second microsequencer device to said first microsequencer device to pass the results of instruction execution made by said second microsequencer device to said first microsequencer device,
whereby error checking of operations is achieved by comparing said results of said instruction execution.
7. For use with a host data processing system for processing file data signals described in data files, an outboard file cache system comprising:
a data mover circuit having input terminals to be coupled to the host data processing system, said data mover circuit having first link terminals;
a host interface adapter circuit having second link terminals;
a bidirectional link coupled between said first link terminals and second link terminals;
system interface circuits coupled to said host interface adapter circuit and including routing control circuits, bidirectional busses, storage interface control circuits, and storage access terminals; and
a file addressable redundant nonvolatile storage system coupled to said storage access terminals to selectively redundantly cache selected ones of the data files processed by the host data processing system.
8. The outboard file cache system as in claim 7 wherein said routing control circuits include an index processor to manage allocation and cache replacement for the storage space available in said addressable nonvolatile storage system.
9. The outboard file cache system as in claim 8 wherein said index processor includes:
first and second microsequencer devices, each having a micro processor and a local control storage;
a control storage device for storing instructions to be executed, said control storage device coupled to said first and second microsequencer devices to provide the same instructions stored in said control store device to both said first and second microsequencer devices;
input circuits coupled to said first and second microsequencer devices to receive control signals from said bidirectional bus structure;
an output circuit from said first microsequencer device coupled to said bidirectional bus structure; and
an intercoupling circuit coupling said second microsequencer device to said first microsequencer device to pass the results of instruction execution made by said second microsequencer device to said first microsequencer device,
whereby error checking of operations is achieved by comparing said results of said instruction execution.
10. The outboard file cache system as in claim 7 and further including a clock signal distribution system for providing clock signals.
11. The outboard file cache as in claim 10 wherein said clock signal distribution system includes redundant oscillators and oscillator control circuits to select one of said redundant oscillators to provide said clock signals.
12. The outboard file cache as in claim 7 and further including a power source for providing power.
13. The outboard file cache as in claim 12 wherein said power source includes redundant power supplies and power supply circuits to select one of said redundant power suppliers to provide power.
14. The file cache system as in claim 7 wherein said bidirectional link comprises fiber optic transmission elements.
15. For use in a host data processing system for processing file data signals and having at least one instruction processor, a storage controller, a file mass storage device for storing file data signals that compose data files, main operational memory for use by the instruction processor, data mover circuits to control reading and writing of the file data signals to and from the host data processing system, each of the data mover circuits are coupled to the instruction processor and to the main operational memory, and a separate transmission link coupled to each of the data mover circuits, an outboard file cache external processing complex comprising:
first and second bidirectional bus structures;
first and second host interface adapter circuits, each having host coupling terminals to be coupled to an associated one of the transmission links to receive the data files and command signals from the associated data mover circuit in the host data processing system and to transmit the data files and said command signals to the associated data mover circuit in the host data processing system, and having bus coupling terminals wherein said first host interface adapter circuit is coupled to said first bidirectional bus structure and said second host interface adapter circuit is coupled to said second bidirectional bus structure;
a first index processor couple to said first bidirectional bus structure and a second index processor coupled to said second bidirectional bus structure, each of said first and second index processors to control transmission of the data files and said command signals on the associated one of said first and second bidirectional bus structures;
first and second addressable nonvolatile storage devices, each adapted to selectively and substantially simultaneously cache multiple copies of the data files and to selectively read the data files;
first and second storage interface control circuits, said first storage interface control circuit coupled to said first bidirectional bus structure and said second storage interface control circuit coupled to said second bidirectional bus structure, and said first and second storage interface control circuits each coupled to said addressable nonvolatile storage devices, wherein said first and second storage interface control circuits control said cache and said read operations of the data files in response to said command signals thereby storing duplicate copies of the data files in said first and second addressable nonvolatile storage devices.
16. The outboard file cache external processing complex as in claim 15 and further including a clock signal distribution system for providing clock signals.
17. The outboard file cache external processing complex as in claim 16 wherein said clock signal distribution system includes redundant oscillators and oscillator control circuits to select one of said redundant oscillators to provide said clock signals.
18. The outboard file cache external processing complex as in claim 17 wherein said clock distribution system includes a first set of selected groups of said redundant oscillators and control circuits arranged to provide said clock signals to said first bidirectional bus structure, said first host interface adapter circuit, said first index processor, said first addressable nonvolatile storage device, and said first storage interface control circuit; and includes a second set of selected groups of different ones of said redundant oscillators and oscillator control circuits arranged to provide said clock signals to said second bidirectional bus structure, said second host interface adapter circuit, said second index processor, said second addressable nonvolatile storage device, and said second storage interface control circuit; and
a synchronization circuit for synchronizing said clock signals provided by first and second groups.
19. The outboard file cache external processing complex as in claim 18 and further including a power source for providing power.
20. The outboard file cache external processing complex as in claim 19 wherein said power source includes redundant power supplies and power supply control circuits to select one of said redundant power supplies to provide power.
21. The outboard file cache external processing complex as in claim 20 wherein said power source includes a first set of said redundant power supplies coupled to provide power to said first bidirectional bus structure, said first host interface adapter circuit, said first index processor, said first addressable nonvolatile storage device, said first storage interface control circuit, and said clock signal distribution system associated therewith; and a second set of said redundant power supplies coupled to power said second bidirectional bus structure, said second host interface adapter circuit, said second index processor, said second addressable nonvolatile storage device, said second storage device, said second storage interface control circuit, and said clock signal distribution system associated therewith, said first set and said second set establishing separate power domains.
22. The outboard file cache external processing complex as in claim 15 and further including a crossover circuit intercoupling said first and second bidirectional bus structures.
23. For use with a host data processing system for processing and modifying data files, each data file being referenced by a file identifier and file offsets, an outboard file cache system comprising:
redundant nonvolatile file cache storage means for receiving copies of selected ones of the data files from the host data processing system and for redundantly storing duplicate cached copies of said selected ones of the data files, each of said selected ones of the data files being stored substantially simultaneously with the associated said duplicate cached copy;
file cache storage interface means coupled to said redundant nonvolatile cache storage means for controlling caching and retrieval operations for said duplicate cached copies of said selected ones of the data files according to file identifiers and file offsets associated with said selected ones of the data files;
index processor means coupled to said file cache storage interface means for controlling transmission of said selected ones of the data files and command signals to and from said redundant nonvolatile file cache storage means;
host interface adapter means coupled to said file cache storage interface means for formatting said selected ones of the data files received from and to be sent to the host data processing system and for providing said selected ones of the data files to be cached by said redundant nonvolatile file cache storage means;
bidirectional bus means coupled to said file cache storage interface means, said index processor means, and said host interface adapter means for transmitting said selected ones of the data files;
bidirectional link means coupled to said host interface adapter means for transmitting said selected ones of the data files and said command signals; and
data mover means coupled to said link means for formatting and controlling said selected ones of the data files at the host data processing system.
24. The outboard file cache system as in claim 23 wherein said index processor means includes:
control storage means for storing instructions to be executed;
first and second microsequencer means coupled to said control storage means for executing instructions read therefrom in parallel;
intercoupling circuit means coupling said second microsequencer means to said first microsequencer means to pass the results of instruction execution made by second microsequencer means to said first microsequencer means for allowing error checking of operations.
25. The outboard file cache system as in claim 23 and further including clock signal distribution means for providing clock signals.
26. The outboard file cache system as in claim 25 wherein said clock signal distribution means includes redundant oscillator means and oscillator control circuit means for selecting one of said redundant oscillator means for providing said clock signals.
27. The outboard file cache system as in claim 26 wherein said clock distribution means includes a first set of said redundant oscillator means and control circuit means arranged to provide clock signals to a first portion of said bidirectional bus means, a first portion of said redundant nonvolatile file cache storage means, a first portion of said file cache storage interface means, a first portion of index processor means, and a first portion of said host interface adapter means for timing operations; and said clock distribution means includes a second set of said redundant oscillator means and control circuit means arranged to provide clock signals to a second portion of said bidirectional bus means, said redundant nonvolatile cache storage means, file cache storage interface means, index processor means, and host interface adapter means for separately timing operations; and
synchronization means for synchronizing said clocking signals provided by said first and second sets of redundant oscillators means and control circuit means.
28. The outboard file cache system as in claim 23 and further including power source means for providing power.
29. The outboard file cache system as in claim 28 wherein said power source means includes redundant power supply means and power supply control circuit means for selecting one said redundant power supply means to provide power.
30. The outboard file cache system as in claim 29 wherein said power source means includes a first set of said redundant power supply means coupled to provide power to a portion of said bidirectional bus means, a portion of said redundant nonvolatile file cache storage means, a portion of said file cache storage interface means, a portion of said index processor means, and a portion of said host interface adapter means and the portion of said clock signals distribution means associated therewith; and a second set of said redundant power supply means coupled to power a second portion of said bidirectional bus means, a second portion of said redundant nonvolatile cache storage means, a second portion of said file cache storage interface means, a second portion of said index processor means, and a second portion of said host interface adapter means, and said clock signal distribution means associated therewith; said first set and said second set establishing separate power domains.
31. The outboard file cache system as in claim 23 wherein said bidirectional bus means further includes a crossover circuit means for providing access to both portions of said redundant nonvolatile file cache storage means in the event of partial failure of a portion of said bidirectional bus means.
32. For use with a data processing system having n hosts for processing file data signals which compose data files, where n is an integer greater than one, an outboard file cache system comprising:
2n data mover circuits, each of said 2n data mover circuits having input terminals to be coupled in pairs to associated ones of the n hosts and each of said 2n data mover circuits having first link terminals;
2n host interface adapter circuits, each of said 2n host adapter circuits having second link terminals;
2n bidirectional links, each of said 2n bidirectional links coupled between associated ones of said first link terminals and said second link terminals;
2n system interface circuits, each of said 2n system interface circuits coupled to adjacent ones of said 2n system interface circuits and coupled to an associated one of said 2n host interface adapter circuits, each of said 2n system interface circuits including routing control circuits, bidirectional busses, storage interface control circuits, and storage access terminals, and
2n addressable nonvolatile storage devices arranged in pairs, each of said pairs of said 2n addressable nonvolatile storage devices coupled to said storage access terminals of associated pairs of said 2n system interface circuits to selectively redundantly cache file data signals of selected ones of the data files.
33. The outboard file cache system as in claim 32 wherein each of said routing control circuits includes an index processor to manage allocation and cache replacement for the storage space available in the associated pairs of said 2n addressable nonvolatile storage devices.
34. The outboard file cache system as in claim 33 wherein said index processor includes:
first and second microsequencer devices, each having a micro processor and a local control storage;
a control storage device for storing instructions to be executed, said control storage device coupled to said first and second microsequencer devices to provide the same instructions stored in said control store device to both first and second microsequencer devices;
input circuits coupled to said first and second microsequencer devices to receive control signals from said bidirectional bus structure;
an output circuit from said first microsequencer device coupled to said bidirectional bus structure; and
an intercoupling circuit coupling said second microsequencer device to said first microsequencer device to pass the results of instruction execution made by said second microsequencer device to said first microsequencer device,
whereby error checking of operations is achieved by comparing said results of said instruction executions.
35. The outboard file cache system as in claim 32 and further including a clock signal distribution system for providing clock signals.
36. The outboard file cache system as in claim 35 wherein said clock signal distribution system includes redundant oscillators and oscillator control circuits to select one of said redundant oscillators to provide said clock signals.
37. The outboard file cache system as in claim 36 wherein said clock signal distribution system includes a first set of groupings of said redundant oscillators and control circuits arranged to provide said clock signals to n ones of said 2n host interface adapter circuits, to a first portion of said 2n nonvolatile storage devices, and to n ones of said first 2n system interface circuits; and includes a second set of groupings of different ones of said redundant oscillators and oscillator control circuits arranged to provide said clock signals to n different ones of said 2n host interface adapter circuits, a second portion of said 2n addressable nonvolatile storage devices, and n different ones of said 2n storage interface control circuits; and
a synchronization circuit for synchronizing said clock signals provided by said first and second sets of groupings.
38. The outboard file cache system as in claim 32 and further including:
2n additional system interface circuits, each of said 2n additional system interface circuits coupled to associated ones of said 2n additional system interface circuits, and coupled to associated ones of said 2n system interface circuits; and
n crossover circuits interconnecting selected pairs of said 2n system interface circuits.
39. The outboard file cache system as in claim 38 and further including a power source for providing power.
40. The outboard file cache system as in claim 39 wherein said power source includes redundant power supplies and power supply control circuits to select one of said redundant power supplies to provide power.
41. The outboard file cache system as in claim 40 wherein said power source includes:
a first set of said redundant power supplies and power supply control circuits to establish a first power domain coupled to n selected ones of said 2n host interface adapter circuits, to n selected ones of said 2n system interface circuits, to n selected ones of said 2n additional system interface circuits, and to a first portion of said 2n addressable nonvolatile storage devices; and
a second set of said redundant power supplies and power supply control circuits to establish a second power domain coupled to n different selected ones of said 2n host interface adapter circuits, to n different selected ones of said 2n system interface circuits, to n different selected ones of said 2n additional system interface circuits, and to a second portion of said 2n addressable nonvolatile storage devices,
wherein loss of power in either one of said first power domain or said second power domain will not cause failure of the outboard file cache system.
42. For use with a host data processing system for processing data files addressed using file identifiers and file offsets, a fault tolerant file cache system comprising:
redundant nonvolatile file cache storage means for receiving data files from the host data processing system and for redundantly storing duplicate cached copies of the data files substantially simultaneously according to the file identifiers and file offsets;
host interface adapter means for formatting the data files received from and to be sent to the host data processing system and for providing the data files to be cached; and
cache control means coupled to said redundant nonvolatile file cache means and to said host interface adapter means for controlling the redundant storage and retrieval of the data files.
43. The fault tolerant file cache system as in claim 42, and further including:
redundant power means for providing isolating power to predetermined separate portions of said redundant nonvolatile file cache storage means.
44. The fault tolerant file cache system as in claim 43, and further including:
redundant clocking means coupled to said redundant power means for providing synchronized isolated clocking signals to said predetermined separate portions of said redundant nonvolatile cache storage means.
45. The fault tolerant file cache system as in claim 44, and further including:
cross over means for selectively providing access to one of said predetermined separate portions of said redundant nonvolatile file cache storage means when another of said predetermined separate portions of said redundant nonvolatile cache storage means fails to function.
Description
BACKGROUND OF THE INVENTION
A. Field of the Invention
This invention relates to a data processing system having file caching capabilities, and more particularly to an Extended Processing Complex coupled to a host processor system for managing file caching. Still more particularly, this invention relates to an Extended Processing Complex that is outboard of a related host processing system, but is closely coupled thereto, the Extended Processing Complex including redundant characteristics that render it highly reliable.
B. General Background
The performance of data processing systems has improved dramatically through the years. While new technology has brought performance improvements to all functional areas of data processing systems, the advances in some areas have outpaced the advances in other areas. For example, advancements in the rate at which computer instructions can be executed have far exceeded improvements in the rate at which data can be retrieved from storage devices and supplied to the instruction processor. Thus, applications that are input/output intensive, such as transaction processing systems, have been constrained in their performance enhancements by data retrieval and storage performance.
The relationship between the throughput rate of a data processing system, input/output (I/O) intensity, and data storage technology is discussed in "Storage hierarchies" by E. I. Cohen, et al., IBM Systems Journal, 28 No. 1 (1989). The concept of the storage hierarchy, as discussed in the article, is used here in the discussion of the prior art. In general terms, the storage hierarchy consists of data storage components within a data processing system, ranging from the cache of the central processing unit at the highest level of the hierarchy, to direct access storage devices at the lowest level of the hierarchy. I/O operations are required for access to data stored at the lowest level of the storage hierarchy.
To help alleviate input/output limitations on throughput, some early systems developed high-speed storage systems internal to the processor system, where groupings of words encompassing addressed words were stored or "cached" for reference by the processor without having to resort to an I/O processor for each reference. While this caching concept improved throughput, it did so at the expense of processor overhead necessary implement the caching function.
Varied attempts have been made to relieve the I/O bottleneck which constrains the performance of I/O intensive applications, while attempting to reduce the overhead impact on the associated instruction processor. Three ways in which the I/O bottleneck has been addressed include solid state disks, cache disks, and file caches.
Solid state disks (SSDs) were developed to address the relatively slow electromechanical speeds at which data stored on magnetic disks or other mass storage devices is read or written. SSDs are implemented using dynamic random access memory (DRAM) technology. The logical organization of the DRAM corresponds to the particular magnetic disk which the SSD is emulating. This allows software applications to access files stored on the SSD in the same manner they would access files stored on a magnetic disk. Files are understood to be records of associated and related data records.
The major advantage SSDs have over magnetic disks is that data can be read or written at electronic speeds rather than the electromechanical speeds of magnetic disks. An application's throughput may be significantly improved if the application makes a substantial number of disk requests to an SSD rather than a magnetic disk.
At least three problems persist with SSDs. First, the data path length for making requests to the SSD remains the same as for magnetic disks; second, the overhead involved in addressing the proper location in SSD storage is still allotted to the instruction processor or central processing unit; and third, a fault tolerant SSD configuration requires two write operations for data security. All three problems result in added processing time and reduced system throughput.
The first disadvantage associated with SSDs remains because a SSD resides at the same level of the data storage hierarchy as a magnetic disk or other addressable mass storage devices. To access a given file at a particular location within the file, which can be considered to be the "offset" from a point of reference, the file and offset must be located in the storage hierarchy: the SSD on which the file is stored must be identified; the disk controller which provides access to the SSD must be identified; the input/output channel to which the disk controller is coupled must be identified; and the input/output processor to which controls the input/output channel must be identified. All this processing is performed by the instruction processor. While the instruction processor is performing these tasks, others must wait, and the result is a reduction in the overall data processing throughput rate. Furthermore, the application software seeking access to the file data must wait for the input/output request to travel to the I/O processor, through the I/O channel, through the disk controller, to the desired disk, and back up the data path to the application software be executed by the instruction processor.
The second disadvantage for SSDs is that the instruction processor is required to map a relative file address to a physical disk address and manage allocation of SSD space. While the instruction processor is mapping file requests and managing disk space it cannot perform other tasks and the data processing system throughput rate suffers.
The third disadvantage associated with SSDs remains because two SSDs are required if fault tolerant capabilities are required. Fault tolerance with SSDs involves coupling two SSDs to a data processing system through two different data paths. A backup SSD mirrors the data on the primary SSD and is available in the event of failure of the primary SSD. To keep the backup SSD synchronized with the primary SSD, the instruction processor must perform two write operations when updating a file: the first write operation updates the primary SSD, and the second write operation updates the backup SSD. This method adds additional overhead to the data processing system to the detriment of the system throughput rate.
The Extended Processing Complex of this invention is outboard of the instruction processor and is closely coupled thereto, whereby the disadvantages of system overhead and operation attendant to SSD systems are overcome.
A cache disk subsystem is an invention which was made to address the I/O bottleneck attendant to caching of data stored on disks. U.S. Pat. No. 4,394,733, issued to Robert Swenson and assigned commonly to the assignee of this invention, discloses a cache disk subsystem. The cache disk subsystem utilizes DRAM storage for buffering selectable groupings of data words to be written to or read from magnetic disks, and resides at the disk controller level of the data storage hierarchy so that groupings of a data words associated with a plurality of magnetic disks can be cached.
The chief advantage of the cache disk subsystem is that I/O requests addressing a portion of a disk which is cached can be processed at electronic speeds rather than the electromechanical speed of a disk. While this advantage is substantial, the cache disk subsystem's position in the data storage hierarchy constricts the flow of I/O requests. The I/O performance gained by cache disk subsystems is limited by the data path length and numerous files competing for limited cache storage space. Because the caching of disk storage takes place at the disk controller level of the data storage hierarchy, the operating system must determine the appropriate data path in the same manner as described with the SSD. As described above, a lengthy data path reduces overall system throughput.
Where a large number of files compete for cache disk subsystem cache space, the I/O performance gains may be severely limited due to excess overhead processing. If two or more files have a high I/O request rate and they are stored on the same or different disks under a common disk controller, a substantial amount of the processing performed by the cache disk subsystem may be overhead. The overhead is incurred when most or all of cache storage is in use, and the cache disk subsystem is experiencing a high miss rate. A miss is defined as an I/O request which references a portion of disk which is not currently in cache storage. When a miss occurs, the cache disk subsystem must select a segment of cache storage to allocate to the latest I/O request (the selected segment may currently hold a different portion of different disk), and read the referenced portion of disk and store it in the cache segment. If this processing is required for a large proportion of I/O requests, the benefit of caching disk storage is lost to overhead processing.
One way in which the aforementioned problem is addressed is by separating files with a high access rate by storing them on separate disks under different storage controllers. This solution is expensive in two respects. First, human resources are required to physically separate the files and ensure that the operating system has the correct configuration information. Continual monitoring is required to detect when the location of files is hampering the I/O rate, and then redistributing files as necessary. Second, hardware costs are substantial because additional disks, disk controllers, and cache disk subsystems are required to physically separate the files.
A third strategy for relieving the I/O bottleneck is file caching. File caching differs from cache disk subsystems in that file data is buffered in main DRAM storage of a data processing system, and file management software manages allocation of main storage for file buffers. In "Scale and Performance in a Distributed File System" by John Howard, et al., ACM Transactions on Computer Systems, 6, No. 1, (1988), 51-81; "Caching in the Sprite Network File System", by Michael Nelson, et al., ACM Transactions on Computer Systems, 6, No. 1, (1988), 134-154; and U.S. Pat. No. 5,163,131, entitled, "Parallel I/O Network File Server Architecture", to Edward Row, et al., three different approaches to file caching are discussed.
The file caching described in "Scale and Performance in a Distributed File System" involves files which are distributed across a network of workstations. Each workstation contains server software for providing access to each of the files it manages. File cache software on the workstation seeking access to a selected file locates the server which controls access to the file and requests the file data from the server software. The file cache software stores the file data it receives on the local disk storage of the client workstation. In contrast, the file cache system described in "Caching in the Sprite Network File System" caches file data to the main memory of the client workstation. The disadvantages with each approach are readily apparent.
With the approach of Howard et al. to file caching, the "cached" file data is stored on a disk controlled by the client workstation. This means that the rate at which file data can be accessed is still dependent upon the access rate of the local disk. Furthermore, any updates to the locally cached file must be written to the server's version of the file before other clients are allowed to access the file.
While the Howard et al. approach provides access to file data at main memory access speed, it is still burdened with the overhead of keeping the server's version of the file consistent with the client's cached version. In addition, file data loss is also possible if main memory on the client workstation fails. In particular, if the cached file is updated and the client workstation fails before the update is forwarded to the server, the file update may be lost. Therefore, to provide file data integrity for a file update occurring on the client workstation, before the operation is allowed to complete, the file update must be transmitted to the server workstation and stored on its disk.
U.S. Pat. No. 5,163,131 also discusses a file cache architecture applicable to a networked workstation environment. In this patent, the file data is cached in the main memory of the server workstation. For other workstations on the network to access the file data cached on the server, network communication must be initiated for the transfer of file data. Thus, the benefits of file caching are limited by the amount of traffic on the network and the network bandwidth.
The current state of file caching schemes involves the tradeoff between the security of storing file data on disk and an increased access rate by storing the file data stored in main memory. Alternatively, the file data can be stored in electronic memories which are closer to the disk in the storage hierarchy, but the access rate is constrained by the length of the data path from an application to the electronic memory. Therefore, it would be desirable for a file cache to provide a high I/O rate while and still maintain data security which is comparable to disk storage.
The Extended Processing Complex of the present invention addresses the instruction processor overhead of prior art systems by providing most of the file caching control and manipulation external to the host processing system, but while being closely linked to the host processing system through a high speed dedicated communication media that is independent of the normal I/O system.
As pointed out above, the prior art caching systems for which high reliability is required, necessarily either requires redundant processing within the instruction processor, or redundant I/O arrangements. The Extended Processing Complex of this invention materially improves the reliability of the file caching system by utilizing nonvolatile storage devices for the cache memory and by providing redundant power and clocking capabilities, together with redundant cache management circuitry, to provide the cache management in parallel such that failure of any individual element within the Extended Processing Complex will not defeat this operation and accurate data files will be maintained.
II. OBJECTS
It is an object of the invention to increase the rate at which access to file data is provided when the file data is not present in the main memory of a host processing system.
Another object is to cache file data in storage which is non-volatile relative to a host processing system.
Yet another object of the invention is to provide an Extended Processing Complex for managing caching of data files as directed by an associated host processing system.
Still another object of the invention is to provide an Extended Processing Complex that is closely coupled through a host processing system through a dedicated high speed transmission system.
Still another object of the invention is to provide a fault and failure tolerant Extended Processing Complex that is outboard of an associated host processing system.
Still a further object of the invention is to provide an improved Extended Processing Complex where file cache management that utilizes a redundant nonvolatile storage for use in providing identical copies of each file caching operation.
A further object of the invention is to provide a fault and failure tolerant Extended Processing Complex or outboard file caching that utilizes separate power domains.
Still another object of the invention is to provide an improved Extended Processing Complex that utilizes multiple independent clocking sources that are synchronized and individually powered by independent power sources.
Still a further object of the invention is to provide an Extended Processing Complex that utilizes redundant processing and transmission paths for performing file cache manipulations in parallel for storage in associated portions of a redundant nonvolatile file cache memory system.
Another object of the invention is to provide an Extended Processing Complex that is coupled by a fiber optic link to an associated host processing system such that the transmission of commands and file data is closely coupled to the processing system.
A further object of the invention is to provide a HUB and street architecture within an Extended Processing Complex that can route file data to and from a redundant nonvolatile memory system under control of associated micro sequencers driven by commands received from one or more associated host processing systems.
Still another object of the invention is to minimize the processing required to write back or destage file data from the cache storage to storage device where the file data resides.
Yet another object is to cache file data from a plurality of host processing systems in shared cache storage.
A further object is to cache file data which is shared between a plurality of host processing systems.
III. SUMMARY OF THE INVENTION
According to the present invention, the foregoing and other objects and advantages are attained by coupling an outboard file cache to a host file data processing system. The host issues file access commands which include a logical file-identifier and a logical offset. The outboard file cache includes a file descriptor table and cache memory for electronic random access storage of the cached files. The file descriptor table stores the logical file-identifiers and offsets of the portions of the files in the cache storage. Cache detection logic is interfaced with the file descriptor table and receives file access commands from the host. The file descriptor table is used to determine whether the portion of the file referenced by the file access command is present in the cache memory. Cache access control is responsive to the cache detection logic, and if the portion of the file referenced in the cache access command is present in cache memory, the desired access is provided. The outboard file cache is non-volatile relative to the main memory of the host because it is a separately powered storage system. Neither the host nor the outboard file cache is required to map the file data referenced in a file access command to the physical storage device and the physical address of the backing store on which the file data is stored if the referenced data is present in cache storage.
The outboard file cache extended processing system is closely coupled to an associated host file data processing system. In this context "close coupling" implies a direct point-to-point transmission path extending from the input/output section of the host to the outboard file cache extended processing complex. It is not directly coupled in the sense that the host data processing system does not directly include the outboard file cache in the addressing architecture of its memory section, but rather requires that the file data signals to be cached be appropriately assembled in packets and addressed with reference to the file itself. Data mover circuitry provides the formatting and establishment of control commands both for transmitting file data signals to be cached and for reading file data signals that have been cached and retrieved. The data mover circuitry also provides the control to drive and receive signals across the associated data link.
A fiber optic data link is utilized to provide the data file signal and command signal transmission to and from the outboard file cache extended processing complex.
The outboard file cache extended processing complex of the present invention utilizes redundant nonvolatile file cache storage means for storing and retrieving duplicate cached file signals. A file cache storage interface means which is coupled to the redundant volatile file cache storage controls caching and retrieval operations of the duplicate cached file signals in accordance with control commands received from the host file data processing system. Index processor means are coupled to the file cache storage interface means and are operative to control transmission of file data signals and command signals via bidirectional HUB and street bus structures. A host interface adapter means includes control circuitry for establishing signals to be sent across the link to the data mover means and to translate signals received across the link from the data mover means to signal levels that can be utilized in the extended processing complex. The host interface adapter means provide the first level of selection and control for controlling caching and retrieval of file data signals stored in the outboard file cache. Index processor means are coupled to the file cache storage interface means and control transmission of file data signals and command signals to and from the redundant nonvolatile file cache storage means.
To provide redundancy and resiliency against error, the file data signals to be cached are stored in duplicate files in separate portions of the redundant nonvolatile file cache storage. All of the control and access circuitry is duplicated such that a complete set is applicable to each half of the redundant nonvolatile file cache storage. By providing the redundant copy of the cached file signals, a failure in any of the control or access circuitry will not result in system failure, but will accommodate recovery of the cached file data signals from the portion of the redundant nonvolatile file cache storage to which access has been maintain.
Another aspect of the extended processing complex system resiliency is established from the redundant nonvolatile file cache storage and the duplicated control circuitry being driven from two separate and independent power sources. If there is a power failure to one half of the system, the balance of the system will continue to function from the other independent power domain source.
With the use of the redundant nonvolatile file cache storage and the redundant control and access circuitry and power domains, it has been found advantageous to also provide redundant clock signal distribution systems with one of the clock distribution systems clocking one portion of the redundant nonvolatile file cache storage and its associated control circuitry and other of the clock distribution systems clocking the other portion of the redundant nonvolatile file cache storage and the control and access circuitry associated therewith. In order to have the redundant file caching proceed in parallel, the redundant clock distribution systems are synchronized one to the other so that the cache of file data signals always remains in step.
The invention also contemplates utilization of the outboard file cache extended processing complex by multiple host data processing systems. It is generally advantageous for each host to have at least two data mover circuits and two interconnecting links coupled to the outboard file cache extended processing complex such that failure of any data mover or any link will not cut off availability of the cached file data signals.
In an additional aspect of the invention a first and a second, host are coupled to the outboard file cache. The cache memory in the outboard file cache extended processing system is shared between the files of the first host and the files of the second host. The outboard file cache includes dual cache detection logic sections. Each of the cache detection logic sections may process file access commands from either the first host or the second host and each section operates concurrently with the other. The outboard file cache includes a first cache access control section and a second cache access control section. The first cache access control section is dedicated to providing access to the cache storage for the first host and the second cache access control section is dedicated to providing access to the cache storage for the second host.
Still other objects and advantages of the present invention will become readily apparent to those skilled in the art from the Drawings and the following Detailed Description of the Preferred Embodiment, wherein a preferred embodiment of the invention is shown, by way of illustration of the best mode contemplated for carrying out the invention. As will be realized, the invention is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the spirit and scope of the invention. Accordingly, the Drawings and Detailed Description of the Preferred Embodiment are to be regarded as illustrative in nature, and what is to be protected by Letters Patent is defined in the appended Claims.
The control of caching and retrieval of cached data signals external to the host data processing system results in substantially increased throughput of the total data processing system through the host being relieved of most of the control and processing necessary to establish, maintain, and utilize the cached file signals.
IV. BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows an exemplary data processing system, or "host" or "host processing system", with which the present invention could be used;
FIG. 2 shows the architecture of an Input/Output Complex of the exemplary Host;
FIG. 3 is a block diagram of a plurality of Hosts coupled to a variety of prior art disk subsystem configurations;
FIG. 4 illustrates an Outboard File Cache in a data storage hierarchy;
FIG. 5 shows the overall file processing within the data storage hierarchy shown in FIG. 4;
FIG. 6 is a functional block diagram of the hardware and software components of the preferred embodiment of the outboard file cache system;
FIGS. 7, 7A, 7B, and 7C contain a data flow diagram illustrating the flow of data between each of the major functional components of the file cache system;
FIG. 8 shows the general layout of a Command Packet and the information contained therein;
FIG. 9 illustrates the Program Initiation Queue;
FIG. 10 shows the information contained in and the format of a Program Initiation Packet;
FIGS. 11 and 12 respectively illustrate the Status Packet Queue and the format and information contained in a Program Status Packet;
FIG. 13 illustrates the HIA ACB Buffer;
FIG. 14 illustrates Activity Queue, and FIG. 15 shows the information contained in each Activity Queue Entry;
FIG. 16 illustrates the file space available in the Outboard File Cache;
FIG. 17 shows the logical organization of a single Segment;
FIG. 18 shows the logical composition of a Block;
FIG. 19 shows the logical division between Cache File Space, Nail Space, and Resident File Space in the File Space of the Outboard File Cache;
FIG. 20 illustrates the File Descriptor Table;
FIG. 21 shows the information contained in a File Descriptor;
FIG. 22 is a flow chart of the general processing the I/O Software performs for file requests from Application Software;
FIG. 23 shows a flow chart of the FILE CACHE INTERFACE processing performed by the File Cache Handler Software;
FIG. 24 shows a flow chart of the general processing for detecting when the processing of a Command Packet (or a chain) is complete;
FIGS. 25, 25A, and 25B respectively show the components of a Data Mover (DM) and Host Interface Adapter (HIA);
FIG. 26 is a functional block diagram of the Index Processor (IXP);
FIG. 27 is a flow chart of the main processing loop of the IXP;
FIG. 28 is a block diagram to further illustrate the functional components of the Street interprocessor communication and storage access network within the Outboard File Cache;
FIG. 29 is an block diagram illustrating a data processing configuration including a plurality of Hosts coupled to a Outboard File Cache;
FIG. 30 is a block diagram of a clock distribution system where a plurality of clock sources powered by separate voltage busses are synchronized across the power domain boundaries in order to simultaneously clock an equal number of equivalent circuit loads;
FIG. 31 is a block diagram of the preferred embodiment of the Fault Tolerant Clock Distribution System;
FIG. 32 shows the Fault Tolerant Clock Distribution System when DC Power Source B or AC Power Source B has failed;
FIG. 33 is a waveform diagram which shows the Synchronized Clock Signals which supply each load, and which further illustrates how the Synchronized Clock Signal in a given power domain continues to clock its respective load upon the loss of the other Synchronized Clock Signal;
FIG. 34 illustrates the AC power source and DC power source redundancy of the preferred embodiment;
FIG. 35 is a block diagram showing the detail of the preferred embodiment;
FIG. 36 is a block diagram of Clock Source A in Power Domain A;
FIG. 37 is a block diagram of Clock Source B in Power Domain B;
FIG. 38 is a block diagram of the components of the Fiber Optic Interface;
FIG. 39 is a block diagram of the clock domains for one end of the Fiber Optic Interface;
FIG. 40 is a table of the symbols transferred from the Light Pipe Frame Control to the PLAYER+components;
FIG. 41 is a table of the symbols transferred from the PLAYER+components to the Light Pipe Frame Control;
FIG. 42 is a block diagram of the Transmitter logic of the Light Pipe Frame Control gate array;
FIG. 43 is a block diagram of the Receiver logic of the Light Pipe Frame Control gate array;
FIG. 44 is a block diagram of the Frame Transfer Facility gate array;
FIG. 45 is a block diagram of the Microsequencer Bus Controller System;
FIG. 46 is a block diagram illustrating the Data and Data Parity paths of the Micro Bus;
FIG. 47 shows the parity domain for the Data path of the Micro Bus when the Microsequencer Bus Controller System is a Data Mover;
FIG. 48 shows the parity domain for the Data path of the Micro Bus when the Microsequencer Bus Controller System is a Host Interface Adapter;
FIG. 49 is a block diagram illustrating the Address and Address Parity paths of the Micro Bus;
FIG. 50 is a block diagram showing how the parity domains for the Addresses on the Micro Bus are distributed;
FIG. 51 is a block diagram showing the two levels of Address Parity checking performed by the Microsequencer Bus Controller System;
FIG. 52 shows the format of an Address for the Micro Bus;
FIG. 53 is a block diagram of the main components of the Microsequencer Bus Controller;
FIG. 54, comprising FIG. 54A through FIG. 54D, is a detailed diagram illustrating the architecture of a Microsequencer Bus Controller;
FIG. 55 shows the allocation of the Local Store memory locations;
FIG. 56 is a block diagram of the Extended Processor Complex (XPC);
FIG. 57 is a block diagram of the outboard file cache System;
FIG. 58 is a block diagram of the interconnect of outboard file cache blocks;
FIG. 59 is a detailed block diagram of the interconnect between system interface cards and the Nonvolatile Memory;
FIG. 60 is a table containing the output priority scheme for the HUB0 and the HUB1 elements;
FIG. 61 is a diagram defining the HUB control format;
FIG. 62 is a diagram defining the Unit Identification field;
FIG. 63 is a diagram defining the HUB error status format;
FIG. 64 is a timing diagram for the basic interface for the transmission of one maximum length packet; and
FIG. 65 is a block diagram showing the HUB Street priority circuitry.
V. DESCRIPTION OF THE PREFERRED EMBODIMENT
A. Host Data Processing System
FIG. 1 shows an exemplary data processing system, or "host", or "host processing system" with which the present invention could be used. The illustrative Host 10 architecture is that of the 2200/900 Series data processing system which is commercially available from the Unisys Corporation, it being understood that other instruction processor, input/output handling devices, and main memory systems, ranging from mainframe systems to workstation systems can function as the Host as used with this invention.
The Instruction Processors (IPs) 12 are the basic instruction execution units of the system. In this configuration there are eight IPs labelled 12-1 through 12-8, and a system may include one or more such IPs. Each IP includes a first level cache (not shown) having a section for instructions and a section for operands. The IPs 12 are functional to call instructions from memory, read data from memory, execute the instructions and store the results, and in general, perform data manipulation.
Each of the IPs 12 is directly coupled via Cables 13, labelled 13-1 through 13-8, respectively, to a Storage Controller (SC) 14, respectively labelled 14-1 through 14-2. The operation of the SC configuration is described in a co-pending patent application entitled DATA COHERENCY PROTOCOL FOR MULTILEVEL CACHED HIGH PERFORMANCE MULTIPROCESSOR SYSTEM, Ser. No. 07/762,276, filed Sep. 19, 1991, naming Kenichi Tsuchiya, Thomas Adelmeyer, Glen R. Kregness, Gary J. Lucas, Heidi Guck, and Ferris Price (Deceased), inventors, and assigned to the assignee of this invention. It is sufficient for this application that SCs are understood to provide control of communication between SCs, and between associated MSUS, IPs, and IOCs. One configuration for the
2200/900 data processing system includes four SCs 14, each SC having two directly coupled IPs 12. The SCs 14 each provide logic and interconnects which provide access to associated Main Storage Units (MSUs) 16, labelled 16-1 through 16-8. The MSUs comprise the high-speed main random access memory of the Host 10. Each SC 14 controls access to two associated directly coupled MSUs 16, and to two associated directly coupled IPs 12, and to two associated directly coupled IOCs 32. Cables 18 labelled
18-1 through 18-8 couple the MSUs to their respective SCs 14.
The SCs 14 contain interconnect logic that ties all IPs 12 together in a tightly coupled system. SC1 is coupled to SC2 via Cable 20; SC1 is coupled to SC3 via Cable 22; SC1 is coupled to SC4 via Cable 24; SC2 is coupled to SC3 via Cable 26; SC2
is coupled to SC4 via Cable 28; and SC3 is coupled to SC4 via Cable 30. Each IP 12 can address every MSU 16 of Host 10 via the SCs. For example, the SC intercoupling allows IP6 to have access to the addressable memory of MSU8. A memory request originating in IP6 is first sent to SC3 labelled 14-3; SC3 sends the memory request to SC4 labelled 14-4; SC4 provides access to the portion of addressable memory; and if requested, SC4 14-4 returns data to SC3 14-3 which in turn forwards the data to IP6.
Each of the SCs 14 also provide interfaces for two Input/Output Complexes (IOCs) 32, labelled 32-1 through 32-8. Cables 34 labelled 34-1 through 34-8 couple each of the IOCs 32 to their respective SCs 14. Each of the IOCs 32 may contain multiple Input/Output Processors (IOPs not shown). The IOPs read data from the MSUs 16 for writing to peripheral devices, and read data from peripheral devices for writing to the MSUs 16. Peripheral devices may include printers, tape drives, disk drives, network communication processors, etc.
The 2200 Series data processing architecture allows a Host 10 to be logically partitioned into one or more independent operating environments. Each independent operating environment is referred to as a partition. A partition has its own operating system software which manages the allocation of resources within the partition. Because a partition has its own operating system, it may be also referred to as a Host. Using Host 10 as an example, it could be partitioned into four Hosts: a first host having the resources accompanying SC1, a second host having the resources accompanying SC2, a third host having the resources accompanying SC3, and a fourth host having the resources accompanying SC4.
FIG. 2 shows the architecture of an Input/Output Complex 32 of the exemplary Host. Input/Output Remote Adapter (IRA) 36 is a non-intelligent adapter which transfers data and messages between an associated SC 14 on cable 34 and an associated IOP
38 labelled 38-1 through 38-12, respectively, via an Input/Output Bus 40. The IRA 36 occupies one physical drop out of the thirteen available on Input/Output Bus 40 and has the highest priority of any unit connected to Input/Output Bus 40. IRA 36 does not participate in any rotational priority operations and can gain access to the Input/Output Bus 26 through the normal request process even when other units coupled to the Input/Output Bus are operating in a rotational priority mode.
The Input/Output Bus 40 provides the communication path and protocol to transfer data between the attached units. The Input/Output Bus 40 can accommodate twelve Input/Output Processors 38. It will be recognized that bus architectures are well known in the prior art and a further discussion of the Input/Output Bus shown is not necessary for the purposes of the present invention.
The IOPs 38 are microprocessor controlled units that control the initiation, data transfer, and termination sequences associated with software generated I/O channel programs. Initiation and termination sequences are executed by the microprocessor (not shown) and data transfer is controlled by hard-wired logic (not shown). Each IOP 38 is coupled to a Data Bus 42, which in turn has available slots for up to four Block Mux Channel Adapters 44, labelled 44-1 through 44-4 and a Word Channel Adapter 46. Channel Adapters 44 and 46 are coupled to their respective peripheral subsystems via Cables 48, labelled 48-1 through 48-4 and Cable 49. It should be understood that each of IOP2, IOP3, . . . , and IOP12 is coupled to its associated Data Bus (not shown). The other 11 Data Buses which are not shown, provide connections for additional Channel Adapters. Lines 50, labelled 50-1 through 50-12, represent the coupling between IOP1, IOP2, IOP3, . . . , and IOP12 and their associated Data Buses.
B. Prior Art Data Storage Hierarchy
FIG. 3 is a block diagram of a plurality of Hosts coupled to a variety of prior art disk subsystem configurations. FIG. 3 serves to illustrate the hierarchical relationship between the configurations. Each Host 10 labelled 10-I through 10-N is coupled to one or more of the Control Units 80, 82, 88, or 92 by Cables 48, labelled 48-I through 48-N. Host-1 is coupled to Control Units 80 and 82. Control Unit 80 provides access to Magnetic Disks 84, labelled 84-I through 84-P and Control Unit 82
provides access to Magnetic Disks 86, labelled 86-I through 86-Q. If application software on Host-1 requests access to a file stored on any Magnetic Disks 84 or 86, operating system software is required to find: (1) the Disk 84 or 86 on which the file is stored; (2) which Control Unit 80 or 82 provides access to the Identified Magnetic Disk; (3) the IOP 38 (see FIG. 2) to which the selected Control Unit is coupled; and (4) the Input/Output Bus 40 to which the IOP 38 is coupled. Once the necessary information is determined, a control program can be constructed and sent along the identified data path to provide access to the file. File data may be buffered in the Main Storage 16 of Host-1 to enhance the retrieval rate for file data; however, the file data must be written back (destaged) to appropriate Disks 84 or 86 to protect against data loss.
Control Unit 82 is coupled to and shared by Host-1, Host-2, and Host-3. Each of the coupled Hosts can gain access to data stored on Disks 86-I through 86-Q. A Multi-Host File Sharing (MHFS) system (not shown), which is commercially available from Unisys Corporation, allows application software on Host-1, Host-2, and Host-3 to share file data stored on Disks 86 and coordinates locking files or portions thereof.
Host-3 is coupled to Cache Disk Controller 88. Cache Disk Controller 88 provides access to Disks 90, labelled 90-I through 90-R, and buffers portions of Disks 90. The cache storage (not shown) that Cache Disk Controller 88 uses to buffer Disks
90 resides within the Cache Disk Controller 88. This configuration is an example of the cache disk subsystem described in U.S. Pat. No. 4,394,733 mentioned above, and is a prior attempt to remove caching overhead and burden from the IPs, through it can be seen that such a configuration is still in the data path of the IOCs in the illustrative configuration. Operation of the Cache Disk Controller 88 is transparent to application and system software on Host-3. The cache storage is allocated to all application and system software having access to files stored on Disks 90 on a first-come first-served basis.
Control Unit 92 is coupled to Host-N and controls access to Disks 94 labelled 94-I through 94-S, and a Solid State Disk 96. The Solid State Disk 96 resides at the Disk 94 level of the data storage hierarchy and provides access to data stored therein at electronic rather than the electromechanical speed of the Disks 94. In order to gain access to data stored on Solid State Disk 96, the data path on which the disk resides must be constructed in the same manner as discussed above for Disks 84.
C. File Cache System Overview
FIG. 4 illustrates an Outboard File Cache in a data storage hierarchy. A plurality of Control Units 104 labelled 104-I . . . 104-N, are coupled to Host 10 via IOPs 38-1 and 38-2 for providing access to Disks 106-1, 106-2, . . . 106-P and
106-N1, 106-N2, . . . 106-NQ. Application and system software executing on Host 10 reads data from and writes data to Files 108a-h. While Files 108a-h are depicted as blocks it should be understood that the data is not necessarily stored contiguously on the Disks 106. The Disks provide a backing store for retaining the Files. In the storage hierarchy, disks would fall into the category of secondary storage, with primary storage being the main memory of a Host.
Extended Processing Complex (XPC) 102 is an outboard file cache that provides cache storage for Files 108a-h which is comparable to Disks 108, with resiliency against data loss. A Data Mover 110 is coupled to the Input/Output Bus 40 (see FIG. 2) in the Host and provides a functionality which is similar to the IOPs 38-1 and 38-2. The Data Mover in Host 10 provides a closely coupled direct high-speed communications Link 112 to the XPC. In the preferred embodiment Link 112 includes a Fiber Optic Interface that will be described in more detail below. All or part of Files 108 may be stored in the XPC 102 depending upon the storage capacity of the Outboard File Cache 102, and the size and number of Files 108 selected to be cached.
The portion of Files 108a-h that are stored in the outboard file cache of XPC 102 are shown as blocks 114a-h. The cached portion of respective Piles 108 are labeled File-A', File-B', . . . , File-H' for discussion purposes. File-A' 114a is all or the portion of File-A that is stored in outboard file cache 102, File-B' 114b is all or the portion of File-B that is stored in outboard file cache of XPC 102, and so on for Files C through H, respectively. The outboard file cache at this level of the storage hierarchy allows references to cached files to be immediately directed to the outboard file cache XPC 102 for processing, in contrast with a non-cached file where an I/O channel program must be constructed to access the proper disk and the request and data must flow through a possibly lengthy data path.
FIG. 5 shows the overall file processing within the data storage hierarchy shown in FIG. 4. The processing begins at Step 122 where a software application executing on Host 10 requests access to a selected file. The access request may involve either reading data from or writing data to the selected file.
A file access command is sent to the outboard file cache XPC 102 at Step 124. Included in the file access command are a file identifier which specifies the file on which the operation is to be performed, an offset from the beginning of the file which specifies precisely where in the file the operation is to begin, and the quantity of data which is to be read from or written to the file. At Decision Step 126, the outboard file cache XPC determines whether the referenced data is present in the outboard file cache based on the file identifier, offset, and quantity. If the referenced data is not in the outboard file cache 102, Control Path 128 is followed to Step 130.
Step 130 involves staging the data from the appropriate Disk 106 (see FIG. 4) to the outboard file cache XPC 102. Staging the data involves reading the required data from a selected Disk and then storing the data in the outboard file cache. Subsequent references to the staged data normally will not result in a miss, and the data can be accessed in the Outboard File Cache. If Decision Step 126 finds that the referenced data is in Outboard File Cache 102, Control Path 132 is followed to Step
134 where access is granted to the referenced data.
1. Functional Block Diagram
FIG. 6 is a functional block diagram of the hardware and software components of the preferred embodiment of the outboard file cache system. The overall system is comprised of hardware and software elements in both the Host 10 and outboard file cache XPC 102. The software on Host 10 is shown by blocks 202, 204, 206, and 208. The blocks are joined to signify the interrelationships and software interfaces between the software elements. The software elements or programs are stored in the Main Storage Unit(s) 16 (see FIG. 1) for execution. Programs may be loaded from Disk(s) 106 (see FIG. 4). The software programs are executed by IP(s). Operating system software (not shown) directs and controls performance of the various software elements.
Application Software 202 provides data processing functionality to end users and includes applications such as bank transaction processing and airline reservations systems. Data bases maintained by Application Software 202 may be stored in one or more the exemplary Files 108 as shown in FIG. 4. File Management Software 204, Input/Output Software 206, and File Cache Handler Software 208 are all part of the operating system (not shown). In general File Management Software 204 provides overall management of file control structures, and in particular handles the creating, deleting, opening, and closing of files.
Input/Output Software 206 provides the software interface to each of the various I/O devices coupled to the Host 10. While not illustrative specification, the I/O devices may include network communication processors, magnetic disks, printers, magnetic tapes, and optical disks. Input/Output Software 206 builds channel programs, provides the channel programs to the selected appropriate IOP 38, and returns control to the requesting program at the appropriate time.
File Cache Handler Software 208 coordinates the overall processing for cached files. In general, File Cache Handler Software 208 provides the operating system level interface to the outboard file cache XPC 102, stages file data from Disks 106 to the outboard file cache XPC 102, and destages file data from the outboard file cache XPC 102 to Disks 106. The File Cache Handler Software 208 provides file data and file access commands to the hardware interface to the outboard file cache via Main Storage 16. Main Storage 16 is coupled to the Input/Output Bus 40 by Line 210. Line 210 logically represents the Storage Controller 14 and Input/Output Remote Adapter 36 of FIGS. 1 and 2.
A Data Mover (DM) 110a provides the hardware interface to the outboard file cache XPC 102. While two DMs 110a and 110b are shown, the system does not require two DMs for normal operations. A configuration with two DMs processing identical cache functions provides fault tolerant operation; that is, if DM 110a fails, DM 110b is available to process file requests. Each of the DMs is coupled to the Input/Output Bus 40 of Host 10. File Cache Handler Software 208 distributes file access commands among each of the DMs coupled to Input/Output Bus 40. If DM 110a fails, file access commands queued to DM 110a can be redistributed to DM 110b.
The DMs 110a and 110b provide functionality which is similar to the IOPs 38 (see FIG. 2), that is to read data from and write data to a peripheral device. The DMs can read from and write to Main Storage 16 without the aid of IPs 12. The DMs coordinate the processing of file access commands between File Cache Handler Software 208 and the outboard file cache XPC 102 and move file data between Main Storage 16 and the outboard file cache. Each of the DMs is coupled to an associated Host Interface Adapter (HIA) 214 logic section within the outboard file cache XPC 102. DM 110a is coupled to HIA 214a by a pair of fiber optic cables shown as Line 112a, and DM 110b is coupled to HIA 214b by a second pair of fiber optic cables shown as Line
112b. The fiber optic interconnection will be described in more detail below.
The outboard file cache XPC 102 is configured with redundant power, redundant clocking, redundant storage, redundant storage access paths, and redundant processors for processing file access commands, all of which cooperate to provide a fault tolerant architecture for storing and manipulating file data. The outboard file cache XPC 102 is powered by dual Power Supplies 222a and 222b, which provide independent power domains within the XPC. The portion of the XPC to the left of dashed line 224
is powered by Power Supply 222a and is referred to as Power Domain A, and the portion of the XPC to the right of dashed line 224 is powered by Power Supply 222b and is referred to as Power Domain B. Each of Power Supplies 222a and 222b has a dedicated battery and generator backup (not shown) to protect against loss of the input power source.
Two separately powered Clock Sources 226a and 226b provide timing signals to all the logic sections of outboard file cache XPC 102. Clock Source 226a provides timing to the logic sections within Power Domain A and Clock Source 226b provides timing to the logic sections within Power Domain B. Redundant oscillators within each Clock Source provide protection against the failure of one, and Clock Sources A and B are synchronized for consistent timing across Power Domains A and B. The clock distribution system will be described in more detail below.
Non-Volatile Storage (NVS) section 220 includes multiple DRAM storage modules and provides the file cache memory. Half of the storage modules are within Power Domain A and the other half are within Power Domain B. The data stored within the storage modules in Power Domain B reflects the data stored in storage modules within Power Domain A. NVS 220 thereby provides for redundant storage of cached file data and the control structures used by the outboard file cache XPC 102. The redundant storage organization provides for both single- and multiple-bit error detection and correction according to techniques that are known in the prior art.
The portions of NVS 220 within each of the Power Domains A and B are each coupled to two Storage Interface Controllers (SICTs) 228a and 228b. While only two SICTs are shown in FIG. 6, each half of NVS 220 is addressable by up to four SICTs. Line 230 represents the coupling between SICT 228a and the portion of NVS 220 within each of Power Domains A and B. Similarly, Line 232 represents the coupling between SICT 228b and NVS 220.
Read and write requests for NVS 220 are sent to the SICTs 228a and 228b via Street Networks 234a and 234b. The Street Network provides the data transfer and interprocessor communication between the major logic sections within the outboard file cache XPC 102. The Street Network is built to provide multiple requesters (HIAs 214a and 214b or Index Processors 236a and 236b) with high bandwidth access to NVS 220, as well as multiple paths for redundant access. Crossover 238 provides a path whereby NVS 220 requests may be sent from Street 234a to Street 234b, or visa versa, if a SICT is unavailable. For example, if SICT 228a fails, NVS requests sent from requesters (HIAs and IXPs) are sent to Street 234b via Crossover 238, whereby NVS 220
access is provided by SICT 228b. The Street Network will be described in more detail below.
The HIAs 214a and 214b (Host Interface Adapters) provide functionality in the outboard file cache XPC 102 which is similar to the functionality provided by the DMs 110a and 110b (Data Movers) on the Host 10. In particular, the HIAs receive file access commands sent from the DM and provide general cache access control such as writing file data sent from the Host to Non-Volatile Storage (NVS) 220 and reading file data from NVS and sending it to the Host. The HIAs also contain the logic for sending and receiving data over fiber optic Lines 112a and 112b.
Index Processors (IXPs) 236a and 236b manage allocation and cache replacement for the storage space available in NVS 220, service file data access commands sent from Host 10, and generally provides for overall file cache management. The IXPs contain microcode control for detecting whether the file data referenced in a file data access command is present in the cache memory, and for managing and coordinating access to the cache memory. The functionality provided by an IXP will be discussed in greater detail later in this specification.
2. Data Flow
FIGS. 7A, 7B, and 7C when arranged as shown in FIG. 7 contain a data flow diagram illustrating the flow of data between each of the major functional components of the file cache system. Each of the blocks represents a major logic section, a software component, or a storage section of the file cache system. Within each of the blocks are data structures which are shown as labelled online storage symbols and circles representing processing performed by the component. Although the circles represent the processing performed, they are not intended to illustrate the flow of control. The directional lines represent the flow of data between processing circles and data structures and are labelled according to the data being transferred. FIGS.
8 through 15 show the information contained within the data structures referenced in FIG. 7. Each of FIGS. 8 through 15 will be discussed as it is encountered in the discussion of FIG. 7.
File access commands begin with application software on the Host 10 (not shown in FIG. 7) requesting input or output services (I/O) for a selected file. I/O requests for cached files are processed by the File Cache Handler Software 208. Data flow Line 300 shows the input of an I/O request to File Cache Handler Software 208. I/O requests are sent from the Host 10 to the outboard file cache XPC 102 in Command Packets. At Process Node 302 the File Cache Handler Software 208 builds a Command Packet (CP) for the specified I/O request and stores the Command Packet in a Command Packet Data Structure 304. Line 306 represents storing the I/O request information in the Command Packet Data Structure 304.
a. Command Packet
FIG. 8 shows the general layout of a Command Packet and the information contained therein. The Command Packet 452 contains information that describes one of the available outboard file cache commands (read, write, stage, destage, etc.). Each of the commands is identified and discussed later in this specification. FIG. 8 shows only the command information which is common to all Command Packets for the various command types.
A Command Packet can have from 4 to 67 36-bit words, depending upon the command type. Words 0 and 1, bits 12 through 23 of Word 3, and Words 4 through n of the Command Packet, respectively referenced by 452a, 452b, and 452c, are dependent upon the command type.
The file cache system permits Command Packets to be chained together. That is, a first Command Packet 452 may point to a second Command Packet, and the second Command Packet may point to a third Command Packet, and so on. The NEXT.sub.-- COMMAND.sub.-- PACKET 452d is used for chaining the Command Packets together. It contains the address of the next Command Packet in the command chain. If the CCF 452e (Command Chain Flag) is set, then NEXT.sub.-- COMMAND.sub.-- PACKET contains the address of the next Command Packet in the command chain. A chain of commands is also referred to as a "program." If CCF is clear, then no Command Packets follow the Command Packet in the command chain. The CCF is stored at Bit 5 of Word 3 in the Command Packet.
The LENGTH 452f of the Command Packet, that is the number of words in the Command Packet following Word 3, is stored in bits 6 through 11 of Word 3. Bits 24 through 35 of Word 3 contain COMMAND.sub.-- CODE 452f which indicates the operation to be performed by the outboard file cache. Bits 0-4 of Word 3 and referenced by 452g are reserved.
Processing Node 308 in FIG. 7 enqueues a Program Initiation Packet (PIP) in a Program Initiation Queue (PIQ) 310. Line 312 represents the flow of Program Initiation Packet information to the Program Initiation Queue 310. The Command Packet (CP) Address from Node 302 is used in enqueuing a PIP. The CP Address supplied to Node 308 is shown by Line 309.
b. Program Initiation Queue
FIG. 9 illustrates the Program Initiation Queue. The Program Initiation Queue 310 may contain up to 32 Program Initiation Packets (PIPs), respectively referenced 456-1, 456-2, 456-3, . . . , 456-32. The Program Initiation Queue may be larger or smaller depending upon implementation chosen. Once the Program Initiation Queue is filled with Program Initiation Packets, further queuing is performed to handle the overflow.
FIG. 10 shows the information contained in and the format of a Program Initiation Packet. VF (Valid Flag) 456a is stored in bit 0 of Word 0 of the Program Initiation Packet 456. VP indicates whether the information in the Program Initiation Queue 310 entry is valid.
Bits 1 through 35 of Word 0 and Bits 0 through 3 of Word 1 are reserved for future use and are respectively referenced in FIG. 10 by 456b and 456c. The PROGRAM.sub.-- ID 456d is stored in bits 4 through 35 of Word 1. The PROGRAM.sub.-- ID uniquely identifies the program being submitted to the outboard file cache XPC 102. The PROGRAM.sub.-- ID is used to associate the status returned from the outboard file cache with the program to which it applies.
Word 2 of the Program Initiation Packet 456 contains the COMMAND.sub.-- PACKET.sub.-- ADDRESS 456e which is the real address of the first Command Packet 452 in a command chain or a single Command Packet. Word 3 contains the NEXT.sub.-- SP.sub.-- ADDRESS 456f. The NEXT.sub.-- SP.sub.-- ADDRESS is the real address in Main Storage 16 of an area where the outboard file cache XPC 102 can write status information.
After the XPC 102 has processed a command, the status of the command is reported back to the Host 10 in a Program Status Packet (PSP). Line 314 shows the flow of a Program Status Packet from the Data Mover (DM) 110 to an entry in the Status Packet Queue (SPQ) 316. The format of the Status Packet Queue 316 and the Program Status Packet is described next, followed by further discussion of Command Packet processing.
c. Status Packet Queue and Program Status Packet
FIGS. 11 and 12 respectively illustrate the Status Packet Queue and the format and information contained in a Program Status Packet. The number of Program Status Packets 460 in the Status Packet Queue 316 is equal to the number of programs queued in the Program Initiation Queue and are respectively referenced 460-1, 460-2, 460-3, . . . , 460-n. Generally, the content and format of a Program Status Packet is as follows:
______________________________________ Word Bit Definition ______________________________________ 0 0-5 Valid Flag (VF) 460a indicates whether the Program Status Packet contains valid status information. If VF = 0, then the Program Status Packet does not contain valid status information. If the VF = 1, then the Program Status Packet does contain valid status information. 0 6-17 Reserved as referenced by 460b. 0 18-35 UPI.sub.-- NUMBER 460c is the Universal Processor Interrupt (UPI) number associated with the outboard file cache interface. 1 0-3 Reserved as reference by 460d. 1 4-35 PROGRAM.sub.-- ID 460e is a value which identifies the Command Packet (or Command Packet Chain) which is associated with the Program Status Packet. If NO.sub.-- PROGRAM in the FLAGS field is set, PROGRAM.sub.-- ID is reserved. Every outboard file cache XPC program issued by a Host has an associated PROGRAM.sub.-- ID which is unique within the Host. When status is returned to the Host, PROGRAM.sub.-- ID is used to relate the status to the program to which it applies. Note that PROGRAM.sub.-- ID applies to all commands within a single program. A status is associated with a command in a command chain by using the COMMAND.sub.-- PACKET.sub.-- ADDRESS. The portion of the File Cache Handler that builds and initiates outboard file cache programs generates the PROGRAM.sub.-- ID. 2 0-35 COMMAND.sub.-- PACKET.sub.-- ADDRESS 460f is a value which contains the real address of the Command Packet to which the status applies. When a chain of commands is submitted to the outboard file cache XPC 102 for processing, the Command Packet Address will point to the Command Packet which caused an error. If all the Command Packets in the command chain were processed without error, then the Command Packet Address points to the last Command Packet in the command chain. 3 3-35 HARDWARE.sub.-- DEPENDENT.sub.-- STATUS-1 460g is an address within Main Storage 16 which was referenced and an error was detected. The File Cache Handler Software 208 takes the RECOMMENDED.sub.-- ACTION. 4 0-35 This word is reserved and is beyond the scope of this invention. 5 0-11 RECOMMENDED.sub.-- ACTION 460i is the processing that should be performed by the File Cache Handler Software 208 upon receiving a Program Status Packet. 5 12-23 REASON 460j indicates the condition that caused the particular status to be returned. 5 24-29 COUNT 460k is the recommended number of times that the File Cache Handler Software 208 should retry when responding to the status in the Program Status Packet. For example, if the RECOMMENDED.sub.-- ACTION returned is Resend, then the Count indicates the number of times which the File Cache Handler Software 208
should resend the Command Packet. If NO.sub.-- PROGRAM in the FLAGS field is not set and the RECOMMENDED.sub.-- ACTION does not equal "no action required", this field specifies the number of times the command specified by the Command Packet pointed to by COMMAND.sub.-- PACKET.sub.-- ADDRESS should be retried. Retries apply only to that command and not to any other commands in a command chain. All retries use the same Outboard File Cache Interface to which the original command was directed. If NO.sub.-- PROGRAM in the FLAGS field is not set and RECOMMENDED.sub.-- ACTION equals "no action required", COUNT must be equal to 0. If NO.sub.-- PROGRAM in the FLAGS field is set, this field is reserved. 5 30-35 FLAGS 460l is a set of bits that relay ancillary information. 5 30 PRIORITY.sub.-- DESTAGE indicates whether priority destage is required. If PRIORITY.sub.-- DESTAGE is set, then the Destage Request Packets in the Destage Request Table (see the READ Status Packet) refer to segments that must be destaged as soon as possible. If NO.sub.-- PROGRAM is set or DESTAGE.sub.-- REQUEST.sub.-- PACKETS is not set, PRIORITY.sub.-- DESTAGE must equal 0. 5 31 DESTAGE.sub.-- REQUEST.sub.-- PACKETS is a flag which indicates whether the Destage Request Table exists (see the READ Status Packet). If NO.sub.-- PROGRAM is set, or the status applies to an invalid command, or the status applies to a non-I/O command, then this flag must be 0. 5 32 TERMINATED.sub.-- POLLING is a flag which indicates that a Program Initiation Queue is no longer being polled. 5 33 Reserved. 5 34 NO.sub.-- PROGRAM is a flag which indicates whether the status is associated with a Command Packet. If NO.sub.-- PROGRAM is set, then the status is not associated with a Command Packet. If TERMINATED.sub.-- POLLING is set, NO.sub.-- PROGRAM must also be set. If the Program Status Packet is returned via the Status Packet Queue, NO.sub.-- PROGRAM must equal 0. This flag is beyond the scope of this invention. 5 35 Reserved and is beyond the scope of this invention. 6 0-35 STATISTICS 460m is a set of codes which indicate how successful the XPC has been in avoiding destaging file data, speculating upon the future file access commands, and the time the XPC spent in processing the Command Packet(s). 7 0-11 RECOVERY.sub.-- TIME is used to indicate to a Host 10 that the outboard file cache XPC 102 is in the process of performing a set of actions to recover from an internal fault condition. The nature of the fault recovery prohibit the Outboard File Cache from responding to any commands received from a Host. When a command is received, it is not processed by the Outboard File Cache and is returned to the sending Host with a RECOMMENDED.sub.-- ACTION equal to "Resend." RECOVERY.sub.-- TIME is only used when the NO.sub.-- PROGRAM flag is not set and the RECOMMENDED.sub.-- ACTION is Resend. The value contained in RECOVERY.sub.-- TIME provides the number of six second intervals required to complete the necessary recovery actions. 7 12-35 See Words 8-127 8-127 These words contain information which is dependent upon the particular command in the Command Packet which is associated with the Program Status Packet. Words 7-119, referenced by 460n depend upon NO.sub.-- PROGRAM and COMMAND.sub.-- CODE (see the READ Status Packet), and words 120 through 127 are reserved for future use as referenced by 460o. ______________________________________
The discussion now returns to Command Packet processing as shown in FIG. 7. Before the enqueue Processing Node 308 writes an entry in the Program Initiation Queue 310, it first obtains the address of an available Program Status Packet 460 from the Status Packet Queue 316, as shown by Line 318. If the Valid Flag 460a in the Program Status Packet is 0, then the Program Status Packet is available for status reporting. The address of the Program Status Packet is stored in NEXT.sub.-- SP.sub.-- ADDRESS 456e in the Program Initiation Packet 456 in the Program Initiation Queue 310.
The Data Mover 110 continually monitors the Program Initiation Queue 310 for the presence of Command Packets 452 to process as shown by the Monitor and Retrieve Processing Node 320. A pointer to an entry in the Program Initiation Queue 310 is used for monitoring the Program Initiation Queue. If the VF 456a for the Program Initiation Packet 456 referenced by the pointer is equal to 1, then the Program Initiation Packet is valid and a Command Packet is available. If the VF equals 0, then the Program Initiation Packet is invalid which means there is no Command Packet available for processing; the same Program Initiation Packet is monitored until the VF is set. Line 322 represents the reading of a Program Initiation Packet from the Program Initiation Queue.
Where the VF 456a in the PIP is set, the Program Initiation Queue 310 pointer is advanced to the next entry in the queue, and the next entry is thereafter monitored. The Program Initiation Packet 456 with the VF set is then used to retrieve the Command Packet 452. The COMMAND.sub.-- PACKET.sub.-- ADDRESS 456e in the Program Initiation Packet is used to read the Command Packet from the Command Packet Data Structure 304 as indicated by Line 324.
The information in the Command Packet 456 is then written to one of the Activity Control Block (ACB) Buffers 326 which is local to the Data Mover 110, as indicated by data flow Line 328. There are three buffers used by the Data Mover 110 to manage Command Packets. Each of the ACB Buffers is described in greater detail in the discussion for the Data Mover. The Buffers are large enough for 16 entries, which allows for a maximum 16 Command Packets to be "active." When there are 16 active commands, the Data Mover 110 suspends monitoring the Program Initiation Queue 310 until one of the 16 commands is complete. In general, the ACB Buffers hold Command Packets and assorted control codes for the transfer of data between the Data Mover 110
and Main Storage 16.
After a Command Packet is written to the ACB Buffers 326, the Send Processing Node 332 reads the Command Packet 452 from the appropriate ACB Buffer as shown by data flow Line 332. The Command Packet is then sent via the Fiber Optic Cable 216 to the Host Interface Adapter 214 as shown by data flow Line 334. The Receive Processing Node receives the Command Packet and enters the Command Packet into the HIA ACB Buffer 338 as indicated by data flow Line 340.
FIG. 13 illustrates the HIA ACB Buffer. The HIA ACB Buffer 338 has 16 entries, respectively referenced 338-1 through 338-16, for managing activities. Each entry in the HIA ACB Buffer contains a Command Packet and Status Information associated with the Command Packet. Associated with each entry in the HIA ACB Buffer is an ACB Number. ACB Number 1 references the first entry 338-1 in the HIA ACB Buffer, ACB Number 2 references the second entry 338-2, . . . , and ACB Number 16 references the sixteenth entry 338-16.
The Monitor and Put Processing Node 342 monitors the HIA ACB Buffer 338 for the arrival of Command Packets. When a Command Packet arrives in the HIA ACB Buffer 338, the ACB Number associated with the HIA ACB Buffer entry is read as indicated by data flow Line 344. Processing Node 342 then puts an Activity Queue (AQ) Entry in the Activity Queue as shown by data flow Line 348. An entry in the Activity Queue 346 indicates to the Index Processor 236 that there is a Command Packet available for processing.
FIG. 14 illustrates Activity Queue, and FIG. 15 shows the information contained in each Activity Queue Entry. The Activity Queue 346 may contain up to n Activity Queue Entries, referenced in FIG. 14 as 347-1, 347-2, 347-3, . . . , 347-n. As shown in FIG. 15, word 0 of an Activity Queue Entry contains a MESSAGE CODE 347a, an ACBID 347b, a HIA UID 347c, and a HIA BPID 347d. Word 1 of the Activity Queue Entry contains a MESSAGE 347e. Each of these fields will be discussed in greater detail in the discussions relating to the Host Interface Adapter and Index Processor. But briefly, the MESSAGE CODE indicates the type of operation to be performed by the Index Processor 236. For an operation type indicating a new entry has been made in the HIA ACB Buffer 338, the ACBID indicates the ACB Number of the entry in the HIA ACB Buffer where the Command Packet information resides. The HIA Identifier field indicates the particular Host Interface Adapter 214 which put the Activity Queue Entry in the Activity Queue 346. In the interest of clarity, the description of the HIA BPID and the MESSAGE fields will be reserved for later sections of the specification.
The Monitor and Retrieve Processing Node 350 in the Index Processor 236 monitors the Activity Queue 346 for Activity Queue Entries. When an entry is added to the Activity Queue, Processing Node 350 reads the ACB Entry from the Activity Queue 346
as indicated by data flow Line 352. Based upon the information in the Activity Queue Entry, Processing Node 350 sends an ACB Request to the HIA 214 as shown by data flow Line 354. The ACB Request contains the ACB Number from the Activity Queue Entry.
Send Processing Node 356 takes the Command Packet from the entry in the HIA ACB Buffer 338 which is associated with the ACB Number specified in the ACB Request and sends the Command Packet to the Process Node 358 of Index Processor 236. Data flow Lines 360 and 362 show the flow of a Command Packet from the HIA ACB Buffer 338 to the Process Node 358.
Process Node 358 decodes the command contained in the Command Packet and references the Control Structures 364 which contain information for managing the available storage space in NVS 220 and referencing Cached Files 366 stored therein. For file access commands, File Information is read from the Control Structures 364 as shown by data flow Line 368. Based upon the File Information and the decoded command, Process Node 358 initiates the appropriate processing. For the rest of this discussion for FIG. 7 assume that either a read or write request was contained in the Command Packet, and the referenced file data is present in Cached Files 366.
Two pieces of information are returned to the HIA 214 from the Process Node 358: a Status and Address as indicated by data flow Lines 370 and 372. Both pieces of information are tagged with the ACB Number so that the Status and Address information are stored in the appropriate entry in the HIA ACB Buffer 338.
Read and Send Processing Node 374 and Receive and Write Processing Node 376 control the flow of data between the Data Mover 110 and the NVS 220. Processing Node 374 is active when file data is read from Cached Files 336, and Processing Node 376
is active when file data is being written to Cached Files 366. For both Processing Nodes 374 and 376, Data Transfer Parameters are read from an entry in the HIA ACB Buffer 338 as respectively shown by data flow Lines 378 and 380. The Data Transfer Parameters indicate the address within NVS 220 where the operation is to begin and the number of words to be transferred.
Read and Send Processing Node 374 sends a Reconnect Message to the Data Mover 110 as shown by data flow Line 382. The Reconnect Processing Node 384 on the Data Mover 110 receives the Reconnect Message and supplies the ACB Number in the Reconnect Message to Receive and Write Processing Node 386. Data flow Line 388 shows the ACB Number flowing from Processing Node 384 to Receive and Write Processing Node 386.
Receive and Write Processing Node 386 retrieves the Data Transfer Parameters from the appropriate ACB Buffer 326 as referenced by the ACB Number. Data flow Line 390 illustrates the Data Transfer Parameters retrieved by Processing Node 386 from ACB Buffers 326. The Data Transfer Parameters indicate the location in Application Storage 392 where the file data is to be written. As File Data is received by Processing Node 386, as shown by data flow Line 394, it is written to Application Storage
392. Data flow Line 396 shows the File Data flowing to Application Storage 392. In Host Interface Adapter 214, the Read and Send Processing Node 374 reads the referenced File Data from Cached Files 366 as illustrated by data flow Line 398.
As previously stated, Receive and Write Processing Node 376 writes file data to Cached Files 366. File Data is shown as being written to Cached Files 366 by data flow Line 400. The transfer of File Data from the Data Mover 110 to the Host Interface Adapter 214 is initiated by the Receive and Write Processing Node 376 by sending a Reconnect Message. Data flow Line 402 shows the Reconnect Message. The Reconnect Message contains an ACB Number which is forwarded to Read and Send Processing Node 404. The ACB Number is shown at Line 406. Read and Send Processing Node 404 obtains the Data Transfer Parameters from the appropriate ACB Buffer 326 as referenced by the ACB Number. Data flow Line 408 shows the Data Transfer Parameters. The Data Transfer Parameters indicate the real address in Main Storage 16 where the file data to transfer resides. Processing Node 404 reads the referenced File Data from Application Storage 392 as shown by data flow Line 410. Data flow Line 412 shows File Data being sent by Processing Node 404 in the Data Mover 110 to the Receive and Write Processing Node 376 in the Host Interface Adapter 214. The File Data is then written to Cached Files 366.
For each of Processing Nodes 374 and 376, when the respective data transfer tasks are complete, a Status is written to the appropriate entry in the HIA ACB Buffer 338. Data flow Lines 414 and 416 respectively show the writing of the Status for Processing Nodes 374 and 376.
Return Status Processing Node 418 reads the Program Status Packet from the HIA ACB Buffer 338 when an activity completes and sends the Program Status Packet to the Write Status Processing Node 420 on the Data Mover 110. Processing Node 420
writes the Program Status Packet to the appropriate entry in one of the ACB Buffers 326. Data flow Lines 422, 424, and 426 illustrate the flow of a Program Status Packet from the HIA ACB Buffer 338 to the ACB Buffers 326 on the Data Mover 110.
Once the Data Mover 110 has received a Program Status Packet in its ACB Buffers 326, the Program Status Packet can be returned to the File Cache Handler Software 208. Return Status Processing Node 428 reads the Program Status Packet from ACB Buffers 326. The Program Status Packet is then written to an available entry in the Status Packet Queue 316. The entry in the Status Packet Queue to which the Program Status Packet is written is selected from a queue of pointers to available entries in the Status Packet Queue 316. The File Cache Handler Software reads the Status from the entry in the Status Packet Queue 316 and returns the appropriate status to the application software from which the I/O request originated. Processing Node 430 and data flow Lines 432 and 434 illustrate the status reporting.
3. File Space Management
This section provides an overview of the logical organization and maintenance of storage space in the outboard file cache XPC 102. The preferred embodiment for this invention is operable with the file management and input/output systems (not shown) associated with the OS1100 and OS2200 operating systems (not shown) available from Unisys Corporation. Those skilled in the art will recognize that this invention could be adapted to the file management systems associated with other operating systems without departing from the spirit of this invention.
FIG. 16 illustrates the file space available in the outboard file cache. The File Space 502 is logically organized in Segments 503-0, 503-1, 503-2, . . . , 503-(n-1), wherein for this embodiment each Segment contains 1792 words. The number of Segments available varies according to the amount of RAM storage configured in the XPC 102. A Segment has the same logical format as a logical track, which is the basic unit of storage allocation in the 1100/2200 file system.
FIG. 17 shows the logical organization of a single Segment. Each Segment 503 contains 64 blocks, numbered consecutively from 0 to 63 and respectively referenced 504-0, 5041, 504-2, . . . , 504-63, with each Block containing 28 words.
FIG. 18 shows the logical composition of a Block. Each block is comprised of 28 words, numbered consecutively from 0 to 27 and respectively referenced 506-0, 506-1, 506-2, . . . , 506-27.
A Segment 503 may either be assigned or unassigned. Assigned means that the Segment is directly associated with a specific track on a Disk 106 which belongs to a particular file and contains data which belongs to that file. An unassigned Segment is not associated with any track or file. When the outboard file cache XPC 102 is first started, all Segments in the File Space 502 are unassigned. A Segment's transition from unassigned to assigned is initiated by Host 10 software and occurs when an appropriate command is sent to the outboard file cache XPC 102. The transition from an assigned state to an unassigned state (hereafter referred to as "deassignment") is jointly controlled by the Host 10 and the XPC 102. Any of the following three events may cause a Segment to deassigned .
First, a Host 10 may send a command to the outboard file cache XPC 102 which specifies that the identified Segment 503 is to be purged. Purged means that the identified Segment 503 should no longer be associated with the identified file. The Segment may thereafter be used for storing Segments of other files.
Second, File Space 502 in the outboard file cache XPC 102 may be in short supply. The Segment may be required to be assigned or "allocated" to a different file. The particular Segment 503 chosen depends upon the cache segment replacement algorithm implemented in the outboard file cache XPC 102.
Third, the outboard file cache XPC 102 may detect that a hardware condition has rendered the RAM space occupied by the segment unusable. The Segment is deassigned and is thereafter unavailable for future assignment.
Deassignment of a Segment may require that the data contained in the Segment be copied to the Disk 106 and track with which it is associated. For example, if a Segment to be deassigned contains data that does not also exist in the track with which it is directly associated, the track may need to be made current with the data contained in the Segment. The data transfer is called destaging.
If the need to deassign a Segment is detected and initiated by Host 10 software, the requirement to destage a Segment is also determined by Host 10 software. The outboard file cache XPC 102 may also initiate the deassignment of a Segment, and the decision whether the Segment must also be destaged is made according to the following rule: If the Segment contains data that is not in its associated track, the Segment must be destaged before it can be deassigned. This is initiated by sending a destage request from the outboard file cache XPC 102 to the Host 10. The Host 10 responds by transferring the data in the identified Segment(s) from the outboard file cache XPC 102 to Disk 106. When the Host 10 has completed destaging the segment(s), the Outboard File Cache 102 may deassign the segment(s). If the segment and its associated track contain identical data, then no destaging is required and the Outboard File Cache 102 may unilaterally deassign the segment.
FIG. 19 shows the logical division between Cache File Space, Nail Space, and Resident File Space in the File Space of the Outboard File Cache. The proportion of Segments allocated between Cache File Space 522, Nail Space 523, and Resident File Space 524 varies according to runtime requirements. Cache File Space is allocated Segment-by-Segment to files. As demand for Cache File Space increases, allocation of Segments is managed according to a cache replacement algorithm. Segments in Resident File Space are assigned to tracks of files which are to remain in File Space for an extended period of time. For example, Resident File Space may be used for files which are accessed frequently and for data which is recovery critical. The Segments in Resident File Space are not eligible for replacement by the cache replacement algorithm for Cache File Space. An overview of Cache File Space management and Resident File Space management is provided in the following paragraphs.
A Segment in Cache File Space 522 may either be "nailed" or "unnail