REGISTER  |  SIGN IN
United States Patent Application20030126356
Kind CodeA1
Gustavson, David B. ; et al.July 3, 2003
Memory system having synchronous-link DRAM (SLDRAM) devices and controller
Abstract
A SLDRAM System is provided with a plurality of in-circuit, calibratable memory modules and a memory controller for issuing unicast and multicast command packets to the memory modules. Command packets are transmitted over a unidirectional command link that includes a complementary pair of command clock lines, a command FLAG line and a plurality of noncomplemented command bit lines. Each of the command clock lines, command bit lines and the FLAG line is a SLIO transmission line. Data transfer operations are carried out in response to the command packets over one or more bidirectional data links that each includes two complementary pairs of data clock lines, and a plurality of noncomplemented data bit lines. Each of the data clock lines and the data bit lines is a SLIO transmission line. Each SLIO transmission line is single-end terminated and preferably tapped into by way of stub resistors.
Inventors:Gustavson; David B. (Los Altos, CA), James; David V.  (Palo Alto, CA), Wiggers; Hans A.  (Saratoga, CA), Gillingham; Peter B.  (Kanata, CA), O'Connell; Cormac M.  (Kanata, CA), Millar; Bruce  (Stittsville, CA), Crepeau; Jean  (Nepean, CA), Ryan; Kevin J.  (Eagle, ID), Lee; Terry R.  (Boise, ID), Keeth; Brent  (Boise, ID), Manning; Troy A.  (Meridian, ID), North; Donald N.  (Saratoga, CA), Rhoden; Desi  (Phoenix, AZ), Stracovsky; Henry  (San Jose, CA), Morooka; Yoshikazu  (Itami, JP)
Correspondence Name and Address:FLIESLER, DUBB, MEYER & LOVEJOY LLP Suite 400 Four Embarcadero Center
Gideon Gimlan, Esq.
San Francisco
CA
94111-4156
US
Series Code:176327
Filed:June 19, 2002
U.S. Current Class:711/105; 711/5
U.S. Class at Publication:711/105; 711/5
Intern'l Class:G06F 012/00
Claims

What is claimed is:
1. A memory module [110] for use in a memory system [100] having a command link [151] and one or more data links [155,156] where the command link includes a first pair of lines [151b] for carrying complementary, command clock signals [CCLK/CCLK#], where each data link includes a second pair of lines [155b] for carrying complementary, first data clock signals [DCLK0/0#], and where each data link further includes a third pair of lines [155c] for carrying complementary, second data clock signals [DCLK1/1#], said memory module comprising: a series of interconnect pins [400] extending between opposed first and second interconnect extremes [pins.sub.--1:64] wherein said series of interconnect pins includes: (a) a first pair of interconnect pins [28,29] for receiving said complementary, command clock signals [CCLK/CCLK#](b) a second pair of interconnect pins [24,25] for transceiving said complementary, first data clock signals [DCLK0/0#] of a first data link, where the second pair of interconnect pins are located substantially adjacent to the first pair of interconnect pins; and (c) a third pair of interconnect pins [26,27] for transceiving said complementary, second data clock signals [DLK1/1#] of the first data link, where the third pair of interconnect pins are located substantially adjacent to the first pair of interconnect pins.

2. A memory module [110] according to claim 1 wherein: (a.1) the first pair of interconnect pins are located substantially centrally within the series of interconnect pins.

3. A memory module [110] according to claim 1 for use in said system [100], wherein a first data link of the system further includes a first plurality of data lines [155a] each for carrying a noncomplemented data signal [DQ(i)], wherein said series of interconnect pins further includes: (d) a first plurality of noncomplemented interconnect pins of a same number as said first plurality of data lines, the first plurality of noncomplemented interconnect pins [417,450] being distributed in a substantially symmetric fashion about each of said first, second and third pairs of interconnect pins [427].

4. A memory module [110] according to claim 3 for use in said system [110], wherein the command link of the system further includes a second plurality of command lines [151a] each for carrying a noncomplemented command signal [CA(j)], wherein said series of interconnect pins further includes: (e) a second plurality of noncomplemented interconnect pins of a same number as said second plurality of command lines, the second plurality of noncomplemented interconnect pins [430] being disposed substantially adjacent to the combination [427] of said first, second and third pairs of interconnect pins.

5. A Synchronous-Link Dynamic Random Access Memory (SLDRAM) System [100] comprising: (a) a command module [150] for issuing unicast command packets [210] directed to uniquely addressable ones of plural memory units [111,211] and for issuing multicast command packets directed to addressable collections of said memory units; (b) a command link [151] for carrying the command packets; (c) one or more data links [155,156] for carrying data corresponding to packet-commanded data-transfer actions; and (d) one or more, in-circuit programmably-calibratable SLDRAM modules [110-180] each having one or more of said addressable memory units, where each SLDRAM module is capable of interfacing with the command link and at least one of the data links for appropriately responding to informational queries provided by command packets, for further appropriately responding to tuning (adjustment, or calibrating) commands provided by command packets, and for yet further appropriately responding to data addressing and other data-transfer-related commands provided by command packets.

6. A method for initializing a SLDRAM system [100] having one or more, in-circuit programmably-calibratable SLDRAM modules [110-180], said method comprising the steps of: (a) first initializing the SLDRAM system by broadcasting from a reference location [150] a predefined first synchronization sequence over command/address lines [CA(9:0)] and data lines [DQ(17:0)] of the system while simultaneously and synchronously outputting from the reference location a continuously-running clock train over one or more clock lines [CCLK,DCLK0,DCLK1] of the system for allowing one or more SLDRAM modules present in the system to each self-adjust local command-receiving circuits [117] and data-receiving [115] circuits of the SLDRAM module to synchronously recognize the predefined first synchronization sequence at the locality of the SLDRAM module; (b) second initializing the SLDRAM system by sequentially assigning identification codes [ID's] to individually-addressable, memory units [111,121] within the in-system SLDRAM modules; (c) third initializing the SLDRAM system by sequentially commanding each in-circuit SLDRAM module to adjust output levels of the SLDRAM module's data-clock driving circuits [118] and data-line driving circuits [116] to levels acceptable to an in-circuit memory controller [150]; (d) fourth initializing the SLDRAM system by sequentially commanding each in-circuit SLDRAM module to respectively output a predefined second synchronization sequence over the data lines [DQ(17:0)] of the system while simultaneously and synchronously outputting from the commanded SLDRAM module, a continuously-running clock train over one or more data-clock lines [DCLK0,DCLK1] of the system, this for allowing the in-circuit memory controller to command adjustments (e.g., individual phase changes) to local data-outputting circuits [116] and local data-clock outputting circuits [118] of the sequence-outputting SLDRAM module so that the memory controller will be able to synchronously recognize the predefined second synchronization sequence at the locality of the memory controller; and (e) fifth initializing the SLDRAM system by sequentially determining data read and data write latency times of respective ones of the in-circuit, individually-addressable, memory units.

7. A method for using a SLDRAM system [100] having one or more, in-circuit programmably-calibratable SLDRAM modules [110-180], said method comprising the steps of: (a) synchronously issuing command packets from a reference location [150] of the SLDRAM system using command/address lines [CA(9:0)] of the system and command-clock lines [CCLK/CCLK#] of the system for transmitting the command packets to one or more of the in-circuit-calibratable SLDRAM modules of the system, where first ones of the issued command packets individually or collectively address one or more individually-addressable, memory units within the SLDRAM modules and define a data-transfer operation to be carried out by the individually or collectively addressed memory units; and (b) causing the addressed memory units to responsively perform the defined data-transfer operation within a time slot that follows receipt by the memory unit of a respective command packet, where latency between the receipt of the respective command packet and the responsive performance the defined data-transfer operation is adjustable; and (c) wherein second ones of the issued command packets individually address one or more of the individually-addressable, memory units and define one or more respective data-transfer latencies for the individually-addressed memory unit.
Description


BACKGROUND

[0001] 1. Field of the Invention

[0002] The invention relates generally to memory systems that use inexpensive random access memory devices (e.g. DRAM devices). The invention relates more specifically to memory devices that may be programmably-calibrated while in-circuit.

[0003] 2a. Cross Reference to Related Non-Provisional Applications

[0004] The following copending U.S. non-provisional patent applications are fully owned by the owner of the present application, and their disclosures are incorporated herein by reference:

[0005] (A) Ser. No. 08/909,299 [Attorney Docket No. SLDM1001] filed Aug. 11, 1997, by inventors David B. Gustavson et al. and entitled, BIFURCATED DATA AND COMMAND/ADDRESS COMMUNICATIONS BUS ARCHITECTURE FOR RANDOM ACCESS MEMORIES EMPLOYING SYNCHRONOUS COMMUNICATION PROTOCOLS;

[0006] (B) Ser. No. 08/933,713 [Attorney Docket No. SLDM1004] filed Sep. 19, 1997, by inventor Bruce Millar and entitled, DE-SKEWING DATA SIGNALS IN A MEMORY SYSTEM;

[0007] (C) Ser. No. 08/933,710 [Attorney Docket No. SLDM1006] filed Sep. 19, 1997, by inventor Bruce Millar and entitled, HIGH-SPEED BUS STRUCTURE FOR PRINTED CIRCUIT BOARDS;

[0008] (D) Ser. No. 08/933,673 [Attorney Docket No. SLDM1008] filed Sep. 19, 1997, by inventors Peter Bruce Gillingham and entitled, READ/WRITE TIMING FOR MAXIMUM UTILIZATION OF BIDIRECTIONAL READ/WRITE BUS; and

[0009] (E) Ser. No. 08/987,328 [Attorney Docket No. SLDM1011] filed Dec. 9, 1997, by inventors David V. James et al. and entitled, DUAL DATA CLOCK FOR GLITCH-LESS TRANSFER OF TIMING INFORMATION.

[0010] 2b. Cross Reference to Related Provisional Applications The following copending U.S. provisional patent applications are fully owned by the owner of the present application, and their disclosures are further incorporated herein by reference:

[0011] (F) Ser. No. 60/055,349 filed Aug. 11, 1997, by inventors Kevin Ryan et al. and entitled, SLDRAM ARCHITECTURE;

[0012] (G) Ser. No. 60/057,092 filed Aug. 27, 1997, by inventors David B. Gustavson et al. and entitled, SLDRAM ARCHITECTURE;

[0013] (H) Ser. No. 60/057,687 filed Aug. 27, 1997, by inventors David B. Gustavson et al. and entitled, SLDRAM ARCHITECTURE

[0014] (I) Ser. No. 60/055,368 filed Aug. 11, 1997, by inventors David B. Gustavson et al. and entitled, A HIGH-SPEED MEMORY INTERFACE; and

[0015] (J) Ser. No. 60/069,092 filed Dec. 10, 1997 by inventors David B. Gustavson et al. and entitled, SLDRAM PROTOCOL AND COMMAND FORMAT.

[0016] 2c. Cross Reference to Other Documents

[0017] The following documents are cited here for purposes of reference:

[0018] (a) (SLDRAM White Paper) SLDRAM Architectural and Functional Overview by Peter Gillingham, MOSAID Technologies Inc., Aug. 29, 1997
SLDRAM Consortium;

[0019] (b) Draft 0.99 Oct. 14, 1996 IEEE Memory Interface (SyncLink) P1596.7-199X entitled, Draft Standard for A High-Speed Memory Interface (SyncLink) by the IEEE Project 1596.x Working Group including Bill Vogley of Texas Instruments Semiconductor Group and Dr. David B. Gustavson of Santa Clara University; and

[0020] (c) (Previous SLDRAM device specs) DRAFT/ADVANCE SLDRAM Inc. SLD4M18DR400 4 MEG.times.18 SLDRAM PIPELINED, EIGHT BANK, 2.5V OPERATION 400 Mb/s/pin SLDRAM, including original of August 1997, and later revisions of: Sep. 22, 1997; Dec. 4, 1997; Jan. 29, 1998; and Feb. 12, 1998.

[0021] 2d. Reference to Appended Documents

[0022] The following documents are appended hereto and form part of the present disclosure. If there are any discrepancies between any of the appended documents and the present, main specification of this application, each shall be deemed as internally consistent with what is described in that document or, main specification without disturbing the other. Cross-consistent details of the main specification and appended documents supplement one another:

[0023] (a) (Latest SLDRAM device specs) DRAFT/ADVANCE SLDRAM Inc. SLD4M18DR400 4 MEG.times.18 SLDRAM PIPELINED, EIGHT BANK, 2.5V OPERATION 400 Mb/s/pin SLDRAM, revision of: Jul. 9, 1998;

[0024] (b) "SLDRAM: High-Performance Open-Standard Memory", Peter Gillingham and Bill Vogley, IEEE Micro Magazine November/December 1997 pp 29-39;

[0025] (c) Draft 0.99 Oct. 14, 1996 IEEE Memory Interface (SyncLink) P1596.7-199X entitled, Draft Standard for A High-Speed Memory Interface (SyncLink) by the IEEE Project 1596.x Working Group including Bill Vogley of Texas Instruments Semiconductor Group and Dr. David B. Gustavson of Santa Clara University;

[0026] (d) Appendices I-IV of above-cited provisional applications including internal sub appendices thereof; and

[0027] (e) 1998 IEEE International Solid-State Circuits Conference (ISSCC), pp.160-161, 431, 126-127, 385, Y. Morooka et al., "Source Synchronization and Timing Vernier Techniques for 1.2 GB/s SLDRAM Interface".

[0028] 2e. Notice Regarding Copyright Claims

[0029] This application includes descriptions of algorithms that may be implemented by one or more computer programs. The owner of this application reserves the right to claim certain copyrights in said computer programs. The owner has no objection, however, to the reproduction by others of the descriptions herein of such algorithms if such reproduction is for the sole purpose of studying the disclosure to understand the invention or inventions described herein. The owner reserves all other copyrights in such computer programs, including the right to reproduce such computer programs in machine-executable form.

[0030] 3. Description of the Related Art

[0031] Dynamic memory or DRAM (dynamic random access memory) devices are well-known in the industry for providing inexpensive and relatively high-speed storage capabilities. The basic dynamic memory cell comprises a charge-storing capacitor and a gating element (e.g. a field effect transistor) for providing addressable access to the charge in the capacitor for sensing, refresh and overwriting.

[0032] Because of the simplicity of the basic dynamic memory cell, many such cells can be crammed economically into industry-standard sized integrated circuit (IC) dice. For example, present day home computers are typically provided with so-called SIMM's (Single Inline Memory Modules) that have 8 or 9 IC packages mounted thereon with each such IC package providing 64 Mb (64 Mega-bits) of DRAM storage. Such SIMM's may be used for providing the main system memory of the computers. Each IC device of the SIMM typically includes a monolithic silicon die or other semiconductor substrate with lithographically defined circuitry provided thereon or therein. The IC device further comprises a sealed package for protecting the substrate and its circuitry, and interconnect pins for providing signal coupling between the package-internal circuitry and external circuits. Multi-Chip Module (MCM) type packages may also be used.

[0033] There remains a long-felt desire in the industry to increase the storage density and speed of dynamic memory (e.g., DRAM) systems while simultaneously reducing costs. However, this is not a simple task. Part of the effort toward reduction of costs comes in the form of making ever-smaller dynamic memory cells. But that is not enough. Aside from a large number of dynamic memory cells, a dynamic memory device generally needs additional circuitry for organizing its dynamic memory cells into addressable words, refreshing data held in the memory cells, moving data along internal buses, and interfacing with external circuitry. All these additional circuits introduce varying delays to the data access times of the overall dynamic memory device.

[0034] Because of this, it has become difficult to mix and match different dynamic memory devices in a single system. For example, if a computer system requires so-called 70 nS (70 nanosecond) devices, it is not advisable to mix-in faster 60 nS devices while retaining the slower 70 ns devices. The differing response speeds of such mixed devices may create timing problems on the memory bus. Even if all the utilized devices are rated for a same 60 nS speed, sometimes timing problems may still arise if the mix of 60 nS devices is from different manufacturers. Tolerances may vary between manufacturers. Because of this each memory device may have slightly different parametric characteristics than that of its neighbors due to, for example, the use of different semiconductor technologies in their manufacture. This presents problems to users who have invested in a first set of memory chips from a first vendor and want to mix them in a same memory system with a second set of more modern memory chips obtained from a different vendor.

[0035] Despite such problems, the relatively low cost of dynamic memory (e.g., DRAM) devices has led to their incorporation into a wide variety of applications including serving as the main memory of both desktop and mobile computer systems, as well as providing image-storing services for real-time and high-resolution video systems. This broad range of applications imposes many demands on future generations of dynamic memory devices including desirabilities for: (a) providing yet-lower per bit cost and higher storage densities, (b) allowing for minimized power usage by each device, (c) providing for wide and sustainable read/write bandwidth capabilities, (d) reducing latency times between each access request and a corresponding read/write operation, (e) providing for easy scalability to deeper and/or wider data storage organizations, (f) permitting mixing and matching of memory devices in legacy systems that still use older technology devices, and (g) providing basic support for different kinds of hierarchical memory configurations.

[0036] In an attempt to meet a subset of these challenges, past generations of DRAM devices have evolved through a number of iterations over the years. Fast Page Mode (FPM) devices were early providers of higher speed access to previously opened pages of memory. Extended Data Out (EDO) devices provided for yet faster memory access by overlapping address decode and output operations. SDRAM (Synchronous DRAM) devices provided a higher speed interface by using synchronously-clocked data buses. DDR (Double Data Rate) devices began to take advantage of both edges on each clock pulse to increase throughput rate.

[0037] However, none of these evolutionary approaches (FPM, EDO, SDRAM, or DDR) are believed to be sufficient on their own for providing a general solution to the challenges that are expected to arrive in the coming years. Next-generation computer systems are expected to operate at ever-higher switching frequencies and use wider word sizes and deeper (larger) address spaces. Small amounts of skew between data and clock phases may become a problem. Small differences in the various' delays that are imposed on parallel signals may become a problem. For example, delay differences may arise due to minor differences between parallel transmission lines that carry parallel clock and/or data signals (e.g., the delay differences may be due to slight mismatches of impedances on printed circuit board traces) and such differences may become a problem. As switching frequencies increase, problems with intersymbol interference, crosstalk, general noise, and so forth are expected to increase. A more comprehensive approach is needed for anticipating such problems and for providing flexible mechanisms to deal with such problems.

SUMMARY OF THE INVENTION

[0038] A Synchronous-Link Dynamic Random Access Memory (SLDRAM) System may be provided in accordance with the invention to include: (a) a command module for issuing command packets to uniquely addressable memory units and/or addressable collections of such units; (b) a high-speed command link for carrying the command packets; (c) one or more high-speed data links for carrying data corresponding to packet-commanded data-transfer actions; and (d) one or more, in-circuit programmably-calibratable SLDRAM modules each having one or more addressable memory units, where each SLDRAM module is capable of interfacing with the command link and at least one of the high-speed data links for appropriately responding to informational queries provided by command packets, for further appropriately responding to tuning (adjustment, or calibrating) commands provided by command packets, and for yet further appropriately responding to data addressing and other data-transfer-related commands provided by command packets.

[0039] A system initializing method in accordance with the invention comprises the steps of: (a) first initializing an SLDRAM system by broadcasting from a reference location (e.g., from pins of a memory controlling module) a predefined first synchronization sequence over command/address lines (e.g., CA(9:0)) and data lines (e.g., DQ(17:0)) of the system while simultaneously and synchronously outputting from the reference location a continuously-running clock train over one or more clock lines (e.g., CCLK, DCLK0, DCLK1) of the system for allowing one or more SLDRAM modules present in the system to each self-adjust local command-receiving and data-receiving circuits of the SLDRAM module to synchronously recognize the predefined first synchronization sequence at the locality of the self-adjusting SLDRAM module; (b) second initializing the SLDRAM system by sequentially assigning identification codes (ID's and/or sub-ID's) to individually-addressable, memory units within the in-system SLDRAM modules; (c) third initializing the SLDRAM system by sequentially commanding each in-circuit SLDRAM module to adjust output levels of the SLDRAM module's data-clock driving and data-line driving circuits to levels acceptable to an in-circuit memory controller (command module); (d) fourth initializing the SLDRAM system by sequentially commanding each in-circuit SLDRAM module to respectively output a predefined second synchronization sequence (which can be the same as the first) over the data lines (e.g., DQ(17:0)) of the system while simultaneously and synchronously outputting from the commanded SLDRAM module, a continuously-running clock train over one or more data-clock lines (e.g., DCLK0, DCLK1) of the system, this for allowing the in-circuit memory controller to command adjustments (e.g., individual phase changes) to local data-outputting circuits and local data-clock outputting circuits of the sequence-outputting SLDRAM module so that the memory controller will be able to synchronously recognize the predefined second synchronization sequence at the locality of the memory controller; and (e) fifth initializing the SLDRAM system by sequentially determining data read and data write latency times of respective ones of the in-circuit, individually-addressable, memory units.

[0040] A system utilization method in accordance with the invention comprises the steps of: (a) synchronously issuing command packets from a reference location (e.g., from a memory controlling module) of an SLDRAM system using command/address lines (e.g., CA(9:0)) of the system and command-clock lines (e.g., CCLK/CCLK#) of the system for transmitting the command packets to one or more in-circuit-calibratable SLDRAM modules of the system, where first ones of the issued command packets individually or collectively address one or more individually-addressable, memory units within the SLDRAM modules and define a data-transfer operation'to be carried out by the individually or collectively addressed memory units; and (b) causing the addressed memory units to responsively perform the defined data-transfer operation within a time slot that follows receipt by the memory unit of a respective command packet, where latency between the receipt of the respective command packet and the responsive performance the defined data-transfer operation is adjustable; and wherein second ones of the issued command packets individually address one or more of the individually-addressable, memory units and define one or more respective data-transfer latencies for the individually-addressed memory unit.

[0041] A pins and/or traces distribution pattern in accordance with the invention, as seen across a plane that intersects the pins or the traces (which traces can be printed circuit board traces), is characterized by: one or plural linear series of pins/traces, with each series extending in a lateral first direction between outer extremes of a signal communicating path extending longitudinally in a second, generally orthogonal direction and wherein said pins/traces include: (a) a first pair of complementary command-clock pins/traces for respectively carrying complementary command-clock signals (CCLK, CCLK#), the first pair of command-clock pins/traces being disposed approximately midway between said outer extremes of said signal communicating path; (b) a set of command-packet carrying pins/traces for carrying command words (CA0:9) that are synchronized with the command-clock signals and define command-packets, the command-packet carrying pins/traces being disposed proximate to the first pair of command-clock pins/traces; (c) a, packet-flagging pin/trace for carrying a FLAG signal that may be used to identify the beginning or ends of consecutive command-packets, the command-flagging pin/trace being disposed proximate to the command-carrying pins/traces; (d) second and third pairs of complementary data-clock pins/traces for respectively carrying complementary first data-clock signals (DCLK0, DCLK0#) and complementary second data-clock signals (DCLK1, DCLK1#), the second and third complementary pairs of data-clock pins/traces being disposed proximate to the first pair of complementary command-clock pins/traces; (e) a set of data-word carrying pins/traces for carrying data-words (DQ0:17) that are synchronized with one or the other of the data-clocks (DCLK0, DCLK1), the set of data-carrying pins/traces being distributed symmetrically about the data-clock carrying pins/traces; and (f) data-output powering pins/traces for providing dedicated power rails (VDDQ, VssQ) for line-drivers that output signals onto the data-carrying pins/traces, the data-output powering pins/traces being distributed plurally and symmetrically adjacent to the data-word carrying pins/traces.

[0042] Other aspects of SLDRAM systems in accordance with the invention will become apparent from the below detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The below detailed description makes reference to the accompanying drawings, in which:

[0044] FIG. 1A is a block diagram showing a simplified, high-level view of a SLDRAM system in accordance with the invention;

[0045] FIG. 1B is a schematic showing a more detailed view of one embodiment of an SLDRAM system in accordance with the invention;

[0046] FIG. 1C is a schematic diagram for illustrating general aspects of a SLIO (SynchronousLink Input/Output) connection;

[0047] FIG. 1D is a schematic diagram illustrating a possible configuration of a controller mother board and module daughter boards;

[0048] FIG. 1E is a legend for symbols used in FIGS. 1B-1D;

[0049] FIG. 2A is a flow diagram showing in simple terms how commands, data clocks and data bus transfers may be pipelined in accordance with the invention;

[0050] FIG. 2B is a timing diagram showing in more detail how commands, data clocks and data bus transfers may be interleaved in accordance with the invention;

[0051] FIG. 3 is a block diagram of a buffered system;

[0052] FIG. 4 is a top-view pinout diagram showing an interleaving of interconnect pins on opposed sides of a bottom of a Vertically Standing Module Package (VSMP) in accordance with the invention;

[0053] FIGS. 5A-5B combine to form a schematic diagram of an SLDRAM module in accordance with the invention;

[0054] FIG. 6A is a block diagram of a memory controller in accordance with the invention;

[0055] FIG. 6B is a schematic of a DLL circuit and a matched delay circuit that may be used in the memory controller of FIG. 6A; and

[0056] FIG. 7 is a block diagram of a SLDRAM module having timing circuits that complement those used in the memory controller of FIG. 6A.

DETAILED DESCRIPTION

[0057] System Overview

[0058] FIG. 1A provides an introduction to a SLDRAM System 100 in accordance with the invention that satisfies at least some of the desirable characteristics set forth above, namely, (a) allowing for low per bit cost, (b) allowing for minimized power usage when devices are not in use, (c) providing for wide and sustainable read/write bandwidth capabilities, (d) where possible, reducing latency times between each access request and a corresponding read/write operation, (e) providing for easy scalability to deeper and/or wider data storage organizations, (f) permitting mixing and matching of memory devices that use different kinds of memory technologies where some are slower than others, and (g) providing basic support for different kinds of hierarchical memory configurations. These aspects will become clearer as various details are discussed in conjunction with the more detailed representation found in FIG. 1B.

[0059] A key feature of the SLDRAM System 100 shown in FIG. 1A is the provision of one or more, command-queriable and command-tunable memory modules (command Q/T-able modules) such 110 and 180.

[0060] The term `command-queriable` (command Q-able) indicates here that a command module 150 can send information-seeking, query packets to individually-addressable memory units (IAMU's) for discovering their internal organization and basic characteristics.

[0061] Examples of such internal organization information includes number of memory banks, number of memory rows per bank, number of memory columns per row, number of bits or words per column and number of parallel data output lines. Examples of basic characteristics include minimum and maximum latency times for accessing a specific column of an already-opened row in a given bank. Another such example is minimum and maximum latency times for accessing a specific column of a not-yet-opened row in a not-yet-opened bank.

[0062] An already-opened row in an already-opened bank is referred to herein as a `page` and its respective read/write latencies are referred to herein as `page read` (PgRd) and `page write` (PgW) latencies. The respective latencies for a specific column of data in a not-yet-opened row of a not-yet-opened bank are referred to herein as `bank read` (BkRd) and `bank write` (BkW) latencies. The minimum and maximum values for each of the PgRd, PgW, BkRd and BkW latencies can vary depending on the internal technologies used within given memory devices.

[0063] As will be seen, it is useful for the command module 150 to be able to discover what the internal organization and basic characteristics are of the various memory modules 110 through 180 that are present, and operatively in-circuit, within the SLDRAM System 100 so that the command module 150 can tune (calibrate) the memory modules while they are in-circuit and so that the command module 150 can properly interleave data transfers of such in-circuit tuned, memory modules to correspond with respective data time slots.

[0064] As used herein, the term `SLDRAM module` can refer to either a single SLDRAM IC device or to a daughter board containing a buffered group of SLDRAM IC devices. Each `SLDRAM module` can contain one or a plurality of individually-addressable, memory units (IAMU's). Each so-addressable memory unit (IAMU) may be assigned an ID (identification) number during system initialization. Thereafter, information-seeking, query packets can be directed from the command module 150 to respective IAMU's for discovering their internal organizations and basic characteristics. Calibrating packets may be directed from the command module 150 to respective SLDRAM modules for tuning each module to match in-circuit conditions. Data access (read and write) command packets may be directed from the command module 150 to respective, pre-calibrated SLDRAM modules for timely accessing their data.

[0065] In one embodiment, SLDRAM System 100 can support up to 256 IAMU's by using a set of nine identification bits, ID0 through ID8 within command packets. A first half (e.g., ID8=0) of the corresponding address space of 512 locations that is addressable by ID(0:9) is reserved for individually addressing (unicasting to) the IAMU's. At least one unique location within the second half (e.g., ID8=1) of the same address space is reserved for globally addressing all IAMU's. Preferably, this globally-addressing or broadcast code is defined as a setting of each of the ID bits to logic high (`1`). Other locations within the upper half of the address space may be used for addressing subsets of IAMU's in multicast fashion. In a second embodiment, SLDRAM System 100 can support up to 512 IAMU's by using a set of ten identification bits, ID0 through ID9 within command packets. In an alternate embodiment, the number of IAMU's is expanded by retaining ID0 through ID8 and supplementing these 9
address bits with additional, sub-ID bits. Of course, larger address spaces are also contemplated through the use of enlarged command packets that have more ID bits and/or more sub-ID bits.

[0066] In FIG. 1A, command packets travel unidirectionally over a CommandLink 151 that couples the command module 150 to each of up to eight further `loads` (SLDRAM modules 110 through 180). The CommandLink 151 comprises ten, single-end terminated transmission lines for respectively carrying command/address bits zero through nine (CA0:9) in parallel. These ten lines are collectively referred to as the CA bus 151a.

[0067] Successive CA0:9 command/address words (each 10-bits wide) are synchronized consecutively with the rising and falling edges of a command clock that is provided as a differential pair of signals, CCLK and CCLK# on two, single-end terminated transmission lines 151b. Description herein of a specific rising or falling clock edge refers to the noninverted signal (e.g., CCLK) rather than to its inverted complement (e.g., CCLK#).

[0068] The CommandLink 151 further comprises four command control lines 151c that respectively carry a FLAG signal (151c1), a LISTEN signal (151c2), a LINKON signal (151c3) and a RESET# signal (151c4). The references, 151c1-151c4 are seen in FIG. 1B rather than in 1A.

[0069] In one embodiment, command packets each consist of 4 consecutive, 10 bit wide words transmitted over the CA(9:0) bus 151a. The first word of a command is indicated by a `1` in a `01000` command-aligning SLIO stream provided on the FLAG line (151c1). The CCLK/CCLK# signals of transmission lines 151b are preferably free-running square waves during an active, LINKON=1 mode. An expected typical frequency for CCLK is about 200 MHz although other frequencies such as about 300 MHz, about 400 MHz, about 600 MHz, and so forth are contemplated as well. Both edges of the differential free running clock CCLK/CCLK* 151b are used by listening SLDRAM modules (110-180) to synchronously latch command words. Thus the 200 MHz version of CCLK provides a command bandwidth of 400 Mega-bits per second per CA line (400 Mb/s/pin). It is contemplated that alternate embodiments may use more CA words per command packet and/or a wider CA bus having more CA transmission lines 151a.

[0070] While the LISTEN line (151c2) is high the SLDRAM modules (110-180) are expected to monitor other parts of the CommandLink 151 for commands. When LISTEN is low there should be no commands present on the CommandLink and SLDRAM modules (110-180) are permitted to enter a power saving standby mode. Each SLDRAM module (110-180) should be able to exit this non-listening standby mode and accept a next command packet within 2 CCLK clock cycles (4 ticks) of LISTEN going back high (to logic When LINKON (151c3) is brought low the SLDRAM modules (110-180) may enter a shutdown mode in which the complementary CCLK/CCLK# signals 151b can be turned off to achieve a low power consumption condition on the CommandLink 151. The RESET# signal (151c4) is used to put the SLDRAM modules (110-180) into a known state such as on power-up so that initialization may begin.

[0071] At least one, bi-directional DataLink (DataLink_A) 155 is provided. This DataLink_A 155 is used by the command module 150 for receiving initialization-query responses and other data (e.g., read data) from each of the SLDRAM modules (110-180). It may also be used by the command module 150 for outputting write data to one or more of the SLDRAM modules. DataLink_A 155 comprises eighteen, single-end terminated transmission lines for respectively carrying data bits zero through seventeen (DQ(0:17)_A) in parallel. These eighteen lines are collectively referred to as the DQ_A bus 155a. Individual lines are respectively referenced by parenthetical number such as DQ(5)_A for line 5 of the DQ_A bus 155a. In one embodiment, aside from the command module 150, each DataLink such as 155 can support no more than eight electrical loads, each of a predefined maximum value of electrical capacitance (e.g., 3
pF).

[0072] Successive DQ(0:17)_A data words (each 18-bits wide) are associated consecutively with the rising and falling edges of a command-specified one of plural data clock signals. The command-specified data clock signal may be provided either as differential pair, DCLK0_A and DCLK0_A# on a respective first pair of single-end terminated transmission lines 155b or it may be provided as differential pair, DCLK1_A and DCLK1_A# on a respective second pair of single-end terminated transmission lines 155c. Description herein of a specific rising or falling data clock edge refers to the non-inverted DCLK0 or DCLK1 signal rather than to its inverted complement, the DCLK0# or DCLK1# signal.

[0073] Each of the DCLK0 and DCLK1 data-clock signals generally operates at or about the same frequency as the CCLK command-clock. Unlike the CCLK signal, the DCLK0 and DCLK1 data-clock signals are generally time-windowed rather than being free-running. The signal-originating point for each of the DCLK0_A and DCLK1_A data-clock signals should change to correspond with the source point of corresponding data that is to be placed onto the DQ_A bus 155a. As different devices take over mastery of the DQ_A bus 155a the origin point of the corresponding DCLK0_A or DCLK1_A signal also typically changes. Data transmission on the DQ_A bus 155a preferably occurs as bursts of at least 4 consecutive data words each, with each such data word being 18 bits wide (thus providing at least 72 bits of data transfer per burst). Data burst length may be specified by a corresponding command packet.

[0074] During read operations, the DCLK0 and DCLK1 signals are each originated from an assigned one of the SLDRAM modules (110-180) that has corresponding mastery over the DQ_A bus 155a for the read operation. During write operations, the command module 150 gives itself corresponding mastery over the DQ_A bus and the DCLK0 and DCLK1 signals are each originated from the command module 150. The phase and magnitudes of the DCLK0 and DCLK1 signals at their respective points of origin can vary depending on the location of the source point along on the DataLink 155.

[0075] Typically, a first of the SLDRAM modules (110-180) acts as an originating source for a first of the data-clock signals (e.g., DCLK0/0#) and as a source of a corresponding first data transmission burst while, in preparation for a successive second data transmission burst, a second of the SLDRAM modules (110-180) or the command module 150 begins to act as an active originating source for the second of the data-clock signals (e.g., DCLK1/1#). The second SLDRAM module or the command module 150
should simultaneously prepare itself to be the source of the corresponding, second data transmission burst. Such preparation preferably occurs in parallel with the first transmission so that the second data burst (which in this example happens to be synchronized with DCLK1/1#) may be immediately placed on the DQ_A bus 155a just after the first data burst (which in this example happens to be synchronized with DCLK0/0#) finishes. This allows for time efficient use of the shared, DQ_A bus 155a.

[0076] Both edges of each of the differential, and time-windowed data-clock signals, DCLK.sub.--0A/DCLK.sub.--0A# (155b) and DCLK.sub.--1A/DCLK.sub.--1A# (155c) are used to latch data words on the shared DQ_A bus 155a. Thus a 200 MHz CCLK version of SLDRAM System 100
provides a per data-link bandwidth of 400 Mega-bits per second per DQ line (400 Mb/s/pin). It is contemplated that alternate embodiments may use a wider DQ_A bus 155a having more than 18 transmission lines. The eighteen bits of each data word may be designated as desired, including two for carrying parity or other error-related code and the remaining sixteen for carrying corresponding data bytes of 8 bits each.

[0077] The illustrated, second DataLink_B 156 is optional and may be used in conjunction with at least some of the SLDRAM modules (such the illustrated module 180) that have data ports which are wider than DQ_A bus 155a. In the illustrated example, module 180 has a 36-bit wide, parallel data-transmission capability while module 110 has only a 18-bit wide, parallel data-transmission capability. All of the SLDRAM modules (110-180) convey their initialization information over the primary DataLink_A 155. However, if the command module 150 learns from the initialization process that module 180 has the wider, 36-bit parallel data-transmission capability, the command module 150 may instruct module 180 to convey the upper half of its 36-bit wide data by way of DataLink_B 156 while the lower half is simultaneously conveyed by way of DataLink_A 155. Thus, scaling to wider data paths and higher bandwidths is supported.

[0078] DataLink_B 156 is preferably structured in the same way as DataLink_A 155 to have eighteen, single-end terminated transmission lines for respectively carrying secondary data bits zero through seventeen (DQ(0:17) _B)) in parallel. These eighteen lines are collectively referred to as the DQ_B bus 156a. Similarly, successive DQ(0:17)_B data words (each 18-bits wide) are associated consecutively with the rising and falling edges of a command-specified one of plural data clock signals of the secondary link 156. The command-specified data clock signal of the DQ_B bus 156a may be provided either as differential pair, DCLK0_B and DCLK0_B# on a respective first pair of single-end terminated transmission lines 156b or it may be provided as differential pair, DCLK1_B and DCLK1_B# on a respective second pair of single-end terminated transmission lines 156c. Again, the outputting source of a respective one of DCLK0_B/B# and DCLK1_B/B# also acts as the outputting source of the corresponding data bursts on DQ_B bus 156a.

[0079] Although not shown, SLDRAM System 100 may have yet further data links in addition to 155 and 156. These additional data links may be similar to 155 or wider or narrower as appropriate for the different kinds of SLDRAM modules (110-180) used in the system 100.

[0080] Mechanically-speaking, for each DataLink (155, 156, etc.), the clock and data transmission lines of that DataLink should be run closely together so that they all exhibit substantially same transmission characteristics for the utilized clock frequency and so that data and clock transmissions closely track one another. This is why the DCLK.sub.--0A/DCLK.sub.--0A# lines (155b) and DCLK.sub.--1A/DCLK.sub.--1A- # (155c) are shown lumped together close to the DQ_A bus 155a. For the same reason, the DCLK.sub.--0B/DCLK.sub.--0B# lines (156b) and DCLK.sub.--1B/DCLK.sub.--1B# (156c) are shown lumped together close to the DQ_B bus 156a. Also, the CCLK lines should run close to the CA(0:9) and FLAG lines so that these exhibit substantially same transmission characteristics for the utilized clock frequency and mechanical layout. Additionally, the CCLK lines should run close to the DCLK0 and DCLK1
lines of at least one DataLink so that these exhibit substantially same transmission characteristics for the utilized clock frequency and mechanical layout. One embodiment follows the trace-wire dispersal pattern shown in FIG. 4.

[0081] A daisy chained, serial bus 152 is further shown in FIG. 1A. This serial bus 152 includes provision of an input SI terminal and output SO terminal on each module, 110-180 and also 150. Serial bus 152 is used on power-up and resets to serially step through the SLDRAM modules (110-180), to tune their command-receiving phases, and to thereafter assign unique ID's to each IAMU (individually-addressable, memory unit). Each SLDRAM module (110-180) may contain one or more respective IAMU's.

[0082] On power-up/reset, RESET# (151c4) is activated. In response all SLDRAM modules (110-180) and the command module 150 drive their respective SO terminal low. This signals to the next module in the daisy chain 152 that the previous module has not yet finished initializing. The command module 150 should be initialized first. Such initialization may or may not include output level tuning and/or clock tuning as appropriate for a specific implementation of command module 150. Initialization of the command module 150 may be self-mediated or controlled through a system bus 90. The system bus 90 may couple to a CPU (central processing unit) or to another, like high-intelligence system. The command module 150 may be a low cost, packaged device that supports only the minimal requirements set forth herein or it may be more complex, as desired. The intent here is to allow the basic SLDRAM System 100 (with the exception of system bus 90 and its interface) to be of low cost and compact size.

[0083] Once initialization of the command module 150 completes, it drives its SO terminal 152a to logic high ("1") for a first time in order to begin a broadcast initialization sequence. The command module 150 asserts mastery over both the DCLK.sub.--0A/DCLK.sub.--0A# bus (155b) and the DCLK.sub.--1A/DCLK.sub.--1A# bus (155c) and begins continuous transmission of the CCLK signal onto these buses 155b,c as well as onto the CCLK transmission lines 151b. At the same time and in synchronism with the CCLK signal, the command module 150 begins continuous transmission of a predefined first synchronization sequence onto each of the lines of the CA(0:9) bus 151a, onto each of the lines of the DQ(0:17) _A bus 155a, and onto the FLAG line (151c1). In one embodiment, this predefined first synchronization sequence comprises a repeat of the 15
-bit, pseudo-random SYNC sequence: `111101011001000` (four 1's, two 01's, two 100's, and a 0). This pattern provides every possible 4-bit sequence except 0000. Other similar sequences may of course be possible.

[0084] In one embodiment, the SYNC sequence is staggered across lines as indicated by Table 0.5.

1TABLE 0.5
SYNC PATTERN ALIGNMENT SIGNAL REPEATING PATTERN . . . FLAG 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0
. . . CA9 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . CA8 1 1 1 1 0
1 0 1 1 0 0 1 0 0 0 . . . CA7 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . CA6 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . CA5 0 0 0 0 1 0 1 0
0 1 1 0 1 1 1 . . . CA4 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . CA3 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . CA2 1 1 1 1 0 1 0 1 1 0 0
1 0 0 0 . . . CA1 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . CA0 1
1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . DQ17 0 0 0 0 1 0 1 0 0 1 1 0 1 1
1 . . . DQ16 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . DQ15 0 0 0
0 1 0 1 0 0 1 1 0 1 1 1 . . . DQ14 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . DQ13 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . DQ12 1 1 1 1 0
1 0 1 1 0 0 1 0 0 0 . . . DQ11 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . DQ10 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . DQ9 0 0 0 0 1 0 1
0 0 1 1 0 1 1 1 . . . DQ8 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . DQ7 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . DQ6 1 1 1 1 0 1 0 1 1 0
0 1 0 0 0 . . . DQ5 0 0 0 0 1 0 1 0 0 1 1 0 1 1 1 . . . DQ4
1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . DQ3 0 0 0 0 1 0 1 0 0 1 1 0 1
1 1 . . . DQ2 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . DQ1 0 0 0
0 1 0 1 0 0 1 1 0 1 1 1 . . . DQ0 1 1 1 1 0 1 0 1 1 0 0 1 0 0 0 . . . CCLK 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 . . . DCLK1 1 0 1 0
1 0 1 0 1 0 1 0 1 0 1 . . . DCLK0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 . . .

[0085] The predefined first synchronization sequence is known to each of the SLDRAM modules (110-180) and each enters into its own self-synchronizing operation wherein it adjusts internal parameters (e.g., internal clock phases and/or delays) so as to optimize local recognition of the predefined synchronization sequence in synchronism with a respective one of the locally-received CCLK, DCLK0 and DCLK1
signals. This will be better appreciated when FIG. 1B is discussed.

[0086] Once each of the SLDRAM modules (110-180) finishes its self-synchronizing operation, it sets its SO line high if its SI line is also high. The low to high transition at the end of the chain, at 152e, informs the command module 150 that all the present modules 110-180 have finished their respective self-synchronizing operations and are now in condition to receive command packets. If a predefined maximum time elapses and command module 150 does not see the expected low to high transition at the end of the chain 152e, the command module 150 may send a message to a system BIOS (Basic Input Output System, not shown) by way of bus 90 to indicate that SLDRAM System 100 has experienced an initialization error and that intervention is needed to fix the fault.

[0087] In response a successful return of a first low to high transition at the end of the chain 152e, command module 150 stops broadcasting the synchronization sequence over the CA, DQ and FLAG lines. Then module 150
brings its SO line 152a low. The low ripples through the serial chain 152
back to the last SI terminal 152e. In response, command module 150 drives its SO terminal 152a to logic high ("1") for a second time in order to begin an identification assignment sequence. Some time before this step, all the SLDRAM modules (110-180) should have reset respective identification registers within themselves to a broadcast-recognition value (e.g., a local ID and sub-ID's of all 1's).

[0088] The first SLDRAM module in the chain 152 that has the broadcast-recognition value as its ID (e.g., module 110) and sees the second ripple-high of its SI terminal (e.g., 152a) responsively enters into an identification assignment procedure. If the SLDRAM module (110) has more than one IAMU (not shown), each IAMU may have its own SI/SO terminal for continuing the identification assignment daisy chain internally within the SLDRAM module. Each IAMU can therefore individually declare its presence during the initialization process. In response to one or more, ID-register write-request packets, each successive IAMU within the SLDRAM module (110)) whose respective SI/SO state is 1/0, and whose current ID is all ones, accepts a next successive and typically-unique identification comprised of a main ID part (e.g., 9
bits) and optionally, a sub-ID part (e.g., 4 bits). Identification assignment is thus daisy chained from one IAMU to the next inside each of the SLDRAM modules (110-180) and similarly from one SLDRAM module to the next. Accordingly, when SLDRAM module 110 is finished, its SI line 152a will be high and its SO line 152b will be also be high. This SI/SO state of 1/1 causes SLDRAM module 110 to ignore further ID assignment commands. The next successive SLDRAM module takes over.

[0089] When the last SLDRAM module in the chain (e.g., 180) is finished, its SI line 152c will be high and its SO line 152e will go high. The end-of chain SO line 152e returns to the SI input of the command module 150 and is used for letting module 150 know that the end of the chain has been again reached. If a predefined maximum time elapses and command module 150 does not see the expected low to high transition at the end of the chain 152e, the command module 150 may send a message to the system BIOS by way of bus 90 to indicate that SLDRAM System 100 has experienced an initialization error during ID assignment and that intervention is needed to fix the fault.

[0090] When a successful return of the second low to high transition occurs at the end of the chain 152e, the command module 150 should know how many IAMU's (individually-addressable, memory units) are present in SLDRAM System 100 and it should know what unique (or nonunique) ID has been assigned to each such IAMU.

[0091] In a subsequent set of individualized query and tuning operations, the command module 150 may query each uniquely-identified IAMU to learn about its internal capabilities. It may then individually tune the DCLK and DQ output drivers of each SLDRAM module so as to optimize synchronous recognition at the receipt plane of the command module 150 of DCLK and DQ signals output from differently situated ones of the SLDRAM modules (110-180). These aspects will become clearer as various details FIG. 1B are discussed.

[0092] Following the individual query and tuning operations, the command module 150 may begin to send individualized data access and other command packets to each IAMU. It may also send group command packets to groups of IAMU's on a multicast basis. It may also send global command packets to all the IAMU's on a broadcast basis as will be seen. Signaling levels and directions within SLDRAM System 100 may vary according to Table 1.0. In Table 1.0, SLIO stands for a 3-level, synchronous link input/output signaling system as will be detailed by FIG. 1C. LVCMOS stands for a binary, low-voltage (e.g., 0-2.5V) CMOS signaling system. TTL or LVTTL may be used instead of LVCMOS if desired. MC is the memory controller (also referred to as command module) 150. (At times throughout this document, an asterisk `*` is used interchangeably with the pound sign `#` to indicated a complementary signal.)

2 TABLE 1.0
Signal Name Description Direction Level Command- CCLK/ Command MC-->SLDRAM SLIO Link CCLK* Clock FLAG Command MC-->SLDRAM SLIO Word Flag CA[9:0] Command MC-->SLDRAM SLIO Address Bus LISTEN Standby MC-->SLDRAM SLIO Mode LINKON Shutdown MC-->SLDRAM LVCMOS Mode RESET* Hard Reset MC-->SLDRAM LVCMOS DataLink DCLK0/ Data Clock 0
MC<->SLDRAM SLIO DCLK0* DCLK1/ Data Clock 1
MC<->SLDRAM SLIO DCLK1* DQ[17:0] Data Bus MC<->SLDRAM SLIO Serial SI Serial Input MC-->SLDRAM, LVCMOS SLDRAM--> SLDRAM SO Serial Output MC-->SLDRAM, LVCMOS SLDRAM-->MC

[0093] SLIO Signaling

[0094] The SLIO signaling system of SLDRAM System 100 may be viewed as an improved and more rigid version of the JEDEC SSTL.sub.--2 signaling system (Series Stub Termination Logic for 2.xV logic circuits). SLIO is used for high speed signaling. Output drive levels are more tightly specified in SLIO to achieve faster bus settling and improved noise margin. The Controller 150 and SLDRAM devices 110-180 each employ a noise-isolated, 2.50V V.sub.ccQ supply to power the respective V.sub.ddQ drain terminals of their respective output drivers. A corresponding noise-isolated, ground line is similarly used to power the respective V.sub.ssQ source terminals of their respective output drivers. See FIGS. 1B and 1C. Other parts of the system that operate at lower frequencies and/or are generally less sensitive to noise (e.g., LVCMOS lines) preferably use a more general 2.50V V.sub.cc supply rail and corresponding signal ground.

[0095] Each differential or nondifferential SLIO signal of the CommandLink 151 and of the DataLinks 155 (and optionally 156, etc.) is transmitted over a single-end terminated, transmission line. Preferably, each such SLIO line is resistively urged to a midpoint reference level of about V.sub.term=1.25V by a termination resistor R.sub.T of approximately 30
.OMEGA., see FIGS. 1B-1C) provided at the far end of the line, away from the command module (MC) 150. Such single end termination helps to save power consumption as compared against more-standard, double-ended termination systems.

[0096] As seen in FIGS. 1B and 1C, a series stub resistor (R.sub.S) of approximately 20 .OMEGA. each or a generally equivalent isolation means is provided at each tap point into each SLIO transmission line to help isolate the major part of the transmission line from load perturbations that may be presented by module inputs and outputs (roughly 3 pF each).

[0097] In FIG. 1C, circuit 130 is representative of a SLIO output driver such as may found for example inside the package of the command module 150. Circuit 140 is representative of a spaced-apart SLIO receiver such as may found for example inside the package of a SLDRAM module (e.g., 110). Clock source 136 is representative of a synchronization means such as may be further found for example inside the package of the command module 150. Clock source 136 is coupled by a transmission means 137
(e.g., an RF transmission line) to the locality of receiver circuit 140. The separation between the SLIO line driver circuit 130 and the SLIO receiver circuit 140 may be on the order of about 50 mm (millimeters) to 175 mm.

[0098] A major portion of an exemplary SLIO transmission line is represented by trace impedance elements 133 (Z.sub.trace1) and 135
(Z.sub.trace2) These elements 133, 135 may be generally implemented as printed circuit board traces on mother and daughter boards. Other forms of implementing compact transmission lines for carrying SLIO signals of 200 MHz or higher switching frequencies may used instead as appropriate.

[0099] A first tap into the exemplary SLIO transmission line 133/135 is represented by dot 134a and may occur for example at the MC-near end of the line. A second tap into the SLIO transmission line 133/135 is represented by dot 134b and may occur for example anywhere between the MC-near end of the line and the termination (139) far end of the line.

[0100] Element 131 (Z.sub.pack1) represents a first package interconnect impedance associated with line-driving circuit 130. The value of Z.sub.pack1 may vary depending on specific package design. In one embodiment, it should be no more than about a 3 pF capacitance to AC ground.

[0101] The internal representation of the SLIO line-driver circuit 130 may also vary depending on what specific IC technology is used. In FIG. 1C, element P1 represents in general, a current sourcing element that couples to the SLIO power rail, V.sub.ccQ. N1 represents in general, a current sinking element that couples to the corresponding SLIO ground rail. The illustrated, tristate CMOS design that includes P-channel MOSFET P1, N-channel MOSFET N1 and the OR and NOR gates may be used for supporting 3-level signaling that includes a HI-Z output state when OE is inactive (when OE# is high). Other equivalents that can support such 3-level signaling may be used instead.

[0102] Elements 130a, 130b represent programmably-configurable tuning means for tuning the respective V.sub.OH and V.sub.OL, binary output levels of line-driver 130. Although elements 130a, 130b are shown as being in series with elements P1, N1, it is to be understood that such an arrangement is conceptual. The tuning of the respective V.sub.OH and V.sub.OL, binary output levels may be instead performed by parallel circuitry in which the effective channel width of transistors such as P1, N1, is modulated, for example by enabling more or fewer transistors that operate in parallel with P1, N1. If such a parallel-tuning approach is used, voltage levels V.sub.ccQ and V.sub.ddQ are essentially the same. In one embodiment, V.sub.ddQ is about 2.50V, V.sub.ssQ is about 0V, and due to the source-to-drain on-resistances (or effective channel widths) of MOSFET elements P1, N1, the respective V.sub.OH and V.sub.OL output levels are approximately 1.85V and 0.65V at the output pin of module 130. The V.sub.term level is about midway between V.sub.OH and V.sub.OL such that source current flow is about +12 mA for representing a SLIO logic high (`1`) and sink current flow is about -12 mA for representing a SLIO logic low (`0`).

[0103] With this circuit configuration, the respective high (H) and low (L) voltage levels at second tap point 134b are about 1.60V and 0.90V. A locally-situated reference voltage level V.sub.ref that is provided at the SLIO receiver circuit 140 is positioned about midway between these H/L levels of tap point 134b and is supplied to one input terminal of receiver circuit 140, for example to a minus (-) input of comparator 143.

[0104] In the illustrated example of SLIO receiver circuit 140, a second input terminal of comparator 143 (e.g., the plus (+) terminal) couples to the SLIO transmission line 133/135 by way of tap point 134b, and by further way of isolating stub resistor 138 and package interconnect impedance 141 (Z.sub.pack2) The value of Z.sub.pack2 may vary depending on specific package design of circuit 140.

[0105] Further in this example, an output of comparator 143 couples to a D input of flip flop 145 and indicates whether at a given point in time, the received signal level is above or below the local V.sub.ref. Alternative methods for detecting the state of tap point 134b may be used. This is merely an example.

[0106] Further in this example, an internal first clock, ICLK.sub.1 of SLIO receiver circuit 140 controls the latching instant of flip flop 145. Alternative methods for defining the state-sampling time or times for tap point 134b may be used. This is merely an example. The internal first clock, ICLK.sub.1 may be generated in a variety of ways. For example, a phase-lock loop (PLL) circuit or delay-locked loop (DLL) circuit 146 may be coupled to clock source 136 by way of coupling means 137. A phase and/or frequency locked output 147a of the PLL/DLL circuit may be supplied to plural delay elements 148a, 148b, 148c, etc. Alternatively, a differently buffered route 147b may be used to couple means 137 to the delay elements 148a, 148b, 148c, etc. Some or all of the delay elements 148a, 148b, 148c, etc. may be programmably-adjustable so as to selectively provide different vernier values of phase shift over the cycle length of clock source 136. The respective, phase-shifted versions, ICLK.sub.1, ICLK.sub.2, ICLK.sub.3, etc. may be distributed about SLIO receiver circuit 140 as appropriate. Again, this is merely an example of how to establish vernier control over sampling time. Other approaches may be taken.

[0107] During power-up and periodic recalibration modes, the output drive levels of respective line drivers such as 130 should be tuned by the memory controller 150 to achieve and maintain over time, the expected 0.90 v and 1.60 v drive levels on the main SLIO transmission line (e.g., at tap points 134a and 134b). Transmission means 137 can be a second, so-tunable SLIO path. However, here we are focusing on the adjustments made for keeping SLIO transmission line 133/135 within calibration.

[0108] In FIG. 1C, C.sub.TAP represents a lumped and parasitic, impedance perturbation due to tap point 134b disturbing the distributed impedance of the main SLIO transmission line 133/135. This lumped, impedance perturbation can be attributed to a via hole passing through a printed circuit board (PCB) and/or a slot connector on a controller mother board that removably receives a SLDRAM module package or IC package. C.sub.PAD represents a lumped and parasitic, impedance perturbation due to an internal IC pad within receiver circuit 140. Stub resistor 132 helps to isolate the main SLIO transmission line 133/135 from load perturbations and signal reflections that may be presented to the line by driver 130. Stub resistor 138 helps to isolate the main SLIO transmission line 133/135 from load perturbations and signal reflections that may be presented to the line by receiver 140. Line terminating resistor 139
(R.sub.T) is approximately 30 .OMEGA..

[0109] Many factors can contribute to variations in the parametric values of elements such as 131-135, 138, 141, C.sub.TAP and C.sub.PAD, where these elements influence the signal levels and phases seen by comparator 143.

[0110] FIG. 1D illustrates a possible mechanical arrangement for an SLDRAM system. Here, a first transmission plane 149a represents a controller mother board. An orthogonally-disposed second transmission plane, 149b represents a SLDRAM module or daughter board. Node 134c is representative of a tap means that allows unit 149b to be removably inserted for tapping into the SLIO transmission line. An example of such connectors are shown at 182 and 188.

[0111] Distance d0 is a lead-in separation between the package pin of SLIO line driver 130.1 and stub resistor 132.1. Distance d1 is a first PCB trace length from the front end of the SLIO transmission line to tap point 134c. Distance d2 is a second line length from the tap point 134c to stub resistor 138.1 on the daughter board 149b of receiver of unit 140.1. Distance d3 is a third line length from the tap point 134c to an R.sub.T termination on the mother board 149a.

[0112] Designers of PCB's (printed circuit boards) and IC's (integrated circuits) are relatively free to optimize this arrangement as they see fit for reducing noise, cross-talk, package size and so forth. As such, the ultimate parametric values that are seen on each SLIO transmission line (e.g., 133/135 of FIG. 1C) may vary. In one embodiment, lead-in distance d0 is approximately 50 mm; the maximum trace length d5 on the mother board is approximately 175 mm; and inter-module spacing d4 is approximately 15 mm. Wire length d6 on each daughter board 149b from connector (182, 188) to the SLDRAM module (including through the stub resistor) is about 15 mm. Further in this one embodiment; trace widths are approximately 6-mils per track and track pitch is approximately 20-mils. The characteristic impedance of each SLIO trace taken by itself is about 60 .OMEGA. to 65 .OMEGA.. The presence of the closely-spaced stub resistors reduces the effective line impedance to about 30 .OMEGA. to 35 .OMEGA., which close to the value of the termination resistor. Ground lines and/or planes may be interleaved between SLIO traces to provide shielding and reduce cross-talk.

[0113] It should be apparent in view of the above description of FIG. 1C that in-circuit tuning during the initialization phase and during periodic re-calibration operations is expected to compensate for design differences, including different values of line distances d1, d2, d3, etc. (FIG. 1D); different values of package impedances such as Z.sub.pack1, Z.sub.pack2; different values of trace impedances such as Z.sub.trace1, Z.sub.trace2; different values of other load disturbances such as C.sub.TAP, C.sub.PAD; and changes over time due to voltage drift, temperature drift, mechanical reconfiguration of where different SLDRAM modules are inserted, and so forth. The tunable values include variable driver-output levels as represented in FIG. 1C by elements 130a, 130b and variable sampling times as represented in FIG. 1C by elements 146, 148a-148c.

[0114] FIG. 1E provides a legend of some symbols used in the drawings. A generally-tunable delay element or phase-defining means is represented by an oval and sine-wave tail as shown at 191. As is understood in the art, there are many ways to define the phase of a given signal. These techniques can employ analog or digital delay elements or latches that are strobed by differently phased clock signals or combinations of such elements. The phase defining operation can operate similarly on both rising and falling edges of a given signal or differently with respect to each edge. The general intent of symbol 191 is to convey that the phases of one or both of the rising and falling edges of the more signals (e.g., input and output) passing along the indicated path are adjustable on one or both of a fine and coarse adjustment scales as appropriate for the context of the illustrated path. For input signals, phase adjustment is to be understood to include some way of defining when the input signal is sampled so as to determine its state at the sampling time. For output signals, phase adjustment is to be understood to include some way of defining when the output signal transitions from one state to another (e.g., from low to high).

[0115] A coarsely and vernierably-fine tunable delay element or phase-defining element is represented by an oval and two sine-wave tails (one dashed) as shown at 192. A vernierably-fine tunable delay element or phase-defining element is represented by an oval and a dashed sine-wave tail as shown at 193. In one embodiment, coarse delay/phase tunings are provided in quantums of CCLK clock ticks while vernier-fine tunings are provided in fractional quantums of a CCLK clock tick. A CCLK clock `tick` is the time length between a rising and successively falling edge of a CCLK pulse.

[0116] Further in FIG. 1E, a generally-tunable, output-level defining element is represented by a circle and a sinewave tail as shown at 195. A "H" in the symbol indicates that it is for tuning the V.sub.OH level of the corresponding driver output while a "L" in the same symbol indicates that it is for adjusting the V.sub.OL output level of the tunable output driver. Thus, the composite symbol shown at 197 represents an SLIO transmitter with individually-tunable V.sub.OH and V.sub.OL output levels. The OE (output-enable) terminal of symbol 197 indicates that the symbol further represents a tristatable driver which provides a high-impedance (HI-Z) state at each of its respective outputs when OE is brought low (deactivated). In accordance with traditional symbolism, the output of the driver 197 that has an open circle (bubble) represents the inverted output while the solid output line above it carries the non-inverted complementary output signal. The dashed line coming from the middle right of symbol 197 indicates that it is a SLIO transmitter that is driven by the noise-isolated V.sub.ccQ and ground power rails.

[0117] Similarly, symbol 196 represents a SLIO receiver. The dashed line at one of its inputs indicates that SLIO receiver 196 is responsive to a locally-supplied V.sub.ref reference level for determining the state at its other, solid input terminal. Although not specifically shown, it understood that a SLIO receiver which receives complementary, inverted and non-inverted SLIO signals may employ differential techniques for removing common mode, noise before comparing against the locally-supplied V.sub.ref reference level.

[0118] Connections to SLIO transmission lines may be bidirectional such a same tap is used for both input and output. The multi-directed symbol at 198 represents a SLIO transceiver that provides bidirectional buffering. It is understood to be a combination of differential driver 197 and either two copies of SLIO receiver 196 (one for each of the differential lines) or a differential SLIO receiver.

[0119] Given the above introduction, reference is now made to FIG. 1B for a better understanding of details of the there illustrated SLDRAM System 100'. It should be understood with regard to FIG. 1B that various liberties were taken due to space limitations in the drawing. For example, LINKON line 151c3 is not shown extending to and coupling to all SLDRAM modules (110', 120', etc.). Such coupling is nevertheless understood. Similarly, the couplings of RESET# line 151c4, LISTEN line 151c2 and FLAG line 151c1 to all SLDRAM modules (110', 120', etc.) is not fully shown but nevertheless understood. Additionally, reference 155b,c is intended to cover four, single-end terminated, clock lines even though the drawing shows only two such lines at the local of reference 155b,c. (All four lines are shown at the local of elements 155j, 155k.) Moreover, symbol 118 is intended to correspondingly represent two SLIO transceivers coupling respectively to the four lines of 155b,c. Symbol 113a may represent a plurality of tunable delay or phase-establishing elements (e.g., each for a respective rising or falling pulse edge and/or each for a respective input or output signal) although only one symbol is shown at 113a. There are other examples of such drawing liberties as will become apparent from the below, detailed description.

[0120] In the illustrated SLDRAM system 100', power supply 101 provides a sequence of power levels upon system turn-on. First, a general LVCMOS voltage rail V.sub.cc (typically +2.50V) is activated. Then, after a predefined start-up delay of say 2 microseconds, the SLIO dedicated supply V.sub.ccQ (also typically +2.50V) is activated. Termination voltage V.sub.term is turned on thereafter. Reference voltage V.sub.ref may be turned on simultaneously with V.sub.term or shortly thereafter. SLIO receivers should ignore their inputs and output a default state at least until their local V.sub.ref is activated. V.sub.term is preferably turned on after V.sub.ccQ so as to avoid latch-up problems.

[0121] The V.sub.cc, V.sub.ccQ, and V.sub.ref power levels are respectively applied to the illustrated command module 150' and the daisy-chained sequence of SLDRAM modules 110', 120', etc. (only two shown) in accordance with the above sequence. It is understood that SLDRAM System 100' can have more than just the illustrated two SLDRAM modules 110', 120' and that general descriptions herein extend generally to such further SLDRAM modules. A free-running oscillator 103 is provided for generating the CCLK (command clock) signal from a reference point when LINKON 151c3 is active. If desired, a power saving switch 102 may be included to disable oscillator 103 when LINKON is low. The frequency, waveshape and voltage levels of oscillator 103 may be programmably adjustable or not, as desired. The oscillator output is coupled to the CCLK bus 151b by way of a differential-output, SLIO driver 151k. Line-driver 151k may have tunable output levels or not, as desired. Additionally, it may be tristatable or not, as desired. If it is tristatable, its OE terminal should be driven by the LINKON signal 151c3
so that power is conserved when LINKON is false. Typically, line-driver 151k and at least part of oscillator 103 will be included within the sealed package of command module 150' so that the pins of module 150' serve as a reference plane. Driver 151k is shown outside for clarity and to emphasize the point that data transfer operations of SLDRAM System 100' are mediated by synchronous command packets, where such command packets are synchronized to a reference point that outputs the CCLK signal. If desired, oscillator 103 may be replaced by a distant clock source at a distal end of system bus 90. A DLL or PLL may be used locally to regenerate a local reference version of that distal clock source.

[0122] Stub resistor 106a couples the non-inverting, CCLK output of line-driver 151k to SLIO transmission line 107a. Stub resistor 106b couples the inverting CCLK# output of line-driver 151k to SLIO transmission line 107b. Node 108a represents a first tap into line 107a by a CCLK-receiving part of a first SLDRAM module 110'. Node 108b represents another tap into line 107b by the CCLK-receiving part (119) of module 110'. Termination resistor (R.sub.T) 109a is provided at the end of line 107a and coupled thereafter to V.sub.term. Similarly, termination resistor (R.sub.T) 109b is provided at the end of line 107b and coupled thereafter to V.sub.term. In one embodiment, each SLIO transmission line such as 107a and 107b is structured to operate with a switching frequency of about 200 MHz or higher.

[0123] SLDRAM module 110' includes at least one individually-addressable memory unit (IAMU) 111. This IAMU 111 comprises a plurality of selectable memory banks. See for example, the selectable Bank0 (511a) in the illustrated group of banks 511 of FIG. 5A. Each bank includes a plurality of selectable rows. See for example, the selectable top row 511b of Bank0
of FIG. 5A. (Note that an already-opened (currently selected) row is referred to herein as a "page".) Each row comprises a plurality of selectable columns. See for example, column 511c of FIG. 5A. Each column (e.g., 511c) comprises a plurality of dynamic memory cells for storing a respective plurality of data bits. In one embodiment of IAMU 511 (FIG. 5A), each column stores 72 bits; each row contains 128 columns; each bank contains 1024 rows (1K of rows); and the IAMU 511 has 8 banks. This provides a DRAM data storage capacity of 9,437,184 bits per bank or 75,497,472 bits (72 Mb) for all 8 banks. Other configurations are possible. Each given IAMU can have a different number of banks, a different number of rows per bank, and a different number of columns per row. Different embodiments of system 100' can even have different numbers of bits per column. FIG. 5A merely provides an illustrative example.

[0124] A page read latency value is associated with-IAMU 111. In FIG. 1B such a PgRd latency is associated with coarse-and-tunable delay element 112a. This tunable PgRd delay 112a indicates at minimum how much time it would take to read an already-opened row (a page) within an already-opened bank after a page-read command packet is issued by command module 150'. The delay of tunable PgRd delay element 112a may be increased by the command module 150' to provide a longer delay as appropriate for scheduling page-read bursts. It is seen in FIG. 1B that the conceptual flow of read data is from IAMU 111 through PgRd element 112a and through output-level tunable line-driver 116 (tristatable) to the DQ_A bus 155a. The value of the PgRd latency 112a is generally tuned or set by the command module 150 during module-read initialization and subsequent, periodic recalibrations.

[0125] Another fine-and-coarse tunable delay element of FIG. 1B, namely 112b (BkRd) represents at least the latency delay of IAMU 111 for a given bank read operation where the bank read operation generally requires the selecting or opening of a specific bank, in combination with the selecting or opening of a specific row. The delay of tunable BkRd delay element 112b may be increased by the command module 150' to provide a longer delay as appropriate for scheduling bank-read bursts. Again, the data flow path is conceptually from IAMU 111 through delay element 112b, through line-driver 116 to the DQ_A bus 155a. In the actual device, the tunable delay is usually established by vernier shifting of phases of internal clocks that strobe respective latches of the data pipeline instead of using analog delay elements. However, any kind of delay technology may be used as appropriate. The SLDRAM system definition does not mandate a specific internal technology for each SLDRAM module.

[0126] Data read operation over DQ_A bus 155a continue in system 100' with the receipt by an SLIO transceiver 155d within the command module 150' of the SLIO transmitted signal. During initial read-calibration and periodic recalibrations, the tunable output elements 114 of line-driver 116 are adjusted by the command module 150' to optimize the transfer of data from line-driver 116 to transceiver 155d by way of bus 155a. Specifics of this optimization process is left to design choices made by designers of the overall system 100'. A main feature of the present invention is that SLDRAM module 110' may be manufactured with loose tolerances so as to minimize its costs. The in-circuit tunability of delay elements such as 112a,b and of level-controlling elements such as 114 allow for in-circuit matching of the characteristics of the SLDRAM module 110' to the characteristics of the in-circuit data bus 155a and to those of the in-circuit command module 150'.

[0127] In one embodiment, after individual ID's have been assigned, the SLDRAM system 106' is further initialized by sequentially commanding each in-circuit SLDRAM module 110', 12', etc., to adjust output levels of the SLDRAM module's respective data-line driving circuits (116, 126) and data-clock driving circuits (118, 128) to H and L levels that are acceptable to the in-circuit command module 150'. This may be done using DC levels as will be seen. After the H and L levels are tuned using command-tunable elements such 114, 124, etc, the SLDRAM system 100' may be further initialized by sequentially commanding each in-circuit SLDRAM module 110', 120' to respectively output a predefined second synchronization sequence (which can be the same as the first synchronization sequence, e.g., `111101011001000`) over the data lines (e.g., DQ(17:0) 156a) of the system while simultaneously and synchronously outputting from the clock-drivers 118, 128 of the commanded SLDRAM module, a continuously-running clock train over one or both of the data-clock lines (e.g., DCLK0, DCLK1) of the system. This allows the in-circuit memory controller to command adjustments (e.g., individual phase changes) to the delay elements such as 112a,b, 122a,b of the local data-outputting circuits and/or to delay elements (or phase-setting elements) such as 113a, 123a of local data-clock outputting circuits of the sequence-outputting SLDRAM module so that the memory controller will be able to synchronously recognize the predefined second synchronization sequence at the locality 156d, 155j, 155k of the memory controller 150'.

[0128] Matching of the in-circuit characteristics for data write operations follows a similar approach. During the initial broadcasting of the first synchronization sequence (e.g., `111101011001000`) by SLIO transceiver 155d and line-driver 155o of the command module 150', vernier delay elements such as 112d may be locally adjusted by their respective SLDRAM modules, 110', 120', etc. to optimally detect the predefined synchronization sequence. In one embodiment, wherein the initial broadcasting of the first synchronization sequence is aligned with the CCLK at the reference plane (the pins) of command module 150', vernier delay/phase units such as 113c,d are locally adjusted by their respective SLDRAM modules, 110', 120', etc. to define the sampling times of SLIO receivers 115, 117 such that units 115, 117 optimally detect the predefined synchronization sequence on the DQ and CA buses. In a subsequent, individual tuning operation, coarse delays such as those of PgW 112e and BkW 112f are set in accordance with commands sent from the command module 150'.

[0129] Thereafter, data flow during write operations conceptually flows from SLIO transceiver 155d, through DQ bus 155a, through vernier delay (or detection phase setting) 112d, through SLIO receiver 115, through a corresponding one of coarse delays 112e, 112f, and then into IAMU 111 for performing either a page write (PgW) or a bank write (BkW) operation.

[0130] Command output operations of the command module 150' are synchronized with the command clock (C_CLK/C_CLK#) of bus 151b, which CCLK generally originates from module 150'. Data write operations are similarly synchronized to the command clock by having the command module 150' drive a corresponding one of the D_CLK.sub.--0/0# or D_CLK.sub.--1/1# from respective SLIO transceivers 155j and 155k. On the other hand, data read operations are synchronized with a command-specified one of the D_CLK0 and D_CLK1 signals, where the command-specified data clock originates from the data-sourcing SLDRAM module 110', 120', etc. These SLDRAM module sourced signals are received respectively by transceivers 155j and 155k of command module 150'. SLIO transceiver 155d uses the received one of the D_CLK0 and D_CLK1 signals to sample its data from DQ_A bus 155a.

[0131] Within each SLDRAM module, internal clocks (ICLK's) of each SLDRAM module are formed as delayed counterparts of the CCLK signal. In SLDRAM module 110' for example, the differential SLIO receiver 119 acquires the CCLK signal from its respective tap points (108a,b) on lines 107a,b. The switching signal that is there detected by SLIO receiver 119 is operatively coupled to one or more vernierably tunable delay means (or phase-defining means) such as 113c,d for producing local ICLK signals. These local ICLK signals may be used during the broadcast synchronization operation to establish the signal latching phases of SLIO receivers such as 115, 117. Additionally, vernierably tunable delay/phase means such as 113a,b are operatively coupled to SLIO transceiver 118 for producing locally-phased versions of the DCLK.sub.--0A/DCLK.sub.--0A# (155b) and DCLK.sub.--1A/DCLK.sub.--1A# (155c) clock signals.

[0132] As seen, SLDRAM module 120' has its own set of separately tunable delay/phase means 123c,d and its own set of counterpart SLIO receivers 125 (for data read) and 127 (for command receive). The specific values of tuned delay in each respective SLDRAM module 110', 120', etc. can be adjusted in accordance with the impedances of respective local tap points such as 108a,b and in accordance with respective transmission line lengths present between the C_CLK source 151k and the respective circuits (e.g., 119, 115, 117, 129, 125, 127) within the corresponding SLDRAM module 110', 120', etc.

[0133] When signals flow the other way during a read operation, from a given SLDRAM module (110', 120') to command module receiver 155i, the respective module 110', 120' is responsible for generating a correspondingly phased data clock onto respective bus 155b or 155c. Due to space constraints, both of the DCLK0 and DCLK1 differential buses are shown in FIG. 1B as combination 155b,c above modules 110' and 120'. It is to be understood that the corresponding, differential bus-transceivers 118, 128 each actually represents two such SLIO differential transceivers, one dedicated to the D_CLK.sub.--0 bus 151b and the other dedicated to the D_CLK.sub.--1 bus 151c. The clock phase for each of these two data clock buses 155b,c are respectively controllable by tunable delay elements such as 113a, 113b and 123a, 123b.

[0134] Thus, the separate tap and transmission line characteristics attributed to each SLDRAM module and to each data clock bus 155b,c may be individually compensated for based on the specific in-circuit location of the data originating SLDRAM module relative to the command module 150'. This compensation accounts for the round trip path taken starting with the issuance of command packets from the command module 150, where these packets travel synchronously to an addressed SLDRAM module by way of the CA lines 151a while accompanying clock signals travel along the CCLK lines 151b. The pre-tuned SLIO receiver such as 117 or 127 of the addressed IAMU introduces the command at the appropriate time into its SLDRAM module because of the synchronization-time pre-tuning of delay/phasing means 112c and 122c, or 113c,d and 123c,d.

[0135] The addressed IAMU responds to its command blindly. Its response latency is compensated for by individually calibrated elements such as 112a,b; 112e,f; 122a,b; etc. Thus, in a read operation, compensation continues by having the data clock signal DCLK0/1 originate in the data-sourcing SLDRAM module and having it move through delay or phase-establishing elements such as 113a, 123a and local clock transceivers 118, 128 onto the DCLK bus 155b,c. At the same time, a correspondingly synchronized burst of read data moves from the addressed IAMU (e.g., 111, 121) and through delays 112a,b or 122a,b and the in-circuit-calibrated driver 116 or 126 back to the command module 150' by way of DQ_A bus 155a. Thus each SLDRAM module includes local compensating means for compensating for its unique in-circuit situation.

[0136] In other words, if data is to be sourced from line-driver 126 of second module 120'--and module 120' has substantially different latencies than first module 110'--the correspondingly tuned data clock (DCLK0/0# or DCLK1/1#) will be generated by transceiver 128 of second module 120'. Corresponding tunable phase-establishers 123a,b will be used to define the local phase of the originated DCLK signal. Corresponding tunable phase-establishing elements 122a,b and 124 will be used to define the local phase and H/L levels of the originated burst of read data. If the read data is to be instead sourced from differently positioned line-driver 116 of first module 110', the correspondingly tuned data clock will be generated by transceiver 118. Corresponding tunable elements 113a,b, may be used to define the local phase of the there originated DCLK signal. Corresponding tunable elements 112a,b and 114 may be used to define the local phase and H/L levels of the there originated burst of read data. As such, the unique positionings and other variances of each SLDRAM module 110', 120', etc. and of the command module 150' may be compensated for individually by appropriate tuning of the tunable elements in each SLDRAM module 110', 120', etc.

[0137] The tuning ranges that are available for each respective set of tuning elements may be set forth in a corresponding, local configuration memory (LCM) 110a, 120a, etc. of the respective module 110', 120', etc. The organization of memory banks, rows and columns should also be set forth in the corresponding LCM 110a, 120a, etc. for each IAMU ill, 121, etc. Local configuration memories such as 110a, 120a, etc. may come in mixed forms such as ROM (read only memory) or PROM (once-programmable memory) or EEPROM (electrically-erasable and re-writable nonvolatile memory) and registers or SRAM. These local configuration memories 110a, etc. are used for storing fixed data and variable calibration settings of their respective SLDRAM modules.

[0138] During initialization (but after ID assignment), the command module 150' should address each SLDRAM module 110', 120', etc. individually and tune its respective V.sub.OH and V.sub.OL output levels for line drivers such as that of 116 and 118 in module 110'. The command module 150' may do so by commanding the addressed SLDRAM module to output a selected one of a DC (direct current) `0` level and `1` level onto a corresponding one of the DQ_A bus lines 155a and/or the DCLK.sub.--0A/DCLK.sub.--0A# (155b) lines and/or the DCLK.sub.--1A/DCLK.sub.--1A# (155c) lines. Then in response to the levels detected at its end of each of these buses 155a, 155b, 155c; the command module 150' may command the addressed SLDRAM module to increment or decrement the tunable V.sub.OH and V.sub.OL levels until a satisfactory state is reached. In general, the command module 150' uses its local V.sub.ref to define what received V.sub.OH and V.sub.OL levels are satisfactory.

[0139] After V.sub.OH and V.sub.OL calibration, the command module 150' should instruct each SLDRAM module in turn to individually output the predefined synchronization sequence (e.g., `111101011001000`) onto each line of the DQ_A bus 155a. In FIG. 1B this output comes from line-driver 116 when 110' is the addressed module. At the same time, the command module 150' should instruct the addressed module to begin a free-running output of its local clock onto both the DCLK.sub.--0A/DCLK.sub.--0A# (155b) bus and the DCLK.sub.--1A/DCLK.sub.--1A# (155c) bus. In FIG. 1B these outputs comes from transceivers 118 (only one of a pair shown) when 110' is the addressed module. Then in response to the synchronized samplings detected at its end of each of these buses 155a, 155b, 155c; the command module 150' may command the addressed SLDRAM module to increment or decrement the tunable vernier setting of delay element such as 112a,b 113a,b until a satisfactory state is reached. That satisfactory state is of course, acceptable discrimination of the predefined synchronization sequence (e.g., `111101011001000`) on each line of the DQ_A bus 155a in synchronism with the received clock signals of each of DCLK buses 155b and 155c.

[0140] After the individualized read synchronization process completes, the command module 150' should instruct each SLDRAM module in turn to individually perform one or more Bank Read operations and one or more Page Read operations while all other SLDRAM modules remain quiet. These read commands should use both of the DCLK buses 155b, 155c, each in turn. A DCLK preamble waveshape (see FIGS. 2A-2B) should be seen on the respective DCLK bus 155b, 155c just before the responsive burst of read data comes in. The command module 150' should measure the time between its command issuance and its receipt of the DCLK preamble waveshape to determine what the current latency of the addressed SLDRAM module is. (During active RESET#, each SLDRAM module should have reset itself to have its own minimum page and bank latencies. Thus, the read latencies measured shortly after RESET should be the respective minimum page read and bank read latencies of each respectively addressed SLDRAM module.) Information collected about the current read latencies of all the SLDRAM modules may next be used to define a smallest-allowed value for each of the page read and bank read latencies for the SLDRAM System 100' as a whole. Typically, these smallest-allowed values will be the page/bank read minimums of the slowest-to-read-from SLDRAM module (110', 120', etc.) in the system 100'.

[0141] After the individualized read latencies measurements complete, the command module 150' should measure the write latencies of each SLDRAM module in turn. This may be done by having the command module 150' output an extended burst of write data onto the DQ_A bus 155a immediately in conjunction with an ungapped series of write commands. The ungapped series of write commands should address consecutive locations within the SLDRAM module that is being tested. The corresponding extended burst of write data should first contain a slate-cleaning code such as all zeroes. Then the operation is repeated over an upper subset of the same set of addresses with the extended burst of write data now containing a predefined unique sequence of codes such as the values, 0, 1, 2, . . . , 31. This latter operation assumes a write latency of zero. The addressed SLDRAM module will have a greater write latency however. As such, one or more of the early members of the unique sequence of codes will not be written successfully into the addressed SLDRAM module. The corresponding memory locations will still have the clean slate code. The first memory location that does get written successfully into will contain a respective member of the unique sequence of codes (e.g., 5, 6, 7, 8, etc.).

[0142] By simply reading back the memory locations to see which memory location is the first to have had a member of the unique sequence of codes over-written successfully thereat, the command module 150' (or an upstream, more intelligent master) can determine what the current write latency is of the SLDRAM module that is being tested. Again, these write latency tests should be conducted for both page write and bank write operations using respective ones of the DCLK buses, 155b and 155c, so as to obtain a fair measure of overall system performance. (During active RESET#, each SLDRAM module should have reset itself to have its own minimum page and bank latencies. Thus, the write latencies measured shortly after RESET should be the respective minimum page write and bank write latencies of each respectively addressed SLDRAM module.) Information collected about the current write latencies of all the SLDRAM modules may next be used to define a smallest-allowed value for each of the page write and bank write latencies for the SLDRAM System 100' as a whole. Typically, these smallest-allowed values will be the page/bank write minimums of the slowest-to-write-to SLDRAM module (110', 120', etc.) in the system 100'.

[0143] During initialization (but after ID assignment and read synchronization), the command module 150' may address each SLDRAM module 110', 120', etc. individually to collect from the addressed module, certain characteristics information stored within the respective LCM (110a, 120a, etc.) of the SLDRAM module. Either the command module 150' or an upstream more-intelligent controller (not shown but understood to couple by way of system bus 90) may store the collected information from the LCM's in its own configuration-tracking memory. In FIG. 1B such a configuration-tracking memory 150b is shown situated within the package of module 150'. If desired, the configuration-tracking memory 150b may be located outside the package of module 150 in another, more intelligent and upstream means that couples to module 150' by way of system bus 90. See FIG. 1A.

[0144] This configuration-tracking memory 150b may be used after initial calibration or after periodic re-calibration operations to keep track of the currently-tuned states of each of the SLDRAM modules 110', 1201, etc. and of their respective memory organizations. This information may be used to appropriately schedule data read and data write operations over a shared data bus (e.g., 155a) such that contention is avoided between SLDRAM modules 110', 120', etc. The tracking information may be further used to optimize read and write scheduling so that wasteful time gaps between data bursts may be avoided or at least minimized. Such minimization of wasteful time gaps helps to obtain maximal usage of the bandwidth provided by the shared data bus 155a.

[0145] It should be understood from FIG. 1B that dashed box 155x represents the resources of a first DataLink (A) corresponding to 155 of FIG. 1A while dashed box 156x represents the resources of an optional second DataLink (B) corresponding to 156 of FIG. 1A. Although not fully shown, the resources 156x of the optional secondary DataLink are understood to include dual differential transceivers such as 155j, 155k and a set of eighteen data line transceivers such as represented by 156d. All the calibration and measurement operations that have been described above for DataLink_A 155 may be further employed for the optional DataLink_B 156 if it is present. It is left to the command module 150`to read the LCM`s (110a, 120a) of the respective SLDRAM modules for determining whether the optional DataLink_B 156 is functionally present in the system and whether and how such an additional DataLink_B 156
should be used to improve read and write bandwidths.

[0146] FIG. 2A provides a simple, first example of how latency information for each SLDRAM module may be advantageously used to closely pack respective data bursts on a shared DataLink (e.g., 155) while avoiding contention. A first command packet 210 consisting of four 10-bit wide command words is posted onto the CommandLink 151 beginning at time to. Successive command words are synchronized with respective `ticks` of the C_CLK. A tick corresponds to the duration between a rising and subsequently falling edge of the C_CLK. A `1` in the 5 -bit FLAG sequence `01000` identifies the beginning of the command packet 210.

[0147] It is assumed here that command packet 210 is a data read command addressed to a given first SLDRAM module (e.g., 110'). The response latency of the addressed first module is represented by time passage 211. Starting at a time point t.sub.1, the addressed first SLDRAM module outputs a predefined preamble waveshape 212 onto a command-specified one of the D_CLK buses, 155b or 155c (assume it is D_CLK.sub.--0 here).

[0148] The predefined preamble waveshape 212 should have a duration that is not longer than that of two minimum data bursts (so that minimum bursts can be interleaved using the DCKL0 and DCLK1 buses). It should be distinguishable from an alternating train of successive 1's and 0's (10101010 . . . ) as well as from maintenance of a power-conserving HI-Z state. It should also include at least one L-to-H transition and one H-to-L transition for removing intersymbol interference (ISI). Furthermore, the preamble waveshape 212 should have a continuous period of at least two clock ticks where its level is all 1's or all 0's during which the receiving device can safely enable its DCLK clock input without seeing any transitional glitches on the DCLK lines. In one embodiment, the predefined preamble waveshape 212 has a duration of 5 DCLK ticks (5
data word time slots) and consists of the 5 bit L/H sequence: 00010.

[0149] A first data word output by the addressed SLDRAM module appears on the DQ_A bus 155a in alignment with an end 213 of the predefined preamble waveshape 212. At the same time, the data originating module (e.g., 110') toggles its assigned D CLK.sub.--0 bus with a windowed running of the data clock train immediately following the preamble waveshape 212. This windowed running of the data clock is used for synchronizing remaining words of its output data burst 230. Illustration here of the DCLK edges centering with, the middle of each data word is purely conceptual and not intended to imply that such is necessarily the case. The phase relationship between a given data clock signal and its data words depends on where physically along their respective transmission lines each one is observed. During data read operations, the phase of the DCLK at its point of origin will generally not be the same as the phase of the CCLK at its point of origin. This is what the illustration here of the DCLK edges centering with the middle of each data word is intended to convey. In general, the embodiments disclosed herein rely on source synchronous clocking. This means that a known phase relationship exists between clock edges and their corresponding data (or clock) words at the source point on the bus where the respective clock and data (or command) signals are injected into the bus. The clock edges and the corresponding data (or command) words should track one another very closely as they move along the bus for receipt by a distant receiving device. The receiving device is expected to have internal, fine vernier phase adjustment means for compensating for fine skews of phase that may occur as respective clock and data (or command) signals move into the receiving device.

[0150] In one embodiment, data burst 230 consists of a minimum of four consecutive data words, each 18 bits wide. The corresponding command packet 210 may define the burst length as being the minimum or longer. Thus, for each read command packet in general, a responsive data burst 230 is produced by the addressed SLDRAM module and this data burst is defined as consisting of M consecutive words, where M is an integer equal to or greater than 4, and more preferably where M is a whole number multiple of 4.

[0151] Command packet 210 finishes using the CommandLink 151 at time point t.sub.4. At a time point t.sub.10, which latter point can be coincident with t.sub.4 or it can come later, a next successive command packet 220
is placed on the CommandLink 151. Interpacket gaps such as 205 are optional and may be used to avoid contention on the shared DQ bus (e.g., 155a) between responsive data bursts.

[0152] In this example, the second command packet 220 is a read command that is addressed to a second SLDRAM module (e.g., 120') different from that targeted by first command packet 210. A second response latency 221
is associated with the reaction of the second SLDRAM module to second command packet 220. In accordance with the invention, the optional interpacket gap 205 should be adjusted such that the beginning 223 of the next data burst 240 (the one output by the second SLDRAM module) comes as close as possible to the end (214) of the first data burst 230 (the one associated with the first SLDRAM module) while avoiding contention on the shared DQ bus 155a. Preamble waveshape 222 may begin to be output by the originator (120') of the second data burst 240 onto the alternate DCLK bus (155c) at a time point t.sub.11 even while the originator (110') of the first data burst 230 is still using the first DCLK bus (155b) and the DQ_A bus 155a.

[0153] Note that the use of dual data clock buses 155b,c allows for simultaneous compensation for the different positionings or other attributes of spaced-apart SLDRAM modules (e.g., 110', 120') and that such use also permits close coincidence between the end 214 of a first data burst 230 output by one such module and the beginning 223 of a second data burst 240 output by another such module. (In the example where 214 is the end of burst 230, it is to be understood that burst 230
consists of just the 4 data words, DATA_WD#0 through DATA_WD#3 rather than the more generic case represented by DATA_WD#M. Second data burst 240 may comprise more than its illustrated 4 data words.)

[0154] Note further from FIG. 2A that the use of dual data clock buses 155b,c allows the recipient (e.g., command module 150') of each corresponding data burst to prepare ahead of time for receipt of the beginning (213 or 223) of each respective data burst (230 or 240) by locking onto the preceding, preamble waveshape 212 or 222. Preambles such as 212 and 222 may overlap in time as will be seen.

[0155] FIG. 2B provides a vertically aligned timing diagram that shows a series of Page Read and Page Write commands as may be issued by the memory controller 150 to respective ones of the SLDRAM modules 110-180. FIG. 2B shows the relationship of these commands and their responsive data bursts to the CCLK, FLAG, DCLK0 and DCLK1 signals. For purposes of illustration all responsive data burst lengths are shown as 4N (where N is length of a CCLK tick) although 4N, 8N and/or longer data bursts can be dynamically intermixed if desired.

[0156] The read access time to a pre-opened row in a pre-opened bank, also known as Page Read Latency (PgRL#0), is shown in FIG. 2B as a 12N distance (e.g., 30 ns for a 200 MHz CCLK) between the end of the first command packet 251 and the beginning of the first responsive data burst 261. Each SLDRAM module can have its own, different Page Read Latency value as well as different Page Write and Bank Read/Write latency values. Thus the illustrated PgRL#0 applies to a respective SLDRAM module #0 but not necessarily to other modules (e.g., not to an SLDRAM #2 which in FIG. 2B is shown to have a shorter Page Write Latency value).

[0157] The first two illustrated command packets 251, 252 (Rd.sub.--0a and Rd.sub.--0b) are Page Reads to different banks in a same SLDRAM #0. The corresponding read data bursts 261, 262 both appear on the same data bus 155a along with a companion DCLK0 signal on bus 155b. As indicated by bracket 271 the preamble waveshape consists here of the 5-bit times, 3-level sequence: `00010` wherefor `0` represents a L voltage level, `1` represents a H voltage level, and `-` represents a M voltage level, in other words the midlevel on the SLIO line between the H and L levels. In this example, the DCLK0 preamble waveshape 271 is immediately preceded by M train 270 (`- -`) and immediately followed by a subsequent, 8
ticks-long `101010 . . . ` train 272. The combination of preamble waveshape 271 and the subsequent `101010 . . . ` train 272 provides the memory controller 150 with synchronizing edges for preparing to, and for strobing-in the read data bursts 261, 262.

[0158] Because the first two Page Read commands 251, 252 are addressed to the same SLDRAM module, the first DCLK0 preamble 271 can provide pre-data synchronization for both of the first two Page Read bursts 261, 262. It is not necessary to insert a gap between the two 4N data bursts 261, 262. The data-outputting SLDRAM module itself can ensure that DCLK0 is driven continuously without any substantial glitch.

[0159] The responsive data burst 263 for the following Page Read command 253 (which command 253 is addressed to a different SLDRAM #1), should be separated from the preceding burst 262 by a data gap (Gap D1) of at least 2N duration to allow settling of the DataLink bus and to compensate for timing uncertainties between the data-clock generators of SLDRAM #0 and SLDRAM #1. A 2N data gap should be included each time control of the shared DataLink 155 passes from one module to another. This includes changeover of bus mastery, as such occurs in successive reads from different SLDRAM modules or as such occurs in read to write and w