Home
Patent Search
IMT Blog
REGISTER
|
SIGN IN
United States Patent
4463372
Bennett , ; et al.
July 31, 1984
Title
Spatial transformation system including key signal generator
Abstract
A spatial transformation system includes a processing system for each component of an input video signal. The processing systems respond to operator commands entered through a control panel to provide spatial transformation such as enlargement, reduction, translation, and three-dimensional perspective rotation. The system further includes a key processing system connected to operate in tandem with the video component processing systems to transform an input key signal indicating a portion of a video image to be viewed in exactly the same way as the image defined by the input video signal. The output key signal thus exactly defines the same portions of the output video image as it did for the input video image notwithstanding significant changes in image shape which may result from a selected given spatial transformation. A proportional switcher is connected to receive a transformed image from the spatial transformation system on one channel and a second video image on a second channel and mix them in proportion to the magnitude of the key signal generated by the key processing system. The image from one channel may thus be smoothly blended into the image of the other input channel to eliminate aliasing along a boundary between two adjacent images. The transformation system may further include a key signal generator providing an internal key signal defining a standard sized video image and a switch coupled to provide an operator selection between the internal key signal and an externally generated key signal. A cropping circuit responsive to operator controls may modify the selected key signal by imposing more restrictive limits on the boundary of the key signal selected viewing area than provided by the original key signal. This selectively restricted key signal is provided as an input key signal to the key processing system. By performing the same spatial transformation upon the key signal as upon the input video image, the key signal precisely defines the borders of the selected viewing region to eliminate undesirable demarcation lines between the boundaries of two intermixed video images when they are displayed.
Inventors:
Bennett; Phillip P.
(Alameda,
CA
)
, Gabriel; Steven A.
(Santa Clara,
CA
)
Assignee:
Ampex Corporation
(Redwood City,
CA
)
Appl. No.:
361576
Filed:
March 24, 1982
Current U.S. Class:
348/580
348/585
348/590
Field of Search:
358/22,183
U.S. Patent Documents
4163992
August 1979
Inaba et al.
4200890
April 1980
Inaba et al.
4240104
December 1980
Taylor et al.
4266242
May 1981
McCoy
4267560
May 1981
Ishikawa et al.
4278993
July 1981
Suzuki
4282546
August 1981
Reitmeier
Primary Examiner:
Murray; Richard
Attorney, Agent or Firm:
Fraser and Bogucki
Claims
What is claimed is:
1. A video special effects system comprising:
at least one video processing system coupled to receive an input video signal representing a video image and output an output video signal representing an operator selected spatial transformation of the video image;
a key processing system coupled to receive an input key signal defining a portion of the video image selected for viewing and to operate in tandem with the at least one video processing system to generate an output key signal which has undergone the same selected spatial transformation relative to the input key signal as the video image.
2. The video special effects system according to claim 1 further comprising a key signal generator generating as an output an internal key signal having characteristics for selecting a complete video image for viewing and an operator selectable switch having a first input coupled to receive the internal key signal and a second input connectable to receive an externally generated key signal, the switch being coupled to provide a selected one of the input signals to the key processing system as the input key signal.
3. The video special effects system according to claim 2 above, further comprising a cropping circuit coupled between the switch and the key processing system to selectively limit the viewing region defined by the input key signal under operator control.
4. A video transformation system comprising:
a proportional switcher having primary and secondary video signal inputs and a multibit digital key signal input, the proportional switcher being operable to generate an output video signal as a combination of video signals received at the primary and secondary video signal inputs in proportion to the magnitude of the digital key signal input;
a video signal source coupled to provide a video signal to the primary input of the proportional switcher;
a transformation processing system coupled to receive a second video signal and a multibit digital input key signal defining a region of the second video signal to be included in the output video signal, operable to perform a given spatial transformation upon both the second video signal and the input key signal to produce a spatially transformed second video signal and a spatially transformed multibit digital key signal having gradual transitions in magnitude, and coupled to provide the transformed second video signal to the secondary video signal input to the proportional switcher and the transformed multibit digital key signal to the key signal input to the proportional switcher.
5. The video transformation system according to claim 4 above, further comprising an object displayed against a contrasting background, the object having a size, shape and position corresponding to a desired viewing region of an image defined by the second video signal; and a video camera focused upon the, object the camera having a video signal component coupled to provide the multibit digital input key signal to the transformation processing system.
6. A video transformation system comprising:
means for receiving a video signal defining a video image and performing a selected spatial transformation upon the video image; and
means for receiving a key signal defining a key image indicating a region of the video image selected for viewing and performing said selected spatial transformation upon the key image.
7. The video transformation system according to claim 6 above, wherein the video signal receiving and transformation performing means includes three video signal component processing channels for a luminance signal component and two chrominance signal components.
8. The video transformation system according to claim 7 above, wherein the two chrominance channels each have a spatial resolution relative to the luminance channel of one-half in a vertical direction and one-fourth in a horizontal direction and the key signal receiving and transformation performing means has a resolution relative to the luminance channel of one-half in the vertical direction and one-half in the horizontal direction.
9. A video transformation system comprising an operator controllable processing system having Y, I, Q and K parallel channels coupled to provide identical operator selected spatial transformations upon respectively Y, I and Q components of a color television video signal and a key signal indicating a selected region of an image defined by the video signal.
10. The video transformation system according to claim 9 above, wherein the key signal is a multibit digital signal and the K channel of the processing system includes a video filter imposing upon a spatially transformed key signal gradual changes between different key signal magnitudes.
11. The method of generating a key signal which precisely defines a desired viewing region of a spatially transformed video image comprising the steps of:
generating a key signal defining a desired viewing region of a video image that is to be spatially transformed; and
subjecting both the video image and the key signal to the same spatial transformation to generate a transformed video image and a transformed key signal defining the desired viewing region of the transformed video image.
12. The method of generating a key signal according to claim 11 above, wherein the key signal and the transformed key signal both have a multibit dynamic resolution and further comprising the step of mixing the transformed video image with a second video image on a pixel-by-pixel basis in proportion to the magnitude of the transformed key signal.
Description
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to systems for mixing a plurality of video images to form a single composite image and more particularly to a spatial transformation system providing a key signal which precisely defines a desired image viewing region notwithstanding a spatial transformation of the image.
2. Discussion of the Prior Art
It is a common practice within the video broadcast industry to selectively mix two or more video images to generate a single combined image. A typical example occurs where a TV newscaster is shown describing a news event while a video image of the news event is shown as an insert located slightly to one side of the announcer. Typically this is accomplished by presenting both the primary image showing the announcer and the secondary image containing the insert to a mixer or switches. A one bit resolution or on/off key signal is also presented to the mixer. When the key signal is off the primary image is output by the mixer and when the key signal is on the secondary or insert image is output by the mixer to provide a combined video image at the output. The key signal may be either a high or a low monochrome intensity level or in some systems may be a chroma key representing a precisely selected color value. A chroma key processing system is described in U.S. Pat. No. 4,240,104
entitled "Measurement of Chroma Key Area in Television Systems".
Known key generating systems provide a rectangular viewing area in the 4:3 aspect ratio of a standard television set and provide operator control over the size and location of the keyed image region. The camera for the secondary image must then be carefully aligned so that the portion of the image which is to be displaced to the viewing audience is located within this key selected viewing region.
The problem of aligning the camera with the key selected viewing region is reduced somewhat by spatial transformation systems which are also known as special effect systems. These systems permit the camera to center upon the desired image activity in a normal manner with the image then being transformed in size and location under operator control to match the key selected viewing region. However, constraints imposed by the rectangular shape of the key selected viewing region severely limited use of functions of these transformation systems such as perspective rotation which result in an irregularly shaped output video image. Furthermore, problems occur in precisely aligning the secondary image with the key signal defined viewing region to eliminate unsightly boundaries and aliasing between the primary and secondary images.
SUMMARY OF THE INVENTION
A video transformation special effects system in accordance with the invention includes at least one video processing system coupled to receive an input video signal representing a video image and output a video signal representing an operator selected spatial transformation of the video image and a key processing system coupled to receive an input key signal defining a portion of the video image selected for viewing and to operate in tandem with the at least one video processing system to generate an output key signal which has undergone the same selected spatial transformation relative to the input key signal as the video image. Typically the video processing system includes Y, I and Q color video signal components and the key processing system generates an 8-bit key signal which may be utilized by a proportional switcher to combine the transformed video image with an image from another channel in proportion to the magnitude of the keyed signal with 256 levels of resolution. This permits a blending of one image into the other at an image boundary to provide a more pleasing combined image and eliminate aliasing along the image boundaries, particularly along boundaries which are not exactly vertical or horizontal.
The system further includes an operator selectable switch which may provide the input key signal in an internally generated full viewing area size or as defined by an externally generated key signal. As an example, the externally generated key signal may be generated as the luminance components of a camera viewing a white object upon a black or other contrasting background. The white of the object defines the key selected region and may have any desired regular or irregularly shaped boundary. A cropping circuit coupled between the output of the switch and the input to the key processing system permits operator selected vertical and horizontal boundaries to be imposed upon the key selected region which may be more restrictive than those provided by the key signal emanating from the switch.
The video special effects system thus provides a highly effective and adaptive, yet precise, system for spatially transforming a secondary video image in a selected manner and mixing the secondary video image with a primary video image in proportion to the magnitude of a key signal. This precise mixing of the two images occurs notwithstanding an irregular shape for the secondary image either before or after the spatial transformation or both.
BRIEF DESCRIPTION OF THE DRAWINGS
A better understanding of the invention may be had from a consideration of the following detailed description taken in conjunction with the accompanying drawings in which:
FIG. 1 is a schematic and block diagram representation of a video special effects system in accordance with the invention;
FIG. 2 is a pictorial representation of a control panel for the video special effects systems shown in FIG. 1;
FIG. 3 is a block diagram representation of a spatial transformation system in accordance with the invention;
FIGS. 4A, 4B, 4C and 4D are pictorial representations that are useful in understanding transportion;
FIG. 5 is a block diagram representation of the spatial transformation system;
FIG. 6 is a block diagram of a transposing frame store for the spatial transformation system shown in FIG. 1;
FIG. 7 is a memory map for the transposing frame store shown in FIG. 4;
FIG. 8 is a schematic and block diagram representation of addressing circuitry for the transposing frame store shown in FIG. 6;
FIGS. 9A and 9B are block diagram representations of a deinterlace filter;
FIG. 10 is a block diagram representation of a predecimator;
FIG. 11 is a block diagram representation of a filter for the predecimator shown in FIG. 10;
FIG. 12 is a schematic and block diagram representation of an interpolation decimation filter;
FIG. 13 is a schematic and block diagram representation of a vertical source address generator;
FIG. 14 is a schematic and block diagram representation of a horizontal source address generator;
FIG. 15 is a block diagram representation of a digital special effects system in accordance with the invention;
FIG. 16 is a block diagram representation of a horizontal to vertical transposing memory shown in FIG. 15;
FIG. 17 is a block diagram representation of a field store memory for the memory shown in FIG. 16;
FIG. 18 is a block diagram representation of an address and timing circuit for the field store memory shown in FIG. 17;
FIG. 19 is a block diagram representation of a motion sensitive de-interlace filter for the system shown in FIG. 15; and
FIG. 20 is a block diagram representation of an advantageous embodiment of chroma predecimation and interpolation decimation filters for the system shown in FIG. 15.
DETAILED DESCRIPTION
Referring now to FIG. 1, a video special effects system includes Y, I and Q video signal component processing systems 1320, 1322, 1324 respectively and a key processing system for an input key signal defining a key selected region of an image to be displayed and associated control circuitry. The control circuitry includes a control panel 1310, a panel processor 1308 coupled to sample control panel operator switch inputs in response thereto and output corresponding control signals, a high level controller 1314 coupled to receive, store and cause the execution of operator selected transformations by the component processing systems, and a transform composer and factorizer 1318 which responds to command signals from the high level controller 1314
to carry out the specific spatial transformations commanded by the high level controller 1314. A vertical source address generator 1326 and a horizontal source address generator 1328 receive commands from transform composer and factorizer 1318 to select particular spatial addresses within a video image being processed by the component processing systems 1320, 1322, 1324, and 12, to carry out the selected image transformation.
Except for slight variations discussed below, the four, Y, I, Q and K components of the processing system may be thought of as being identical and operate in tandem to provide identical processing of input image components. The 8-bit input key signal to K processing system 12 is treated as though it defines a fourth image component of the input video image. The component processing systems operate in response to operator commands to provide spatial transformations such as enlargement, reduction, translation, and three-dimensional perspective or nonperspective rotation. The component processing systems and the control thereof is fully described below.
A proportional switcher 14 has a channel A input receiving the Y, I and Q color television video signal components from the Y, I, and Q processing system components at a channel A input and Y, I and Q video color television signal components from a video source 16 at a channel B input. As an example, the video source 16 may provide a primary video signal such as a view of a TV news announcer while the channel A input provides a secondary signal source which is to be inserted within a selected region of the primary image. An 8-bit key input received as the output of a processing system 12 causes the channel A and channel B inputs to be mixed on a pixel-by-pixel basis in proportion to the value of the 8-bit key signal received for each given pixel. For example, a key signal value of 255 will cause pure channel A data to be displayed while a key signal value of 0 will cause pure channel B data to be displayed. A key signal value of 127 would cause a 50--50 weighted combination of the pixel for the channel A and the pixel for the channel B input images to be displayed as the corresponding pixel for the output image. At the boundary between the primary and secondary images, the image may thus be smoothly blended from one into the other in a manner which is pleasing to the viewer under control of the multibit key signal.
The outputs of the processing systems including key processing system 12 include video filters which preclude step function changes in output signal amplitudes and impose a gradual change in signal levels thereon. The transformed key signal from key processing system 12 thus inherently commands a smooth transition at the boundaries between the primary and secondary video images.
The key signal generator 24 generates a conventional key signal corresponding in size and shape to a full sized video image display. A switch 20 having an operator control mechanism 22 is connectable to receive the internally generated standard key signal at a first input and externally generated key signal at a second input. One of the first and second inputs is selectively gated by switch 20 to its output which is connected to a cropping circuit 34. Panel processor 1308 responds to cropping circuit commands entered by an operator through control panel 1310 to provide to cropping circuit 34 outer horizontal and vertical boundaries for the key selected image region selected by the key signal. For example, it might be commanded that the key selected region lie between columns 150 and 300 and between rows 100 and 200.
In the present example a 0 value on the key signal represents a command to display the primary image and a nonzero value represents a command to proportionately display the secondary image produced at the channel A input to proportional switcher
14. Thus, in the present example the input key signal output by cropping circuit 34 would be arbitrarily constrained to a value of 0 outside the operator defined cropping boundaries and permitted to have the value received from switch 20 within these cropping boundaries. It should be apparent that the cropping boundaries can be set to match the outer boundaries of a standard video image so that the cropping circuit 34 selectively has no effect upon the input key signal.
In the illustrated example, the externally generated key signal is provided as the 8-bit luminance video signal component of a camera 26 which is aimed at a surface 28 having an image comprising a white object 30 upon a black background. The white image may be of any arbitrary shape and when switch 20 is set to the external position, the region of the secondary image selected for viewing provided at the channel A input to proportional switcher 14 will correspond to the portion of the input video signal image which overlays the white object. It will be appreciated that both the input video signal and the corresponding input key signal may be selectively distorted by the spatial transformation imposed by the processing system components, but nevertheless the key selected region of the video image appearing at the channel A input of proportional switcher 14 will correspond to the portion of the input video image which overlays the white object 30. As an example, the white object 30 may be a star centered within the black background on surface 28 and the input video image may be the face of a celebrity centered within the image of the input video signal so that the face of the celebrity appears within the star. The Y, I and Q and K component processing systems may then be commanded to provide a selected size change and positional translation to produce as a combined output video image the primary video image provided at channel B with the insert of the celebrity face within the star at any desired position within the primary video image and with any desired size.
A dashed line rectangle 33 is shown superimposed upon the white object 30 to illustrate the arbitrary imposition of cropping boundaries upon the white object 30 by cropping circuit 34. In the present example, any areas of the input video image lying outside the rectangle 32 would be blanked from the channel A secondary image when combined with the primary image presented to proportional switcher 14 on channel B.
An advantageous arrangement of control panel 1310 is illustrated in FIG. 2. It includes a three axes rate-control joystick 1410, two status displays 1412, 1414 providing feedback to a panel operator, and several groups of pushbutton or key switch controls. By utilizing the key switch groups to specify modes, channels, and functions, a relatively complex set of controls can be implemented with a single three axes rate-control joystick 1410. With the joystick in the return or neutral position, no change of status occurs. With the joystick pushed to the right the selected X parameter is increased and continues to increase so long as the joystick is held to the right. The farther to the right the joystick is moved the faster the parameter increases. Similarly, the parameter decreases as the joy-stick is pushed to the left. The Y and Z axes operate in a similar manner. Motion of the joystick upwards towards the top of the FIG. 2 as shown represents an increase in the Y axis parameter while motion downward commands a decrease. For the Z axis control, counterclockwise rotation commands an increase in the parameter while clockwise rotation commands a decrease. While electrical connections have been omitted for clarity, it will be appreciated that the joystick as well as each of the key switches and the status displays is connected for communication with the panel processor 1308.
A channel select group of switches 1418 permits selection of one of four available channels for control of the associate video image. The last channel key selection determines the channel to which the transformation commands pertain. A clear group of switches 1420 permits the clearing of selected axes or alternatively a master clear for all axes back to the normal or input video state for a currently selected parameter. For example, if positioning (translation) has been selected and the joystick moved to the right to cause the video image to move to the right, actuation of the clear X key will cause the image to return to its normal location. A mode select group of keys 1422 determines the overall operating mode of the transformation system and also facilitates the implementation of special features. Selection of the program key places the system in the program mode to permit the entry of transformation commands at each of the available knots starting with the first knot. Actuation of the rightward pointing arrow causes the selected knot number to be incremented while selection of the leftward pointing arrow causes the current knot number to be decremented.
Actuation of the run key places the system in a run mode with a stored sequence of knots being executed.
Actuation of the test key places the system in a test mode of operation in which diagnostic programs within the various microprocessor subsystems test for and indicate error conditions. Actuation of the duration key followed by one or more keys from a number key group 1424 specifies a total time in fields for an operating sequence. In program mode, the duration function specifies the transition time in fields from the current to the succeeding knot. A pair of keys labeled store effect and recall effect permit an entire effect or sequence of knots to be stored on a floppy disk and then recalled. The number group 1424 also includes enter and recall keys. This enter key permits a selected number to be entered in storage and terminates number entry. The recall key zeros the number being entered to allow erasure of errors.
A parameter selection group 1426 determines the meaning of the various axes of the joystick. An aspect/skew key causes the video image to be selectively enlarged or decreased in size in the horizontal and vertical directions in response to motion of the joystick in the X and Y direction respectively. At the same time, the Z axis control of joystick 1410 may be utilized to introduce a skewing of the video image. That is, the top of the image is translated relative to the bottom of the image so as to turn squares into parallelograms.
The axis select key positions in three dimensions the point about which image rotations occur. When this function is selected, a cursor is displayed to assist the user in positioning the point of rotation. All rotations occur about one of three mutually perpendicular axes passing through this center of rotation.
The locate key permits positioning in three dimensions of the incoming image.
A blur key permits the video image to be selectively defocused. Only the Z.sub.y or .theta. axis control of joystick 1410 is effective upon actuation of this key.
A position/size key permits horizontal and vertical translation of the output video image relative to the input video image using the X and Y axes while the Z axis control of the joystick controls the size of the output video image relative to the size of the input video image.
A rotate key permits control of three dimensional rotations of the image about the center point. Each of the joystick axes controls a corresponding axis of rotation. Vertical movement controls rotation about the X axis, horizontal movements controls rotation about the Y axis and rotation of the joystick causes rotation of the image about the Z axis. Any reasonable number of rotations may be specified. For example, zero rotations may be specified at one given knot with ten rotations being specified at the next knot. The interpolating capability of the high level controller 1314 will then cause ten rotations to occur between the given and the next knot. Multiple rotations are accomplished by actuating the joystick to cause rotation about a desired axis and maintaining the joystick actuated until the desired number of rotations have been counted.
A depth of perspective key is effective only with the Z axis of the joystick to control the rate at which objects become smaller as they move rearward of the plane of the initial video image or larger as they move forward of this plane, as by rotation about the X axis. This can be visualized by imagining the video image rotating about an X axis at the bottom of the image. As the image rotates away from the viewer the top portion of the image becomes farther from the initial plane and hence smaller. The depth of perspective key permits control over the rate at which the image becomes smaller relative to the angle of rotation.
A LUM HUE SAT key permits specification of the background color of the output video image in regions not occupied by the initial image. For example, in the above perspective rotation example, as the top of the image rotates away from the plane of the viewing screen, the top of the image becomes smaller and the upper right and upper left hand corners of the viewing screen are no longer occupied by the initial image. The LUM HUE SAT key permits the Y, Z and X axes respectively of the joystick
1410 to control the corresponding components of the background video image. This control over the background image can be especially useful when used in conjunction with a switcher 14 or 1306 which is programmed to respond to color or luminance keys to substitute video data of one channel for video data of another channel when putting together a composite image for a single channel.
A programming group of keys 1428 facilitates the programming of the various knots for a given video effect. Actuation of an insert knot key permits a new knot to be inserted between the current and previous knots while the remove knot key similarly permits a preprogrammed knot to be deleted from a sequence of knots. Actuation of a save knot key causes all of the parameters at a given knot with the exception of the rotations to be stored for later recall by actuation of the recall knot key. This save and recall feature is useful where the parameter state at a given knot is to be duplicated at a subsequent knot. Some or all of the parameters can of course be changed after duplication. Actuation of the pause knot key causes execution of the effect to halt at the current knot during run mode and await further user commands. A loop key actuation followed by selection of a duration number through the number key group 1424 causes a loop back from the last knot to the first knot of an effect. The duration of the transition from last to first knot is taken to be the number entered by the user after selecting the loop key. The loop back causes the intermediate sequence of knot states to be continuously and sequentially executed until the actuation of the stop key terminates the continuous loop sequence.
A freeze update rate key permits the specification through the number key group 1424 of the number of fields that a frozen video image will be held before being updated. That is, if the freeze update key is followed by the number 8 key, on every
8th field two new video frames will be sampled and held until the next update time. In effect, the horizontal-to-vertical transposing memory 1330 is inhibited from receiving a new input video field until the specified number of input video fields have occurred.
The control panel 1310 also includes H CROP and V CROP mode selection keys. In the H CROP mode the joy stick theta axis controls the horizontal width between the vertical cropping boundaries while the X axis controls horizontal translation of the uncropped region. Similarly, in the V CROP mode the theta axis controls the vertical width between the horizontally extending cropping boundaries while the Y axis controls vertical translation of the uncropped region.
As an example of entering an effect for a given channel, assume an example in which the full size picture is to shrink to one-half size at midscreen, rotate 360.degree. about the Y axis with perspective and then return to full size. After selecting the desired channel with switch group 1418, for example channel 1, the program switch in group 1422 is actuated to set all conditions to an initialized state yielding a full size picture, without any manipulation supplied. This initial picture condition is now the first knot point of the effect. The save knot key in group 1428 is now actuated to preserve the initial condition for later use at the end of the effect where the effect is to be returned to the initial condition.
A duration time is now specified through the number keyboard 1424 to define the time to the next or second knot. For example, the number 600 will cause the first knot time to be 600 field times or ten seconds. During this first 10 second interval the "zoom" from full size to half size will occur. The knot number is displayed on status display 1412 and as the knot time duration number is entered it appears on status display 1414. In general, the status display reflects the present knot as well as the state of a selected parameter. The forward or rightward pointing arrow in group 1422 is now actuated to cause incrementing to the next or second knot state. Actuation of this key closes the programming for the events of the first knot and opens the second knot for programming.
The 2D position size key is now actuated from group 1426 and the joystick can now be used to position the picture on the screen according to the XY movements of the joystick or can change picture size according to movement of the rotating knob atop the joystick in the Z or .phi. axis. For the present example the XY position is to remain constant while the joystick knob is rotated until the picture reaches one-half size. A duration time is now specified for the time to the next or third knot, which in this case will specify the time of the picture rotation. The duration button is actuated and a time such as 300 field times or 5 seconds is entered. With the programming of the second knot complete, the forward key is again actuated to close the second knot and open the third knot. The third knot is to define a 360.degree. rotation of the half-size picture about its Y axis with some three dimensional perspective effects added. The rotate key in group 1426 is now actuated to make the joystick active in rotating the picture about any of its three axes. In the present example the joystick is moved in a horizontal direction to the right to rotate the picture about its Y axis. Moving the joystick in a vertical direction would cause rotation about the X axis while rotation of the joystick knob in the Z or .phi. axis would rotate the picture about its Z axis. In the present example, the joystick is moved to the right and held until the image has rotated through a sufficient angle to permit observation of the perspective effect, for example 30.degree.-45.degree.. It should be appreciated that the rotation is required since the perspective effect is not observable until the image is rotated out of the plane of the viewing screen. With the image partially rotated, the joystick is released and the depth of perspective key is actuated in group 1426. The joystick .phi. control is now active in controlling the amount of perspective desired and the desired amount of perspective is added to the picture. The rotate button is now again actuated and the picture rotation through the desired full 360.degree. is completed by holding the joystick to the right until the rotation has occurred as viewed on the screen. A duration time is now specified for knot 3 by actuating the duration key and a set of number keys within group 1424. In this example let us assume that the time is entered as 600 field times corresponding to 10 seconds. In this case the 10 seconds will specify the time to knot 4 which is the unity or unaltered picture state. Thus during the final 10 second interval the picture will zoom back to full size.
The advance arrow key is now selected to close knot 3 and open the final knot 4. The recall knot button is actuated from programming group 1428 to store the previously stored initial full size or unity parameters in the current knot point 4. The effect is now complete and can be stored on disk by selecting the store effect button from the mode control 1422. Further editing of the effects in terms of durations and manipulation changes or additions can also be made by returning to additional knots, inserting additional knots at selected locations, or deleting knots.
Alternatively, the total run time of the total effect can be modified without changing the relative time durations between each knot point. For example, the commanded run time of the effect is 25 seconds. However, the total run time can be easily increased to 30 seconds, as for a 30 second commercial by actuating the duration key in group 1422 and then entering 1800 through keyboard 1424. Effectively each of the individual knot times is consequently increased by 30/25. That is, the first knot time will be effectively increased from 600 field times to 720 field times, the second knot point will be effectively increased from 300 field times to 360 field times and the third knot will be effectively increased from 600 field times to 720
field times. This will result in a total run time for the effect of 30 seconds as commanded.
While the Y, I, Q and K processing systems are substantially as disclosed below, certain modifications, combined with the commercial availability of higher speed memory components provide manufacturing advantages. As presently contemplated, the Y processing system 1320 has a sampling rate or resolution rate of 487 lines with 720 pixels in each line. The chrominance resolution provided by I processing system 1322 and Q processing system 1324 is half of the luminance resolution in the vertical direction or 244 lines and one-fourth the luminance resolution in the horizontal direction or 180 pixels per line. Similarly, the resolution of K processing system 12 is one-half the luminance resolution in the vertical direction or 244 lines and one-half the luminance resolution in the horizontal direction or 360 pixels per line. This selection of resolution values permits a sharing of memory circuits and memory control circuits within the I, Q and K processing systems 1322, 1324 and 12
respectively. Instead of storing the I and Q pixels in separate memories, they are stored as I, Q byte pairs in a common memory to provide a combined effective resolution of 360 bytes or pixels per horizontal line. This provides an overall size of
244.times.360 which exactly matches the resolution of the K processing system 12 so that internal field store memories for the combined I, Q and K processing system may be identified and may be operated under control of common control circuitry. Furthermore, by reducing the resolution by one-half in both the vertical and horizontal directions for a total reduction of one-fourth and by using higher speed memory components, complicated tesselating memories can be avoided. Instead, the memories are simply implemented as being two bytes wide to store horizontally adjacent pixel pairs in the case of the K processing system or corresponding I and Q byte pairs in the case of the I and Q processing systems. By reading or writing two bytes at a time, commercially available memory circuits have sufficient speed and bandwidth to accommodate required video data transfer rates in view of the reduced resolution. When accessing memory data in a horizontal direction the memories are simply randomly accessed with the addresses corresponding to pixel locations distributed along a horizontal row. Similarly, when accessing stored data in a vertical direction, the memories are simply randomly accessed with the address sequence defining pixel locations occurring along a selected column. In the event of vertical addressing, it will be recalled that within the K processing system 12 the memories produce horizontal adjacent pairs of pixels. Relatively small high speed column buffers are provided to buffer two columns of pixel data so that the accessed data can match the output requirements of the system. For example, as a second column of horizontally adjacent pixel pairs are read from a K processing system 12 memory, a previously buffered first column of pixel pairs is output to the key input of proportional switcher 14 with a first column of bytes being output from one buffer followed by a second column of bytes from the other buffer. It will be appreciated that these changes affect only the cost of manufacturing the special effects system and do not change the general nature of system operation.
SPATIAL TRANSFORMATION SYSTEM
Referring now to FIG. 3, a spatial transformation system 3010 in accordance with the invention which operates separately upon each dimension of a coordinate transformation is shown in the specific embodiment of a transformation system for a standard raster scan television video signal. The transformation system 3010 includes three color component processors 3012-3014, one for each of the Y, I and Q color components of a color television video signal. It will be appareciated that other representations for the television such as red, green, blue or Y, U, V signal could be used alternatively. Each of the component processors 3013 and 3014 may be implemented as duplicates of the component processor 3012 which is shown in greater detail in FIG. 3 and which will be described in greater detail herein.
The Y component processor 3012 receives as an input a Y digital video component of a video image in raster scan television order and is passed serially through a signal processing path 3016 which includes a horizontal to vertical transposing memory 3018, a vertical transformation system 3020, a vertical to horizontal transposing memory 3022, and a horizontal transformation system 3024 to produce a digital Y video output component which has been fully transformed in two dimensions, one dimension at a time. A transform composer and factorizor 3026 receives operator input commands and in response thereto generates transformation information for the separate vertical and horizontal directions which is communicated to a vertical address generator 3028 and a horizontal address generator 3030 respectively. Because the image transformations for each of the color components are substantially identical, the vertical and horizontal transformation information may also be communicated to I component processor 3013 and Q component processor 3014 without need for duplication of the transform composer 3026 for each color component. A timing and control circuit 3032 develops basic timing and control signals for use throughout the spatial transformation system 3010 in response to an incoming synchronization signal.
THEORY OF SPATIAL TRANSFORMATION
We describe a procedure for spatially transforming a two dimensional sampled image. Common examples of spatial transformations are translation, contraction and expansion, rotation and perspective projection. The concept however is quite general and includes any odd warping of an image such as that produced by a fish-eye lens or a fun house mirror.
Mathematically an image is determined by three functions of position that give the intensities of the three color components at each point within the boundary of the image. We denote our original or source image as
where u and v are linearly independent coordinates that range over the area of the picture and i selects one of the primary color components. The transformed target image will be written as
where x and y range over the area of the target. A spatial transformation is a relation that ties x and y to u and v such that the following is true
The primary intensities at each point (x,y) in the target are determined by those at some point (u,v) in the source. For each (x,y) there should be only one (u,v) to avoid the possibility of specifying two intensities for the same primary at the same point; thus the relation between them is a function of (x,y):
or
in component form. Any spatial transformation can be completely specified by giving its u and v components f.sub.u and f.sub.v. These functions simply tell where to look in the source to find the primary intensities at a point in the target. Many spatial transformations are invertible and are given by ##EQU1## These functions tell where in the target to move each source intensity. Since a transformation is the same for each primary we will drop the subscripts and write one respresentative equation for what is actually a group of three. We then have
If we are given a transformation in the form of eq. (10) we must first invert f.sup.-1 to get a relation of the form in (8) to be able to compute target points with eq. (12).
The problem of two dimensional spatial transformation is considerably simplified by the discovery that many transformations can be factored into a product of two one dimensional transformations. The factorization is derived as follows. What we seek is an intermediate r such that
then
with
The image r is produced from s by motion only in the second coordinate direction, since the first parameter in the equation relating the two is the same. Similarly r transforms into t by motion only in the first coordinate direction. To find g we have
and
For every y we can define a one dimensional function
If this function is invertible we may write
and substitute this into f.sub.v to get
Two important examples of spatial transformations are the affine and projective. An affine transformation in two dimensions is given by ##EQU2## in three dimensions by ##EQU3## and in general by ##EQU4## It is known that affine transformation of dimension N are isomorphic to N+1 dimensional matrices of the form ##EQU5## therefore the composite of two affine transformations can be calculated by taking the product of their respective matrices. Thus a general affine transformation can be built out of a product of simpler ones. Also the inverse of a transformation is found by inverting its matrix.
To use the matrix on an N-vector x, the vector is first mapped to an N+1-vector (x,1) by appending a 1 as its N+1.sup.th coordinate. The matrix M is then applied to this new vector forming an N+1.sup.th dimensional result. This is projected back to N space by dropping the N+1.sup.th coordinate which was undisturbed by M. As a two dimensional example we have the transformation in (20). In matrix form this is the 3.times.3 array ##EQU6## We map (x,y) to the three vector (x,y,1) and apply M ##EQU7## Dropping the third equation, which is an identity, we are left with (u,v).
If M is invertible we may express (x,y) as a function of (u,v) ##EQU8## This normally how transformations are specified. For calculation purposes though, we are given individual target coordinates (x,y) and must find what (u,v) in the source contributes intensity to that location.
Translation, scaling, rotation and shearing are all special cases of affine transformation. These four taken together can produce all possible affine mappings. The matrices and formulas for these are shown below for the two dimensional case. The transformations are described verbally in the source to target direction and we first show the M.sup.-1 that corresponds to that description. Translation of each source point (u,v) by a vector (T.sub.x,T.sub.y) to a ##EQU9## Matrix for source as a function of target: ##EQU10## Expansion by factors S.sub.x and S.sub.y ##EQU11## Clockwise rotation by an angle .theta. ##EQU12## Right shear of x coordinate by an angle ##EQU13## Note the simple relationship between each of these matrices and their inverses. If we are given a sequence of operations specified in the source to target direction and need the M corresponding to the composite target to source transform, we may find this M by inverting each matrix in the sequence and concatenating in the reverse order according to the formula
instead of inverting the composite directly. As an example, suppose we wish to rotate our source, then translate it. The M.sup.-1 for this is the product ##EQU14## by direct calculation using cofactors since det M.sup.-1 =1. This same result can be had by taking the reversed production of inverse ##EQU15## Three dimensional affine transforms behave analogously except that there are three matrices for rotations about X, Y and Z and three for shears along those axes. Projective transformations are given by the general form ##EQU16## These transformations are isomorphic to the set of all N+1 dimensional square matrices. Affine transformations are thus special cases of projective ones.
The distortion of distance produced when a three dimensional scene is projected onto a flat plane by a lens can be molded by a projective transformation. In fact, analysis of that distortion, called perspective, was the impetus for the creation of projective geometry.
Perspective distortion is quite familiar to anyone involved in art, architecture, photography, drafting, computer graphics, etc. A two dimensional perspective projection of a three dimensional scene is produced by dividing X and Y coordinates of each point in the original by its Z value, where Z points in the direction of view of the lens. Thus ##EQU17## This mapping collapses all points lying on a line passing through the focal point of the lens onto a single point in the view plane.
We can construct a two-dimensional projective transformation from a three dimensional affine one. The transformation models the image formed by a camera viewing a flat picture that has been rotated, sheared, scaled and translated to locations throughout 3-space. We start with an image in the u,v plane and map the points in it to the 3-space coordinate (u,v,0) and apply an affine transformation of the form (22) to obtain an x',y',z'). Dividing by z' we have ##EQU18## The a.sub.13, a.sub.23
and a.sub.33 terms are missing since w is zero in this case. Equations (28) and (29) are specifications for an f.sub.x.sup.-1 (u,v) and f.sub.y.sup.-1 (u,v). We want to invert and factor this transformation to obtain the f.sub.u (x,y) and g(u,y) needed in equations (14) and (15). Since we are starting with inverses the procedure for factorization is somewhat different from that described above. We first solve (2) for v to get g(u,y) directly. ##EQU19## Substituting for v in (28) solving for u we have after some manipulation ##EQU20## If the terms a.sub.31 and a.sub.32 are zero and a.sub.34 equals one, the projection reduces to an affine transformation within the plane and we have ##EQU21## A three dimensional affine transformation from a source array having three dimensional variables u, v and w to a target array having dimensional variables x, y and z would be defined by the generalized equation: ##EQU22##
Although the actual manipulations become quite extensive and are therefore hereafter omitted, it will be appreciated that equation (37) can be solved for u to produce
Determining u at each possible combination of values of v, w and z and using u as a source address to obtain data corresponding to each source address, a three dimensional first intermediate array of data is established having the coordinates v, w and z. The target dimension z has now been substituted for the source dimension w.
Next, substituting equation (38) into equations (35) and (36) to eliminate u, the result is
Equation (40) can now be solved for v to obtain
Determining v for each possible combination of values of w, y and z and using the determined a values as array address locations to obtain data from the first intermediate v, w, z array, a second intermediate array of data is established having dimensions w, y and z and values at coordinate points thereof corresponding to the addressed locations in the first intermediate array.
The final target matrix of data having dimensions x, y and z is obtained by substituting equation (41) into equation (39) to eliminate v. The result is
Solving equation (42) for w we obtain
The values of w can be determined for all possible combinations of values x, y and z and used as source address locations within the second intermediate w, y, z array to obtain data from the second intermediate array and establish the three dimensional target array T(x, y, z) as the values obtained from the second intermediate array at the locations defined by w, y and z for each possible combination of values x, y and z.
DISCUSSION OF REAL TIME VIDEO IMAGE TRANSFORMATION SYSTEMS
The preferred embodiment of the device accepts separate digitized versions of the Y, I and Q color components of a horizontal left to right scan NTSC television signal. This signal is 525 line, 2 to 1 interlaced, 8 bits per pixel, with a field rate of 60 Hz. Vertical scan is from top to bottom. The Y or luminance signal is sampled at 4 times the NTSC color subcarrier frequency of 3.579545 MHz (f.sub.sc). The I and Q signals are sampled at the subcarrier rate. We discuss the transformation of the Y signal first. I and Q are handled similarly.
The period between Y pixels is 1/(4f.sub.sc) or approx. 70 nS. There are exactly 910 pixels per 63.5 uS horizontal scan line. Only 486 of the 525 lines in a frame contain active picture data, the rest are devoted to retrace blanking.
The 8 bit parallel, byte serial, data stream enters the first transposing memory. This block contains three memories, each large enough to hold one active field of data. Every field the memory containing the oldest data is used as a buffer to store the current field while the previous two fields are read simultaneously from the other two for processing. This arrangement prevents timing conflicts that occur when trying to write new data to a memory that still contains parts of an old field not yet processed. Only that data representing visible picture is stored, thus each memory contains 243 lines of 768 pixels. The main function of the transposing memory 3018 is to change the direction of scan of the fields stored within it. Each field memory is written in horizontal order as shown in FIG. 4A, but can be read in vertical order as 768 columns of 243 pixels as shown in FIG. 4B. This produces a digital data stream representing a vertically scanned version of the input data. The horizontal and vertical dimensions of the picture are interchanged by this means. What was the left to right direction in the original becomes top to bottom and what was top to bottom becomes left to right. The output data stream can be considered as a horizontal scan of the original image mirrored about its vertical center line and rotated about its Z axis 90.degree. counterclockwise as illustrated in FIGS. 4B-4D. In this manner vertical processing of the input data can be achieved by operating on the output data with a device only capable of transformation along the direction of scan. Vertical processing of the original horizontally scanned signal is difficult because vertically adjacent samples are separated widely in time. After transposition, however, vertically adjacent samples are close together while horizontal ones are far apart.
Referring now to FIG. 5, two 70 nS data streams representing the two fields previous to the current input field leave a transposing memory 3018 to enter a deinterlace filter 600. These two fields together contain information describing the entire spatial area of the image, except that one field was scanned 1/60 second earlier than the other. The deinterlace filter 600 blends the two fields to create a new frame that appears to have been scanned at a time midway between them. The filter effectively operates at twice the original data rate of 4f.sub.sc. The deinterlace filter 600 is implemented as two filters in parallel and data from these filters is carried in two 70 nS streams. Throughout the machine, paralleling of data paths, memory modules and computational elements is used to prevent the data rate required on any single path from rising above 4f.sub.sc, while still retaining the enormous total rates required for real time processing. The machine is built with commonly available Schottky TTL logic devices which can comfortably respond to a 70 nS clock. The predecimation filter 700 has a triple line buffer memory, one memory absorbs the present column of data while the previous column is read from another. The third stores intermediate results. The predecimator 700 provides coarse size change by powers of two in the direction of scan. Each column is processed by the filter multiple times. Every pass of the filter reduces the length of the column by a factor of two until it is only one pixel long. Each pass takes half the time of the last and produces half as many pixels, therefore the total amount of pixels produced including the original is twice the length of a column since the sum of 1+1/2+1/4+1/8+ . . . =2. The predecimator output rate is thus twice its input rate and we require four 70 nS streams to carry its output to the interpolator-decimator filter.
An interpolator 800 has a double line buffer, each side of which is long enough to contain a column and all of its predecimated copies. The filter can interpolate between two pixels to a resolution of 1/64 of a pixel and vary its low pass frequency response on a point by point basis over a range appropriate for the smooth compression of a column to half its normal size. Compressions to less than half size are done by selecting one of the predecimated versions for interpolation and filtering. For example, if it is desired to compress the picture to 1/15 normal size the interpolator would select the 1/8 size decimated copy and interpolate and filter it to shrink it further by a factor of 8/15 a number between 1 and 1/2.
Referring now to FIG. 6, the transposing frame store 3018 includes three field buffer components 50-52 designated respectively field buffer 0, field buffer 1, and field buffer 2. Two multiplexers 54, 56 are coupled to output bytes of video field information from one of the field buffer components 50-52 in response to selection signals from a memory address and control circuit 58. Memory address and control circuit 58 also provides address and control information to each of eight components of each of the frame buffer stores 50-52.
The field buffer stores 50-52 operate on a continuous revolving basis in which one of the three field buffers receives an incoming field of data while the other two field buffers provide a newest complete field of data and a next oldest complete field of data to the newer and older field multiplexers 54, 56 respectively. A frame start signal provides the identification of the beginning of a frame interval while the pixel clock signal provides a basic clock signal at the incoming data rate.
The revolving nature of the field buffers 50-52 and the multiplex selection can be better understood by looking at what happens at three successive field time periods beginning with an arbitrarily selected field time N. At field time N field buffer 0 is selected to have incoming bytes of video data written therein while field buffer 1 outputs the oldest frame through older field multiplexer 56 and field buffer 2 outputs the newer field through newer field multiplexer 54.
At the next field time, N+1, field buffer 1 becomes the write field buffer while field buffer 2 outputs the older field through older field multiplexer 56 and field buffer 0 outputs the newer field through newer field multiplexer 54.
At the next field time N+2, field buffer 2 becomes the write buffer while field buffer 0 outputs the older field through older field multiplexer 56 and field buffer 51 outputs the newer field through newer field multiplexer 54.
At the next field time, N+3, the cycle repeats itself with field time N+3 being identical to field time N. It will be appreciated that during each cycle of three field times each field buffer is written into once and then read out through newer field multiplexer 54 and then read out through older field multiplexer 56. As a result, the older field multiplexer 56 always outputs field N-2 while newer field multiplexer 54 always outputs field N-1 where field N is considered to be the field which is currently being written into one of the field buffers 50-52. The two most recent stored fields are thus continuously output to the next stage and are updated for each new field time.
Read and write accessing of the frame buffers 50-52 is complicated somewhat by the fact that practically available memory storage chips cannot read and write at the 70 nanosecond pixel clock rate. In order to accommodate the required bandwidth, each of the field buffers is implemented as 8 modules of 32K.times.8 memory. By sequentially accessing the 8 modules, each individual module has 8 pixel clock periods to read and write a byte of data corresponding to a sampled pixel location. However, in order to assure proper sequencing of the memory modules for both the horizontal and vertical accessing which are required to obtain a horizontal to vertical transposition, care must be taken in implementing the addressing scheme.
One advantageous addressing scheme is shown by way of example for field buffer 50 in FIGS. 7 and 8. FIG. 7 illustrates the lower addresses of an address map for field buffer 50. The 1 byte memory components 0-7 are represented vertically in ascending order from top to bottom while hardware memory word addresses ascend from left to right as indicated immediately above the map. However, for convenience of address implementation these memory addresses may be further divided into row and column addresses which are indicated above the chip address in FIG. 7.
Horizontal accessing of the first row is the most straightforward. Horizontal accessing begins with address 0 of component 0 and proceeds through the components in sequence. After address 0 has been written in module 7 the column address is incremented with pixel (row, column) position (0,8) being accessed at word 1 of module 0. The 768 pixels of the first row of a field are written into the first 96 word positions of the memory modules in sequential order.
In the event of a vertical access, it must be remembered that the 2 pixels located at column 0 and rows 1 and 2 will be accessed in sequential order. Care must therefore be taken that these two pixels are stored in sequential memory modules and not in the same memory module. This is accomplished by storing pixel 1,0 in module 1 with the word address being skipped to address 128 which corresponds to a resetting of the column address to 0. The memory modules are then again accessed in sequence with a wraparound to component 0 before the word address is incremented to column address 1 which corresponds to chip address 129, Similarly, for the second row the first pixel of the second row must be stored in module 2 and the modules then continue to be accessed in sequence with a wraparound until the word address is incremented after module 1 has been accessed. The starting module for the first pixel of a row continues to be incremented in similar fashion until all 8 components have received the first pixel of a row. The process then recycles with component 0 receiving the first pixel of row 8.
When making vertical accesses to the frame buffer, the components are again accessed in sequence except that the row address is now incremented for each pixel. At the beginning of each new column, the row address is returned to 0 and the column address is incremented to 1. It will be observed that pixel 0,0 occurs at row 0 column 0, component 0; pixel 1,0 occurs at row 1, column 0, component 1 and pixel 2,0 occurs at row 2, column 0, component 2. This addressing arrangement thus meets the requirement that the components of the frame buffer can be accessed sequentially for both vertical and horizontal accessing.
An advantageous implementation of this addressing scheme is shown in FIG. 8 wherein the frame buffer 50 includes eight 32K.times.8 storage modules designated module 0-7. Each module has a corresponding data latch and an address latch. The least significant address bits 0-6 are provided by a 7 bit column counter 70 while the most significant 8 address bits 7-15 are presented by an 8 bit row counter 72. Row counter 72 is reset at each field start and incremented for each pixel in a vertical mode and at row start in a horizontal mode. The column counter 70 is reset to 0 at field start and at row start when in a horizontal mode and is incrmented in response to the maximum count output of a 3 bit counter 74. Counter 74 is coupled to be reset at field start and is clocked by the pixel clock signal. The count enable input to counter 74 is continuously enabled in a horizontal access mode and is enabled at column start for a vertical mode. Consequently, the column counter 70 is incremented for every eighth pixel clock in a horizontal mode and for every eighth column in a vertical mode.
Selection for the 32K.times.8 modules 0-7 is controlled by a 3 bit counter 80, a 3 bit counter 82, and a 3 to 8 module select decoder 84. Three bit counter 82 is incremented at the pixel clock rate to control the sequential accessing of the individual memory modules. The output of counter 82 is decoded by decoder 84 to select one of the eight modules in sequence for the simultaneous loading of the data latch and address latch for the selected module. Three bit counter 80 provides the required staggered module offset at row or column start. Counter 80 is reset at field start and is incremented in a horizontal mode at row start and in a vertical mode at column start. Three bit counter 82 is loaded at column start or row start with the contents of 3 bit counter 80 immediately prior to incrementing.
It should be noted that the addressing of the field buffers 50-52 is described in terms of vertical mode accessing and horizontal mode accessing. Under most circumstances these frame buffers provide a transposition by being accessed in a horizontal mode for writing and in a vertical mode for reading. However, under some circumstances the frame buffers may be accessed in a horizontal mode for both reading and writing. The failure to provide a transposition at frame store 3018 coupled with a transposition at frame store 3022 effectively imposes a 90.degree. rotation upon the video image. As an image is rotated toward 90.degree. the image effectively becomes mapped into a line of 0 width and resolution is lost. However, the resolution of the video image can be better preserved for large angle rotations by transposing the image at only one of the frame stores 3018 and 3022 and then imposing a negative rotation of between 0.degree. and 45.degree. to account for the difference between the desired rotation angle and the 90.degree. rotation imposed by failing to provide a transposition at frame store 3018.
Transposing frame store 3022 of FIG. 3 is implemented in a manner substantially identical to frame store 3018 except that frame store 3022 requires only two field buffers. A field of data is written vertically into one buffer while a previously written field is read horizontally out of the other buffer. The two buffers are then interchanged with the one buffer being read horizontally while the other buffer is written vertically.
The deinterlace filter 600 of FIG. 5 is illustrated in FIGS. 9A and 9B. Filter 600 includes a 2 byte wide three stage shift register 602, a filter component 604, and multiplexers 606, 608. The even and odd line data from the transposing frame buffers 50-52 is clocked at the pixel rate through the shift register 602 having stages R0-R5 which are numbered in scan sequential order for interlaced vertically scanned data from the transposing frame store 3018. Although the connections are not explicitly shown for simplicity, the purpose of the shift register 602 is to make the contents of each stage R0-R5 available to filter 604. Multiplexers 606, 608 respond to a vertical scan signal to select odd and even outputs respectively from filter
604 when data is being output from frame store 3018 in vertical scan order. When data is being output in horizontal scan order multiplexers 608 selects the output of register stage R2 to drive the even byte data stream while multiplexer 606 selects the output of register stage R3 to drive the odd data byte stream. In the event of horizontal accessing of the frame store 3018 a similar deinterlace filter subsequent to the vertical to horizontal transposing frame store provides deinterlace filtering.
The filter 604 contains substantially identical components for the even and odd data streams, each of which provide a -1/8, 2/8, 6/8, 2/8 and -1/8 filtering function. Each of the two odd and even components of filter 604 is advantageously implemented as shown in 7B with multiply by two functions 610, 611 multiply by four functions 612, four addition functions 614-617, one subtraction function 618 and a divide by eight function 620. It will be noticed that the multiply and divide functions are implemented as powers of two and that they can therefore be easily accomplished by merely shifting the relative positions of the data bit lines for incoming and outgoing data streams. The inputs to the even and odd data stream filters are indicated by the information shown in the even and odd columns of shift register 602. Each element in the table refers to a shift register stage within shift register 602 whose output is connected to a filter input as indicated.
Referring now to FIG. 10, the predecimator 700 includes five line buffers designated line buffer 0 through line buffer 4, each of which has a 256 word.times.32 bit storage capacity. Line buffers 0-3 each receive two 8 bit data streams from multiplexers 702-705 respectively. Each of the multiplexers 702-705 is capable of selecting one of four input signals and placing the selected input signals on one of the 8 bit buses to its corresponding line buffer. In some modes of operation the two
8 bit bus inputs to the line buffers are driven in parallel. The multiplexers 702-703 must thus be capable of either selecting two of the four input byte streams or one of the four input byte streams depending upon the mode of operation. Line buffer 4
receives two 8 bit data streams as even and odd outputs from a filter 708.
A 32 bit wide 5 to 1 multiplexer 710 provides a 32 bit output which is split into four 8 bit data streams and communicated to a 4 byte wide 3 stage shift register 712. Data is loaded into the line buffers and in such an order that it may be read out to fill the 12 bytes of shift register 712 with a serial sequence of pixel information for a scan line. That is, each register stage of shift register 712 stores 1 pixel of information and the pixel information is arranged in raster scan order as designated by the numbering of the registers R0-R11. Registers R8-R11 provide data output to the next stage of the transformation system as well as data output to the second stage of the shift register containing registers R4-R7. The purpose of shift register 712 is to make available to filter 708 12 bytes of sequential pixel information in a predetermined order. Although not explicitly shown for simplicity, the outputs of each of the registers R0-R11 are communicated to filter 708.
Filter 708 actually contains two separate filters operating in parallel. One of the filters generates even numbered pixel data at the pixel rate while the other generates odd numbered pixel data at the pixel rate. The even and odd outputs 716,
718 thus provide feedback data at twice the pixel rate in combination. A disable signal may be utilized to drive an output disable input to multiplexer 710 at the end of the processing for a scan line to cause zeros to be loaded into shift register 712. This loading of zeros creates an aesthetic blending by filter 708 at the end of a scan line and prevents information from the end of a scan line from affecting information at the beginning of the next scan line. Six extra clock signals are provided at the end of each scan line pass through the filter 708 before data is input through multiplexer 710 for the next scan line to clear the pipeline of the predecimator system, and particularly the shift register 712.
While the wide distribution of the four scan line signals stored by the line buffers 0-4 in order to accommodate different operating modes makes the predecimator 700 appear complex, its operation is actually quite straightforward. In the normal mode of operation vertical scan line information from corresponding vertical scan lines of a pair of sequential fields is received over the even and odd input lines and gated into line buffer 0. Because these even and odd input lines represent data from consecutive fields, they each carry alternate pixels for a frame. That is, for a given scan line column, the pixel information for rows 0 and 1 appear on the even and odd bus respectively followed by pixel information for rows 2 and 3 on the even and odd bus respectively followed by the pixel information for rows 4 and 5 on the even and odd bus and so forth. Multiplexer 702 connects the upper output stream 720 to the even input bus and simultaneously connects output stream 722 to the odd input bus. Gating at the input latches to line buffer 0 directs the first or row 0 pixel information to byte position 0 of the input data latch while the row 1 pixel information on bus 722 is gated to byte position 1 of the input data latch. At the next pixel clock period the pixel information for frame row 2 appearing on bus 720 is gated to the position 2 input data latch and the frame row 3 pixel information appearing on bus 722 is gated to the position 3 input data latch. The first four pixel bytes are thus stored in the input data latch in sequential scan order at the end of 2 pixel clock times with the data being written into address word location 0 and the input data buffer being reloaded with pixel information for row positions 4-7 during the third and fourth pixel clock times for storage at address word location 1. It is thus seen that a vertical scan column from a pair of sequential fields is deinterlaced and stored in line buffer 0 in raster scan order during a vertical line scan time period which will be designated scan time N to provide a frame of reference.
During this same vertical scan time and simultaneously with the writing of a frame scan line into buffer 0, previously written vertical scan line information is read from line buffer 2 four bytes at a time and output through 5 to 1 multiplexer
710 to the first stage of shift register 712 comprising registers R8-R11. Subsequent 4 byte words are read from line buffer 2 and shifted through shift register 712 at each pixel clock time. Since the data read out of line buffer 2 and shifted through shift register 712 contains a 4 byte parallel data stream, the effective bandwidth of this data transfer operation is four times the pixel rate. Filter 708 responds to the data content of the individual byte registers R0-R11 in shift register 712 to output 2 bytes of data designated even and odd on bus lines 716 and 718 at the pixel rate. Since the input scan line information to line buffer 0 and the even and odd output information from filter 708 each contain 2 bytes in parallel while the information being read from line buffer 2 contains 4 bytes in parallel, the line buffer output information has twice the effective bandwidth of the other two data streams. Line buffer 4 is gated to provide to its input data latch alternate bytes from the even and odd data streams from filter 708 in a manner similar to the gating of even and odd frame data into line buffer 0. Consequently, as 4 byte sequences of input pixel information are loaded into line buffer 0 four byte sequences of filtered information from filter 708 are loaded into line buffer 4. At the point in time during a scan line cycle where half of the pixels for the incoming vertical scan line have been loaded into line buffer 0, half of a scan line worth of pixel information from filter 708 will have been loaded into line buffer 4 since the bandwidth of the two data stream inputs to line buffer 0 and line buffer 4 are the same, i.e. twice the pixel rate. However, while line buffers 0 and 4 are being loaded at twice the pixel rate, line buffer 2 is being output at four times the pixel rate so that as half lines of pixel information are loaded into line buffers 0 and 4 a complete line of pixel information has been passed through shift register 712 and processed by filter
708. The half line of data stored in line buffer 4 thus represents a 2:1 compression ratio since the processing of a full line of information has resulted in the storage of a half line of information.
It will be noted that during the first half of the scan line period the full, uncompressed pixel information was transferred through shift register 712 and presented to down path circuitry for possible use thereby by the outputs of shift register stages R8-R11. Thus, even though a 2:1 data compression has taken place the original data may be stored and preserved for further use by the down path circuitry. During the next one-fourth of the scan line time period (time one-half to three-fourths) line buffer 0 continues to receive pixels of video input information in scan line order while line buffers 2 and 4 are interchanged. The 2:1 compressed data is read out of line buffer 4 at four times the pixel clock rate, passed through shift register
712 to filter 708 for compression processing and written into line buffer 2. As the 2:1 compressed data is read from line buffer 4 and passed through shift register 712 it also is made available for storage and later use by down path circuitry through the data outputs from registers R8-R11. At the end of threequarters of the vertical scan line period, a scan line of 4:1 compressed data has been loaded into line buffer 2. During the next one-eighth of a line scan period the 4:1 compressed data is read out of line buffer 2 and in response 8:1 compressed data is stored in line buffer 4. This process of sequentially further compression by two with alternate storage in line buffer 2 and line buffer 4 is continued to the end of vertical scan line time period at which a complete vertical frame line has been loaded into buffer 0 and the scan line being circulated through filter 708 has been compressed to a single pixel or byte.
This predecimating thus provides down path circuitry with a selection of scan line information which has been processed in a high quality filtering process and having compression ratios in powers of two. This predecimating performs much of the burden which would otherwise be incurred by the vertical transformation circuitry to provide an improved final fully transformed video image for a given data resolution of the data transformation system. For example, if a compression ratio of 17:1 is required, the transformation system may select from the predecimated data having a compaction ratio of 16:1 and provide only a very small additional compaction required to increase the ratio to 17:1.
At the end of vertical scan line time period N a new vertical scan line time period N+1 begins with multiplexer 704 gating a next vertical scan line pair of even and odd field data into line buffer 2 in sequential order just as the previous scan line had been written into line buffer 0. At the same time, a flip-flopping data exchange begins between line buffer 0 and line buffer 4 with the scan line data being predecimated to provide sequential compressions by factors of two as the scan line data is recirculated through shift register 712 and filter 708 as previously done for the data stored in line buffer 2 during vertical scan line time N. For the next vertical line time period N+2 the cycle is repeated with the incoming scan line pixel data stream being loaded into line buffer 0 while the contents of line buffer 2 are predecimated.
In the mode of operation wherein data is received on the even and odd input buses in horizontal rather than vertical scan line order the data buffering process must be slightly different because each even and odd input carries a complete sequence of pixel row information by itself rather than information for alternate pixel locations as was the case for the interlaced vertical scan line information. Complete even row data is on the even line while complete odd row data is on the odd line. For this mode of operation multiplexers 702 and 703 operate to select and gate the even and odd incoming horizontal data streams to line buffer 0 and line buffer 1 respectively. Multiplexer 702 causes the even line incoming data stream to be alternately gated onto upper bus 720 and lower bus 722 to permit the loading of the incoming pixels into the 4 byte input data buffer for line buffer 0 in sequential scan order. Similarly, multiplexer 703 operates to alternately gate the incoming odd horizontal scan line information onto upper bus line 724 and lower bus line 726 to permit the loading of the odd horizontal scan order. While each of the line buffers 0 and 1 are now loaded at the pixel rate instead of twice the pixel rate for the vertical scan mode of operation, the total incoming data rate remains at twice the pixel rate since two line buffers are used in parallel instead of one. As the horizontal scan time interval continues previously loaded data is read out of line buffer 2 at four times the pixel rate and passed through shift register 712 and filter 708 to be stored by line buffer 4 with a 2:1 compaction ratio. Since the data is read out of line buffer 2 at four times the rate that data is being written into each of the line buffers 0
and 1, a full scan line of data will have been read out of line buffer 2 and passed through filter 708 by the time one-fourth of a line of data has been stored in each of the line buffers 0 and 1. The original contents of line buffer 2 will have been fully predecimated by the time line buffers 0 and 1 are each loaded with one-half of a line of information. During the second half of the horizontal line time interval the previously written contents of line buffer 3 are predecimated,. During the next horizontal scan line time period interval horizontal scan line information is written sequentially into line buffer 2 and line buffer 3 for even and odd scan line row information respectively while the previously stored contents of the line buffer 0 are predecimated during a first half of the scan line time period interval and the previously stored contents of line buffer 1 are predecimated during the second half of the scan line time period interval. It is thus apparent that the predecimation process is substantially the same for both vertical and horizontal scanning although the buffering of the incoming data must be somewhat different to account for the differences in the interlaced and non-interlaced incoming video data streams.
Filter 708 contains two parallel filters providing a -1/16, 0, 5/16, 1/2, 5/16, 0, -1/16 filtering function and are identical except for their input connections within shift register 712.
A highly advantageous implementation for filter 708 is illustrated in FIG. 11 to which reference is now made. While only one filter 708 is shown, it will be appreciated that duplicate even and odd filters are employed with their inputs connected to the respective even and odd registers indicated by the table at the inputs to the filter. It will be known that the filter is very conveniently implemented with four adders 730-733 and a single subtractor 734. No actual multiplication or addition is required since the multiply blocks 736 and 737 and the divide block 738 are implemented in powers of two to permit the operations to be accomplished by merely shifting the relative bit positions of the incoming and outgoing data information. Because of the elimination of actual multiply and divide operations, the filter 708 can be implemented at far less expense than conventional seven point filters and can operate at the 70 nanosecond pixel clock rate.
Interpolation decimation filter 800 and 906 are essentially the same and are representatively illustrated by interpolation decimation filter 800 as shown in FIG. 12 to which reference is now made. Filter 800 provides the ultimate functional relationship between the source or input video data and the target data in the vertical dimension.
Vertical source address generator 912 (FIG. 5) calculates and supplies to interpolation decimation filter 800 a sequence of vertical pixel source addresses corresponding to a sequence of output target video data in response to the vertical target address counter 914 and transform composer and factorizer 916. The addresses supplied by vertical source address generator 912 have a resolution of 1/64 pixel and includes a 4 bit magnification factor parameter of between 0 (for a 1/1.99 sized image or larger) and 15 (for predecimated data compressed by 2.sup.15 or more). Interpolation decimation filter 800 supplies a video data value calculated from four pixel locations appearing on each side of the source address. Sixteen filter functions are available for calculating the output video data value. One is selected in response to a four bit parameter alpha in accordance with the desired compaction ratio provided by interpolation decimation filter 800 in addition to a selected predecimation compaction.
A two line double buffer 809 is implemented in 8 segments 801-808 and receives video data 4 bytes parallel from the R8-R11 data outputs of predecimator 700 (FIGS. 5 and 10). For each vertical scan line of a frame the received data includes a full line of video data plus all of the predecimated copies of the full line which copies occupy a second full line of data. Hence, there is a need for storing two lines of data in each half of the double buffer 809. The double buffering permits a new two lines of video data to be received while the immediately preceding two lines of data are operated upon to provide one line of target image video data.
As video data is received by double buffer 809 the first four bytes are stored respectively in the four segments 801-804, the second four bytes are stored respectively in the four segments 805-808, the third four bytes are stored respectively in the four segments 801-804 and so forth. The eight part segmentation of the double input buffer 801-808 thus assures that the pixel data for the four adjacent pixel locations on each side of an address point (8 total) can be read in parallel from the double input buffer.
A barrel shifter 810 receives the 8 bytes of pixel data from double input buffer 809, circulates the data to a desired position in response to the three least significant bits of the nonfractional portion of the source address and presents the circulated video data to an eight segment multiplier 820 having segments 821-828. The data is circulated such that the pixel data corresponding to the nonfractional portion of the source address is presented to a central mutiplier segment 824. The pixel data for the three pixels sequentially to the left thereof are presented to segments 823, 822 and 821 while the pixel data for the four pixels to the right are presented to segments 825-828 respectively. The eight multiplier segments 821-828 thus receive as first inputs 8 bits of video data for each of 8 sequential pixel locations centered about the source address point.
Multiplier segments 821-828 each receive as a second input an eight bit coefficient or weighting function from an 8 segment coefficient memory 830 having segments 831-838. Each segment is configured as 1024 words of eight bits each. The coefficient memory 830 receives as a partial address the six bit fractional part of the source pixel address. These six bits provide a phase factor .phi. which defines the one of 64 subpixel points for the 1/64 pixel resolution of the source address. A filter function may thus be centered about the subpixel source address with the pixel data being weighted in accordance with its position on a filter function curve relative to the subpixel address.
The coefficient memory 830 further receives four bits of address in accordance with the parameter alpha which is related to the magnification produced by interpolation decimation filter 800. Coefficient memory 830 may thus contain 16 different filter functions for each of the 64 subpixel source addresses. The filter function may thus be tailored to the degree of magnification (compaction) provided by interpolation decimation filter 800. For example, if the output target image is to be at least as large as the selected original or predecimated copy of the source image, it may be desirable to use a filter function which heavily weights video data for pixel locations very close to the source address. On the other hand, if compaction approaching 1/2 is desired, a filter function giving at least some weight to all eight pixel locations near the source address may be desirable. It will be recalled that the predecimator 700 provides compaction by all practical powers of 1/2 so that the further compaction provided by interpolation decimation filter 800 can always be by a magnification factor greater than 1/2.
An addressing circuit 840 is illustratively represented by a segment 841, which is one of eight segments providing address inputs to the eight double buffer memory segments 801-808 respectively. Address segment 841 includes an adder 851, a magnification factor ROM 861 and a carry ROM 871. Adder 851 receives as a first input the nonfractional part of the source address divided by eight. Division by eight is of course accomplished by merely shifting off the three least significant bits of the integer portion of the source address. The four bit magfactor parameter is presented as an address input to ROM 861 which generates an address shift in accordance with the magnification factor. If the target image is to be larger than half the size of the source image magfactor is zero and the full size copy of the source image is output to barrel shifter 810. For a target image compressed to between 1/4 and 1/2 the size of the source image the ROM 861 translates the source address to the half size predecimated copy of the source image and so forth.
Carry ROM 871 receives the three least significant bits of the integer part of the source address and selectively provides a carry output to increment the translated buffer memory 809 word address when the three least significant bits designate a number between 4 and 7 inclusive. This selective incrementing accommodates situations where the desired eight pixels cross a word boundary for buffer memory 809. It will be noted that the addresses for segments 806-808 must be selectively decremented rather than incremented.
As an example, assume that the source address is 25 5/64 (binary 00011001.000101) for a full size target image. The divide by 8 pixel address input to adder 851 thus becomes 3 (Binary 00011). Magfactor=0 will designate a full size image and the output of ROM 861 to adder 851 will be zero. For the given address it is desired to read from buffer memory 809 video data for pixel locations 22-29. The data for pixels 24-29 is stored at word location 3, buffer segments 801-806, while data for pixel locations 22 and 23 is stored at word location 2 in buffer segments 807 and 808 respectively. C-ROM 871 thus outputs a zero in response to a 1 (binary 001) input and buffer address word 3 is presented to segment 801 by adder 851. Similarly segments
802-806 will receive address word 3 from their respective address circuits 840 and segments 807 and 808 will receive a decremented address word 2.
Data for the pixel defined by the integral portion of the source address (pixel 25) is output from segment 802 and is circulated downward (as shown) two places by barrel shifter 810 in response to the three least significant address bits (001) so that data for the designated source pixel is presented to multiplier segment 824. Coefficient memory 830 can thus be programmed with the assumption that the video data for the eight pixels about a source point will always be presented in ascending order to multiplier segments 821-828. The same effect could be accomplished by eliminating barrel shifter 810 and adding three address inputs to coefficient memory 830 with each of the segments receiving additional programming to accommodate the eight possible locations where the data for the designated source pixel might occur.
The multipliers 821-828 of arithmetic network 812 thus receive the 8 pixels of video data from barrel shifter 810, multiply the pixels by their appropriate coefficient factor from coefficient memory 830 segments 831-838 and output the results to a summing network 881-887 which sums the eight products to generate the pixel of video data corresponding to the input source address. The resulting stream of pixel data from interpolation decimation filter 800 is fully processed in the vertical dimension and is then presented to the vertical to horizontal transposing memory 900 for the initiation of processing in the horizontal dimension separately from the processing in the vertical dimension.
Referring now to FIG. 5, interpolation decimation filter 800 receives vertical lines of the source image. Even if the video data is read out horizontally from H to V transposing memory 3018, the video data is still treated as a vertical scan. The net effect is a 90.degree. rotation and mirror imaging which is compensated by transform composer and factorizor 916.
It will be recalled that x and y are used to identify pixel locations within the target or output image while u and v are used to identify pixel locations within the source image. At interpolation decimation filter 800 each vertical scan line corresponds to a constant u value with the u value being incremented for each sequential vertical scan line starting with zero and moving from left to right. For each vertical scan line interpolation decimation filter 800 receives a sequence of v address inputs from vertical source address generator 912 specifying a sequence of pixel addresses within a scan line for a sequence of video data pixels. Interpolation decimation filter 800 responds to each v address received by outputing a pixel of video data as a function of the pixels positioned about the v point in the vertical scan line.
Equation 31 defines v as a function of u and y and a number of constants from an "a" matrix (Table I) which defines the desired relationship between the target and source images. During each vertical retrace time between fields the transform composer and factorizor 916 calculates the required matrix constants in response to operator input commands and supplies them to vertical address generator 912. Vertical address generator 912 itself generates the u and y terms by in effect starting at zero for the first pixel of the first scan line and incrementing y for each successive pixel and incrementing u for each successive vertical scan line.
Similarly, for the horizontal dimension horizontal address generator 908 receives the appropriate "a" matrix constants from transform composer and factorizer 916 and calculates the horizontal source addresses u for each horizontal scan line in accordance with equation 31 as a function of x and y. X and y are, in effect, established by starting at 0,0 for the first pixel of each field and incrementing x for each pixel and incrementing y for each horizontal scan line.
While the vertical and horizontal addresses v and u could of course be generated from the equations therefor by microprocessors, it would be very difficult to accomplish this at the 70 n sec pixel rate. The vertical source address generator 912
and horizontal source address generator 908 are special purpose circuits for calculating the v and u equations 30, 31 at the pixel rate. It is of interest to note that video data enters interpolation decimation filter 800 at twice the pixel rate because of de-interlacing but passes through the remainder of the system at the pixel rate. The vertical and horizontal source addresses need therefore be generated at only the pixel rate and not twice the pixel rate.
Referring now to FIG. 13, the vertical source address generator 912 includes a numerator calculation circuit 915, a denominator calculation circuit 917, a divider circuit 918 to divide the numerator by the denominator and a timing and control circuit 920 for generating the various timing and control signals used throughout the vertical source address generator 912.
A previous v register 924 receives and temporarily stores each vertical address v. A subtractor 926 subtracts the stored previous v address from the current v address to generate an 18 bit difference parameter on signal path 928. The most significant bit of the 17 bit difference parameter is a sign bit while the six least significant bits represent a fractional part. The difference parameter is used as an estimate of the derivative of v with respect to time from which terms Magfactor and Alpha are derived.
A magfactor ROM 930 receives the integer portion of the difference parameter and outputs the term magfactor as the integer part of the log base 2 of the absolute value of the difference parameter. Magfactor equals 0 for difference parameters of
0-1.99, 1 for difference parameters 2.00-3.99, 2 for difference parameters of 4.00-15.99 and so forth. Only absolute values are considered. Magfactor commands interpolation decimation filter 800 to use a particular predecimated copy and is communicated to a barrel shifter 934 which shifts (divides) the vertical source address by a number of bit positions equal to magfactor to produce an adjusted source address. When a predecimated copy of a line of data is selected having compaction by a given power of 2, the source address must be divided by the same power of 2 for compatibility and barrel shifter 934 performs this function.
The difference parameter is the reciprocal of the magnification of the target image relative to the source image. For example a double size target image will produce difference parameters of 0.5 while a half size target image will produce difference parameters of 2.0 and so forth. The difference parameter is thus a measure of the magnification (including compaction) of the target image relative to the source image. Barrel shifter 136 receives the difference parameter and shifts it toward less significant bit positions by a number of bit positions indicated by the parameter magfactor to generate an interpolator difference signal in signal path 938 which represents the magnification (compaction) which must be performed by interpolation decimation filter 800 over and above that performed by a predecimated copy selected by the parameter magfactor.
A parameter interpolator difference is used as an address input to alpha ROM 938 which responds by generating a 4 bit parameter, alpha, which selects one of 16 filter functions for use by interpolation decimation filter 800. To improve target image quality it is desirable to use different filter functions for different degrees of magnification (compaction) of the target image by the interpolation decimation filter. Filtering of the predecimated copies is handled by predecimation filter 700
so that only the additional filtering by interpolation decimation filter 800 is of interest at this point.
For example, if the target image is to be full size or larger, a high peak, narrow filter function should be used which places great weight on the source pixels nearest to the vertical source address point. As the target image is compacted by greater and greater amounts, the filter function should become flatter and broader, thus putting less weight on pixels immediately adjacent the source address point and more weight on pixels farther from the source address point.
Interpolation decimation filter 800 provides all degrees of image enlargement but a maximum compaction by a factor of 1.99. Any additional compaction would be accomplished by selecting a smaller predecimated copy. For example, compaction of the target image by a factor of 16 would be accomplished by selecting the fourth predecimated copy (magfactor equal 16) and by introducing a compaction factor of 1 in the interpolation decimation filter 800 (no further compaction). The term magfactor would be 4, the difference parameter would be 16 and the interpolator different parameter would be 1. For compaction of the original image by 32 the fifth predecimated copy would be selected, magfactor would be 5, the difference parameter would be 32 and the interpolator difference parameter would be 1. For compaction of the original image by the factor of 15.4, the third predecimated copy would be selected, magfactor would be 4, the difference parameter would be 15.4 (binary 1111.011001), and the interpolator difference parameter would be 1.92 (binary 1.111011).
The integer part of the interpolator difference parameter on signal path 938 has a maximum value of 1 and its fractional part has 6 bits of accuracy. The interpolation difference parameter thus has 7 bits and alpha ROM 932 can have a size of 128
by 4. Since a single filter function is adequate for all degrees of image enlargement, full size and slight compaction, it is desirable to divide the range of the interpolation difference parameter between 1.00 and 1.99 into 16 equal parts along a logarithmic scale with each part being assigned a different alpha parameter and a corresponding filter function.
The alpha ROM 932 is thus loaded to output 0 for addresses 1-1.04 (binary 1.000011), 1 for input addresses 1.05-1.09 (Binary 1,000100 to 1,000110); 2 for input addresses 1.10 to 1.14 (binary 1.000111 to 1.011001) and so forth up to 15 for input addresses 1.91 to 1.99 (binary 1.111010 to 1.111111). A different filter function can thus be provided for each of the 16 values of alpha ranging from narrow and steep for alpha equal 0 to broad and flat for alpha equal 15. The same filter function is thus used for full size images, enlarged images and the largest sized group of compacted images.
The vertical source address generator 912 includes a numerator circuit 914, a denominator circuit 916, and a divider and denormalization circuit 918 which divides the output of the numerator circuit 914 by the output of the denominator circuit
916 and then denormalizes the quotient before outputing the vertical address, V, to barrel shifter 934. A timing and control circuit 920 responds to commands received from vertical target address counters 914 on signal paths 940 which indicate the end of a frame interval as well as information from transform composer and factorizor 916 received on a communication bus 942 to generate the various timing and control signals used throughout the vertical source address generator 912. It will be appreciated that the actual circuitry of the source address generator has been represented in a simplified form for clarity of explanation. For example, multiplexers 944, 946 and 948 can be implemented through selective gating of tristate logic circuits rather than as separate integrated circuits called multiplexers and data can be sequentially loaded one byte at a time into 32 bits (94 byte) data registers 950, 951, 952, 953, 954, and 955 where the communication bus 942 includes for example an 8 bit data bus from an 8 bit microprocessor. Data register 956 is also a 32 bit register while bias data register 957 may be implemented as an 8 bit register.
It will be recalled that vertical source address generator 3012 solves equation 30 to generate vertical source addresses at the pixel rate. During each vertical retrace time interval transform composer and factorizor 916 loads the constants for equation 30 into corresponding registers of vertical source address generator 912 over communication bus 942. For example, the numerator constants a.sub.31, a.sub.34, a.sub.21, and a.sub.24 are loaded respectively into 32 bit registers 950, 951, 952 and
953. Timing and control circuit 920 renders the select A inputs of multiplexers 944 and 946 at logic zero during this interval to permit the data to be communicated to the inputs of registers 951 and 953. Thereafter the select A inputs are set to logic
1 so that register 951 receives data from a 32 bit adder 960 through A input of multiplexer 944 while register 953 receives data from a 32 bit adder 962 through the A input of multiplexer 946.
Similarly, during the vertical retrace vertical constant a.sub.32 is loaded into 32 bit register 954 and constant -a.sub.22 is loaded into 32 bit register 955 through the B input to multiplexer 948. Thereafter, the select A input to multiplexer
948 is set to logic 1 so that data may be communicated from 32 bit adder 964 through the A input of multiplexer 948 to the input of data register 955.
It will be appreciated that adder 960 receives inputs from the output of a.sub.31 register 950 as well as register 951 to present the sum of these inputs to the main input of multiplexer 944. Similarly, adder 952 adds the output of a.sub.21
register 952 to the output of register 953. Numerator circuit 914 further includes an adder 966 which adds the output of register 951 to the output of register 956 and presents the sum back to the input of register 956. A subtractor circuit 968
subtracts the output of register 956 from the output of register 953 to generate a difference signal which is the solution to the numerator portion of equation 30 and which is presented to divider circuit 918. Adder circuit 964 adds the output of a.sub.32 register 954 to the output of register 955. The output of register 915 becomes the solution to the denominator portion of equation 30 and is also presented to divider circuit 918.
As a frame of reference pixel time intervals will be defined as a function of u and y corresponding to the vertical addresses, v(u,y), such that at the output of numerator circuit 915 and the output of denominator circuit 917 data for a given pixel address shall be valid at the occurrence of a corresponding pixel clock transition. For example, at pixel clock time t.sub.0,0, data shall be valid for the pixel corresponding to vertical source address v.sub.0,0 and at pixel clock time 2,2 data shall be valid for pixel source address v.sub.2,2 and so forth. It will be appreciated that at the vertical source address generator 912 vertical addresses are measured in terms of the target image pixel locations y while horizontal addresses are measured in terms of the first image pixel locations u.
During the vertical retrace interval register 956 is cleared while constants are loaded into the other registers. Looking at equation 30, it will be observed that for the first pixel clock time t.sub.0,0, the variables u and y will both be 0 so that the solution to v is a.sub.24 divided by -a.sub.22. Since register 953 has been preloaded with constant a.sub.24 while register 956 has been cleared during the vertical retrace interval, at time t.sub.0,0 subtractor 968 generates the appropriate numerator term a.sub.24 as the output of numerator circuit 914. Similarly, register 955 has been preloaded with the constant -a.sub.22 and outputs this term as the proper denominator term for equation 30.
Clock signal CK3 loads register 956 with the output of adder 966 at each pixel clock time. Thus, at pixel clock time t.sub.0,0 register 956 is loaded with the sum of 0+(1) (a.sub.34). The output of register 956 thus represents the proper value of u=0 and y=1 for the first portion of the numerator of equation 30 at the second clock time t.sub.0,1. Clock signal CK3 is active at this time and at each additional pixel clock time so that the constant a.sub.34 stored in register 951 is added to the contents of register 956 at each pixel clock time. Since y is incremented at each pixel clock time, the next result is multiplication of a.sub.34 by y by means for successive additions. That is, the output of the first portion of the numerator of equation 30 for y=0, 1, 2, 3, 4 etc. is generated at the output of register 956 by adding to register 956 the value a.sub.31 u=a.sub.34 0 times, 1 times, 2 times, 3 times, 4 times, and so forth respectively. In a similar manner, the register 951 is clocked with clock signal CK2 during an interval between successive line scans so that register 951 stores the constant a.sub.34 during the first line scan for line 0, the value a.sub.34 +(1) (a.sub.31) during the second line scan for vertical line 1, a.sub.34 +(2) (a.sub.31) for the third line, vertical line 2, and so forth. The output of register 951 thus continually represents the term a.sub.31 u+a.sub.34. This value is added to the contents of the register 956 at each pixel clock time so that the effect is the same as multiplying the output of register 951 by y as y is incrementally stepped through successive pixel locations within a vertical line scan. Between each successive line scan the register 956 must be cleared or reset to reflect the new vertical line scan starting position of y=0.
This general concept of repeatedly adding or accumulating a term at the pixel clock rate to accomplish multiplication by y and repeatedly adding a term at a line clock rate to accomplish multiplication by u is used throughout the vertical source address generator line 12. In the horizontal source address generator 908 a similar technique is used with successive additions at the pixel clock rate being utilized to accomplish multiplication by x and successive additions at the horizontal line clock rate being utilized to accomplish multiplication by y.
It will be observed that the second term of the numerator is generated at the output of register 953 which is initially loaded with a constant a.sub.24 before the beginning of each field time and then clocked with signal CK5 at the vertical line clock rate between each successive vertical line scan so that the output of register 953 represents a.sub.24 +a.sub.21 u. Similarly, the clock signal input to register 955, CK7, is activated to initially load the constant -a.sub.22 to register 955 and then to add the term a.sub.32 to the contents of register 955 at the pixel clock rate so that the output of register 955 represents the value a.sub.32y -a.sub.22. This is the demoninator of equation 30. A barrel shifter 970 receives the successive 32
bit words of video data for successive pixel addresses and operates in conjunction with an exponent detector 972 to convert the numerator into a floating point form with the output of barrel shifter 970 providing 16 bits of data representing the mantissa of the numerator and exponent detector 972 outputing 8 bits representing the exponent of the numerator term. Conversion to the floating point representation eliminates the need to carry leading 0's and permits the 16 bit output of barrel shifter 970 to carry the most significant 16 bits of actual numerical data. In a similar manner the barrel shifter 974 and exponent detector 976 convert the denominator term to a floating point representation. A reciprocal circuit 978 receives the 16 bit mantissa of the denominator term and outputs the reciprocal thereof. One suggested approach for accomplishing this reciprocation at the 70 NSEC pixel clock rate is to utilize the most significant 8 bits of the denominator term to address a conversion table storing reciprocal values and to utilize the least significant 8 bits of the denominator term to generate a linear interpolation between adjacent values in the reciprocal table. The reciprocated mantissa of the denominator is multiplied by the mantissa of the numerator in a hardware multiplier 980 and the product is presented to a barrel shifter 982. A subtractor 984 subtracts the exponent of the denominator from the exponent of the numerator to accomplish the division function and an adder 986 adds the difference to a bias term which is stored in bias register 957 before the beginning of each field scan time. The A const