REGISTER  |  SIGN IN
United States Patent5574843
Gerlach, Jr.November 12, 1996
Title
Methods and apparatus providing for a presentation system for multimedia applications
Abstract
A data processing system has a central processing unit, a memory for storing a plurality of data structures corresponding to a presentantion, and a display device. The display device has a display screen including a presentation area for displaying a presentation. Each one of the data structures includes an action identifier and a plurality of attributes. The process provides for receiving a one of the plurality of data structures of the presentation, analyzing the received data structure to determine an action to be performed in response to the action identifier of the data structure, and performing the action corresponding to the action identifier in accordance with a plurality of the attributes of the received data structure.
Inventors:Gerlach, Jr.; John D. (Falls Church, VA)
Assignee:Escom AG (Heppenheim, DE)
Appl. No.:384735
Filed:January 17, 1995
Current U.S. Class:345/418 715/835 
Field of Search:395/140,141,154,155,159,161
U.S. Patent Documents
4315315February 1982Kossiakoff
4449180May 1984Ohshima et al.
4455619June 1984Masui et al.
4536840August 1985Borta
4546435October 1985Herbert et al.
4569019February 1986DiOrio et al.
4644423February 1987Buntsis et al.
4656603April 1987Dunn
4681548July 1987Lemelson
4689022August 1987Peers et al.
4723210February 1988Barker et al.
4734764March 1988Pocock et al.
4736320April 1988Bristol
4739477April 1988Barker et al.
4779080October 1988Coughlin et al.
4813013March 1989Dunn
4821211April 1989Torres
4821220April 1989Duisberg
4827404May 1989Barstow et al.
4860204August 1989Gendron et al.
4872167October 1989Maezawa et al.
4885717December 1989Beck et al.
4893256January 1990Rutherfoord et al.
4899136February 1990Beard et al.
4905163February 1990Garber et al.
4931950June 1990Isle et al.
4953080August 1990Dysart et al.
5010500April 1991Makkuni et al.
5072412December 1991Henderson, Jr. et al.
5084813January 1992Ono
5204947April 1993Bernstein et al.
5208665May 1993McCalley et al.
5208745May 1993Quentin et al.
5214756May 1993Franklin et al.
5220657June 1993Bly et al.
5317732May 1994Gerlach, Jr. et al.
Foreign Patent Documents
WO88/07719Oct., 1988WO
Other References
H G. Okuno et al., "Firmware Approach To Fast Lisp Interpreter," ACM Journal, 1987, pp. 1-11. .
H. G. Okuno et al., "TAO: A Fast Interpreter-Centered System on Lisp Machine ELIS," ACM Journal, 1984, pp. 140-149. .
R. P. Ten Dyke et al., "Object-Oriented Programming," IBM Systems Journal, vol. 28, No. 3, 1989, pp. 465-478. .
Dr. Dobbs Journal, Dec. 1989 (4 Advertisments). .
IBM `Using Disk Operating System Version 4.00`, First Edition (Jul. 1988) pp. 29-31; Backup. .
Miastkowski, S., "Windows Shopping, Pricey and Elegant Multimedia Development", BYTE, vol. 15, No. 8, Aug. 1990, pp. 114-115. .
Hirakawa, M. et al., IEEE Transactions On Software Engineering, "An Iconic Programming System," vol. 16, No. 10, Oct. 1990, New York, NY, pp. 1178-1184. .
Tsuda et al., "IconicBrowser: An Iconic Retrieval System for Object-Oriented Databases," IEEE, pp. 130-137, Apr. 1989. .
Marcus, B., "Joyce: An Object-Oriented Decision Tree Builder.". .
Karjalainen et al., "Block Diagram Compilation and Graphical Editing of DSP Algorithms in the QuickSig System," IEEE, pp. 1057-1060, Aug. 1988. .
Coote et al., "Graphical and Iconic Programming Languages for Distributed Process Control: An Object Oriented Approach," IEEE, pp. 183-190, May 1988. .
Sugimura et al., "TAO/ELIS: An AI Software Development Environment Based on a Multiple Programming Paradigm Language," Review of the Electrical Communications Laboratories, vol. 37, No. 1, pp. 77-84, 1989. .
Bentz, B., "An Automatic Programming System for Signal Processing Applications," Pattern Recognition, vol. 18, No. 6, pp. 491-495, 1985. .
Shu, Nan C., "Visual Programming," Van Nostrand Reinhold, New York, 1988..~
Primary Examiner: Zimmerman; Mark K.
Attorney, Agent or Firm:Ratner & Prestia
Parent Case Text


This application is a continuation of application Ser. No. 07/691,984 filed Apr. 26, 1991, now abandoned.
Claims

I claim:
1. In a data processing system including a central processing unit, a memory for storing a plurality of presentations which each include operations which are represented by parent icons and child icons, each of said child icons associated with a respective one of said parent icons, and a display device having a display screen including a presentation area for displaying said parent icons and child icons representing said operations performed in each of said plurality of presentations, a method comprising the steps performed in the data processing system of:
providing, for each of said plurality of presentations, a plurality of data structures including a plurality of event structures and a plurality of command structures, each of said plurality of event structures and each of said plurality of command structures including a) a respective parent list, and b) a respective specific data member which designates respective data for a respective one of said operations, each of said plurality of event structures further including c) a respective child list;
providing a plurality of link structures, each associated with respective ones of said plurality of data structures, each of said plurality link is structures including respective link nodes each a) to couple a respective one of said plurality of event structures with a respective one of said plurality of command structures and b) associated with at least a respective one of said plurality of data structures based on contents of said respective parent list and child list for coupling to at least one of said parent icons and child icons, respectively;
receiving one of the plurality of event structures of one of said plurality of presentations:
locating, using one of said plurality of link structures associated with said received one of said plurality of event structures based on one of said respective parent list and said respective child list for said received one of said plurality of event structures, one of said plurality of command structures of the one of said plurality of presentations;
determining an action to be performed which corresponds to an action identifier which has been stored in said located one of said plurality of command structures; and
performing the action corresponding to the action identifier in accordance with a plurality of attributes which have been stored in said located one of said plurality of command structures.

2. The method of claim 1, further including the steps of:
determining a further action to be performed in response to a further action identifier which has been stored in said located one of said plurality of event structures; and
performing the further action in accordance with a further plurality of attributes stored in said located one of said plurality of event structures.

3. A method of claim 1 wherein the data processing system further includes a multimedia device and wherein the step of performing the action which corresponds to the action identifier includes the step of:
signaling the multimedia device to perform the action which corresponds to the action identifier.

4. The method of claim 1, wherein one of said plurality of link structures is for coupling another one of said plurality of link structures to one of said plurality of command structures for conditional execution of said action.

5. A presentation system in a data processing system including a central processing unit, a memory for storing a plurality of presentations which includes operations which are represented by parent icons and child icons, each of said child icons associated with a respective one of said parent icons, and a display device having a display screen including a presentation area for displaying said parent icons and child icons representing said operations performed in each of said plurality of presentations, the presentation system comprising:
a plurality of data structures corresponding to each of said plurality of presentations including a plurality of event structures and a plurality of command structures, each of said plurality of event structures and each of said plurality of command structures including a) a respective parent list, and b) a respective specific data member which designates respective data for a respective one of said operations, each of said plurality of event structures further including c) a respective child list;
a plurality of link structures each associated with respective ones of said plurality of data structures, each of said plurality of link structures including respective link nodes each a) to couple a respective one of said plurality of event structures with a respective one of said plurality of command structures and b) associates with at least a respective one of said plurality of data structures based on contents of said respective parent list and child list for coupling to at least one of said parent icons and child icons, respectively;
means for receiving one the plurality of event structures of one of said plurality of presentations;
means for locating, using one of the plurality of link structures associated with the received one of the plurality of event structures based on one of said respective parent list and said respective child list for said received one of said plurality of event structures, one of the plurality of command structures of the one of said plurality presentations:
means for determining an action to be performed which corresponds to an action identifier which has been stored in said located one of said plurality of command structures; and
means for performing the action corresponding to the action identifier in accordance with a plurality of attributes which have been stored in said located one of said plurality command structures.

6. The presentation system of claim 5 further comprising:
means for determining a further action to be performed in response to a further action identifier stored in said located one of said plurality of event structures; and
means for performing the further action in accordance with a further plurality of attributes stored in said located one of said plurality of event structures.

7. The presentation system of claim 5 wherein the data processing system further includes a multimedia device and wherein the means for performing the action corresponding to the action identifier comprises:
means for signaling the multimedia device to perform the action represented by the action identifier.

8. A presentation system according to claim 5, wherein one of said plurality of link structures is for coupling another one of said plurality of link structures to one of said plurality of command structures for conditional execution of said action.
Description
RELATED APPLICATIONS

This application is related to U.S. Ser. No. 07/691,865 entitled "METHODS AND APPARATUS PROVIDING FOR A MULTIMEDIA AUTHORING AND PRESENTATION SYSTEM," U.S. Ser. No. 07/691,965 entitled "METHODS AND APPARATUS PROVIDING FOR AN ICONIC PROGRAMMING AND DATABASE SYSTEM FOR MULTIMEDIA APPLICATIONS," and U.S. Ser. No. 07/692,230 entitled "METHODS AND APPARATUS PROVIDING FOR A PRESENTATION AND RESOURCE RELOCATION SYSTEM," all filed the same day as this application.

FIELD OF THE INVENTION

This invention relates to computer authoring systems and, more particularly, to a computer system for creating and presenting interactive multimedia presentations and coursework. The invention facilitates the creation and presentation of interactive multimedia presentations and coursework using a graphic interface display. This invention also relates to visual (or iconic) programming systems and, more particularly, to a visual programming system for creating software applications.

BACKGROUND OF THE INVENTION

Interactive multimedia presentations and coursework have become an important and effective method of presenting information and teaching. Additionally, the ability to program computers has also become an important skill which can take years to develop and master. Therefore, conventional computer systems have been developed which address each of these items. However, no known conventional computer system addresses both of these items.

A. Creation of Interactive Multimedia Presentations

When people communicate with each other, they use many different methods to creatively convey information. Among these methods of communication are: sound/music, pictures, words, numbers, animated sequences, and full motion video. The use of these methods in a presentation is typically referred to as the multimedia approach to communication.

Historically, multimedia presentations have been encumbered by the use of multiple technologies, such as slide projectors, videotapes, and computer graphics. But today, powerful computers offer a single delivery system or platform for integrated multimedia presentations. Thus, the speaker or teacher only needs to handle a single piece of equipment. The difficulty then remains in how the speaker or instructor is going to create and present multimedia presentations using these powerful computers.

In addition to creating an environment for multimedia presentations, the computer has also made it possible to create interactive presentations which means that the viewer can actually participate in a presentation by communicating with the computer. This has given rise to a class of computer software applications called courseware which is a powerful teaching and training tool. Again, the difficulty remains as to how the instructor is to create and present these interactive multimedia presentations.

One conventional computer system provides a method for specifying and executing independent, multimedia tasks along a synchronized time-line in the form of a matrix with the event elements making up the rows and the time periods making up the columns. Although this conventional system addresses the issue concerning the time consuming task of creating the presentation, this system fails to provide important interactive capabilities. Furthermore, this conventional system employs a time-line for the control of events in a presentation. Using a time-line requires the operator using such a conventional system to duplicate events so that the events can be executed in more than a single time period. This requires additional computer resources which is not desirable.

B. Visual Programming Systems and the Method of Visually Programming an Interactive Multimedia Presentation

In general, programming may be defined as specifying a method for doing something the computer can do in terms the computer can interpret or understand. There are many aspects of programming: the languages and environment used for the specifications; the specifications themselves; the determination of whether the computer has executed a specification as expected; the display of data involved in the execution of the specification; etc.

In the past, traditional programming systems in well-known or standard programming languages, e.g., FORTRAN or PL/1 have been used to program computers. However, the problem with traditional programming systems is that they require programmers to learn the cryptic statements and rigid structure or syntax used by standard programming languages. It is for this reason that icons have been used to replace the cryptic statements of standard programming languages with visual programming languages to develop visual programming systems.

Visual programming can be applied to all aspects of programming. The important issue is creating meaningful graphic objects involved in an aspect of programming. This is addressed in the creation of visual programming systems.

One example of a visual programming system is Pict which is designed to aid program implementation using computer graphics. With Pict, users sit in front of a color graphics display and communicate with the system throughout all phases of their work by pointing to icons in a menu tree. Pict permits the user to select images that visually represent the data structures and variables needed; to draw the desired algorithm as a logically structured, multi-dimensional picture; to watch the program run; to see the results being generated; and if the program isn't doing what is expected, to see where and when the error occurs. Although Pict is a visual programming system having control structures for writing computer applications, Pict requires arrows or series of arrows to show the flow of an application. Using arrows to show the flow of an application is somewhat archaic, requires additional computer resources, and is not necessary to depict a program flow.

Although visual programming systems have been developed, these systems fail to appreciate the need to create a visual flowchart that symbolizes the logical flow of the application (or presentation) being developed. These visual programming systems also fail to concentrate on the flowchart metaphor to remove the tedium from program creation. These conventional visual programming systems fail to permit the programmer to assemble pictures, brushes, sounds, speech, animations, music, video, text, and datafiles and control them interactively via a mouse, keyboard, touchscreen, or joystick. Therefore, a single visual programming system which addresses all of these shortcomings of conventional visual programming systems is desirous.

IV. OBJECTS AND SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide for a computer system in which users can create multimedia presentations and coursework.

It is a further object of the present invention to provide for a visual programming system in which users can create applications.

It is still a further object of this invention to enable computer users to program interactive multimedia presentations using a visual programming system.

It is yet another object of the present invention to provide a method for designing presentations using integrated computer technologies on a single platform that enables the user to input, create, manipulate, and output text, graphics, audio, and video utilizing a single-user interface.

To achieve the objects of this invention and attain its advantages, this invention uses a graphic interface display which is implemented as a part of a flow editor and is used to create and to program interactive multimedia presentations and coursework. The present invention also includes other editors (e.g., a database editor, an expression editor, and an object editor) used to perform other editing functions required to create presentations. Furthermore, this invention includes control systems (e.g., an applications mover, a videodisc controller, and a help system) which also enable the user to create, program and execute interactive multimedia presentations. Finally, this invention includes an evaluator which evaluates a programmed presentation (or application) and implements the presentation.

More particularly, the process of this invention is in data processing system including a central processing unit, a memory for storing a plurality of data structures corresponding to a presentantion, and a display device. The display device has a display screen including a presentation area for displaying a presentation. Each one of the data structures includes an action identifier and a plurality of attributes. The process includes receiving a one of the plurality of data structures of the presentation, analyzing the received data structure to determine an action to be performed in response to the action identifier of the data structure, and performing the action corresponding to the action identifier in accordance with a plurality of the attributes of the received data structure.

The accompanying drawings which are incorporated in and which constitute part of this specification, illustrate an implementation of the invention and, together with the description, explain the principles of the invention.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating the components of an exemplary computer system or platform in which the present invention may be implemented.

FIG. 2 is a block diagram which illustrates the software components of the preferred implementation of the present invention.

FIG. 3 is an illustration of a flow window generated by the flow editor of the preferred implementation of the present invention.

FIG. 4 illustrates the main icon menu 400 of the preferred implementation of the present invention.

FIG. 5A illustrates the control icon submenu of the preferred implementation of the present invention.

FIG. 5B illustrates the interrupt icon submenu of the preferred implementation of the present invention.

FIG. 5C illustrates the data icon submenu of the preferred implementation of the present invention.

FIG. 5D illustrates the wait icon submenu of the preferred implementation of the present invention.

FIG. 5E illustrates the AV icon submenu of the preferred implementation of the present invention.

FIG. 5F illustrates the module icon submenu of the preferred implementation of the present invention.

FIG. 6 illustrates the icon menu operational flow of the main icon menu and the icon submenus illustrated in FIGS. 5A-5F.

FIGS. 7A-7D illustrate the relationships used by the preferred implementation to describe icons on the GRID of the flow-window of FIG. 3.

FIG. 8 illustrates an example an icon requester using the animation icon of FIG. 5E.

FIGS. 9A-9B illustrates a block diagrams of an exemplary data structures associated with a presentation created with the preferred implementation and which may be evaluated with the preferred implementation.

FIG. 10 illustrates a block diagram of the flow editor of FIG. 2 and the relationship of the flow editor to other components of the computer system of FIG. 1 and the preferred implementation of the present invention of FIG. 2.

FIGS. 11A-11G are flow diagrams for the preferred implementation of the edit window handler of FIG. 8.

FIG. 12 is a flow diagram for the preferred implementation of the icon requester handler of FIG. 8.

FIG. 13 illustrates a preferred example of the expression editor window displayed on the display screen when the user enters the expression editor.

FIG. 14 is a flow diagram of the expression editor of FIGS. 2.

FIG. 15 illustrates a block diagram of the object editor of FIGS. 2 and the relationship of the object editor to other components of the computer system of FIG, 1 and the preferred implementation of the present invention of FIG. 2.

FIGS. 16A-16M illustrate flow diagrams of the object editor of FIG. 15.

FIG. 17 illustrates a block diagram of the database editor of FIGS. 2 and the relationship of the database editor to other components of the computer system of FIG. 1 and the preferred implementation of the present invention of FIG. 2.

FIG. 18 is an illustration of the database window of the database editor of the preferred implementation.

FIG. 19 is an illustration of the key window of the database editor of the preferred implementation.

FIG. 20 is an illustration of the edit database window of the database editor of the preferred implementation.

FIGS. 21A-21C are flow diagrams of the preferred implementation of the videodisc controller of FIGS. 2 and 8.

FIG. 22 is a flow diagram of the preferred implementation of the applications mover of FIG. 2.

FIGS. 23A-23G are flow diagrams of a preferred implementation of the evaluator of FIG. 2.

FIG. 24 illustrates a flow diagram of the authoring system of the preferred implementation.

DETAILED DESCRIPTION OF THE PREFERRED IMPLEMENTATION

Reference will now be made in detail to the preferred implementation of the invention as illustrated in the accompanying drawings.

This invention is preferably implemented by a microcomputer or other data processing system. Such a data processing system may be conventional, however, the present invention is implemented in an Amiga microcomputer manufactured by Commodore Electronics Ltd. The architecture for and procedures to implement the present invention in the Amiga microcomputer, however, are also not conventional, as they provide for an unique approach to the creation and execution of interactive multimedia presentations and coursework as well as to the programming of applications software. The preferred implementation, which is disclosed hereinafter in functional schematic form, is written primarily in the C programming language.

Referring to FIG. 1, the computer system or platform 100 is comprised of a central processing unit (or CPU) 102, a disk drive 105, a mouse 110, a keyboard 115, and a display 120. The platform 100 may also optionally include other external devices 125, such as a videodisc system or an electronic instrument system.

The CPU 100 is comprised of an input/output unit 130, a random access memory unit (or RAM) 135, a display memory unit 140, a video interface circuit (or VIC) 145, and a microprocessor 150. These units are all well known and operate under control of the system software to process various inputs and provide the outputs necessary to generate desired textual and graphic display information on the display screen 122 of the display 120 or other output unit such as the optional external device 125.

Display memory 140 is a specialized section of RAM which is used to store bit patterns (pixel data) which are read out by the video interface circuit 145 in an appropriate synchronization with the display beam of the display 120 in order to provide the desired display graphics and text.

The disk drive 105 is also conventional and is provided to permit the ready interchange of control and application software and to provide a source of mass storage for the computer system.

The mouse 110 of the computer system 100 includes a roller ball 111 and control buttons 112 and 113. The buttons actuate momentary contact switches to generate selection signals and other commands. These switches and signals are well known and, as is also well known, the user moves the mouse 110 along a planar surface, such as a table top, to generate cursor position input commands which are supplied to the CPU 102. The roller ball 111 cooperates with a mechanism which converts the movement of the mouse 110 into X-Y signals which are used by the CPU 102 to control positioning of the cursor symbol on the display screen 122 of the display 120. The conversion of the motion of the roller ball into x-y commands is also conventional.

The keyboard 115 may replace the activities of the mouse 110 by presetting a number of keys on the keyboard to emulate the positioning function of the mouse. Additionally, other keys on the keyboard 115 may replace the functions of the buttons
112 and 113 of the mouse 110. However, in the preferred implementation of the present invention, a mouse 110 is used for positioning the cursor on the display screen 122 and for performing other functions described below. As is generally the case with conventional data processing systems, the keyboard 115 of the computer system 100 of the preferred implementation of the present invention acts as a means of inputting textual or alphanumeric information into the CPU 102. As stated above, the display
120 is comprised of a display screen 122 for displaying the graphic and alphanumeric information output from the CPU 102. In the platform 100 of the preferred implementation the display 120 may be a touchscreen display in which commands may be entered into the CPU 102 via the display 120. Such touchscreen displays are also conventional.

Finally, in the preferred implementation of the present invention other external devices 125 may be connected to the platform 100 to participate in the execution of a presentation created by the user. Examples of these external devices are videodisc systems or electronic instrument systems. These systems are also conventional and when connected to the platform 100 may be used to create and present multimedia presentations and coursework.

A. The Major Components of the Preferred Implementation

The preferred implementation of the present invention is comprised of several software components 200 (FIG. 2) which would reside in the disk drive 105 (FIG. 1). When the user employs the preferred implementation of the present invention, all or part of the software components 200 of the preferred implementation may be input to the CPU 102 to service the needs of the user.

FIG. 2 is a block diagram which illustrates the software components 200 of the preferred implementation of the present invention. The preferred implementation is comprised of a flow editor 210, an applications mover 220, a database editor 230, an evaluator 240, an object editor 250, a videodisc controller 260, a help system 270, and an expression editor 280.

When the preferred implementation is invoked or begins execution in the platform 100, the flow editor 210 is the initializing component which provides for an editing environment which is supported of one or more flow windows generated on the display screen 122 of the display 120. The flow window is the canvas on which the user creates presentations. In this environment, the user can create and edit one or more presentations simultaneously by selecting icons, placing the icons in a particular location of a flow window, and defining the selected icons. The icons represent operations or activities to be performed during the execution of an presentation.

From the flow editor 210, the user can invoke the applications mover 220, the database editor 230, the evaluator 240, the object editor 250, the videodisc controller 260, and the expression editor 280, each of which will be described below. Additionally, the user can invoke the help system 270 from the flow editor 210.

In the preferred implementation menus, which are collections of system options, may be displayed on the current display screen 122 when and while the user presses the right mouse button 112. The user may then move the cursor, using the mouse
110, through the options of the displayed menus. As the cursor passes over a menu option, the option is highlighted on the display screen. The user may make a selection of one of the menu options by releasing the right mouse button 112 while the preferred implementation highlights the selected option. The preferred implementation supports menus in the flow editor 210 and the object editor 250 to allow the user to move throughout the editors and other support systems of the preferred implementation, as well as to add objects to the current display screen and select other editor-mode selections, e.g., alter color of current display screen of the flow editor.

FIG. 3 illustrates an example of a flow window 300. The flow window 300 is the interface used to create and edit presentations in the preferred implementation of the present invention. The flow window 300 is comprised primarily of the Graphic Interface Display (or GRID) 310 upon which a user can place selected icons (discussed below). One or more icons on the GRID 310 can form a presentation.

The flow window 300 also consists of a number of gadgets. A gadget is an area in the flow window 300 which allows the user to change what is being displayed by communicating a command to the CPU 102 (FIG. 1). The flow window 300 includes a close window gadget 315, a drag bar gadget 320, window-to-front gadget 325, a window-to-back gadget 330, a vertical positioning gadget 335, a horizontal positioning gadget 340, scrolling gadgets 345a and 345b, and 350a and 350b, and a flow window resizing gadget 355.

When the user manipulates the mouse 110 so as to position the cursor on the close window gadget 315 and clicks the left mouse button 113, the flow editor 210 of the preferred implementation receives a command to close the flow window 300. A click of the left mouse button 113 is a quick press and release of that button 113.

The drag bar gadget 310 serves two purposes. First, the drag bar gadget 320 serves as an area of the flow window 300 in which the name or title of the presentation may be displayed. When the user has not yet titled the presentation presently displayed on the GRID 310, the area of the drag bar gadget 320 in which the title would be displayed contains the title: Untitled, as illustrated in FIG. 3. A presentation name would appear in place of the untitled area of the drag bar gadget 320 if the user names the current presentation in the GRID 310 or loads from the disk drive 105 a previously saved presentation (discussed below). Second, the drag bar gadget 320 may be used to reposition the flow window 300 horizontally or vertically within the display screen 122 of the display 120 (FIG. 1). Again, to reposition the flow window 300, the user positions the cursor on the drag bar gadget 320 using the mouse 110 and depresses the left mouse button 113. Using the mouse 110, the user can then drag or move the flow window 300 within the display screen 122 until the left mouse button 115 is released.

The window-to-front gadget 325 and the window-to-back gadget 330 serve opposite purposes. The window-to-back gadget 325 permits the user to move a currently displayed flow window 300 to the back of all currently displayed flow windows. The window-to-front gadget 330 permits the user to move a currently displayed flow window 300 to the front of all currently displayed flow windows. Again, these gadgets are activated by positioning the cursor on the selected gadget in the display screen 122
using the mouse 110 and clicking the left mouse button 113 thereby instructing the flow editor 200 to reposition the currently displayed flow window 300 in accordance with the selected gadget.

The vertical positioning gadget 335 and horizontal positioning gadget 340 permit the user to instruct the flow editor 210 to view a viewable portion or region of the presentation presently displayed on the GRID 310. The viewable portion of the presentation is determined by the selected size of the flow window 300. The scrolling gadgets 345a and 345b, and 350a and 350b permit the user to scroll vertically or horizontally within a presentation and the flow window resizing gadget 355 permits the user to resize the flow window 300. These gadgets and all other gadgets and buttons on the display screen 122 during execution of the preferred implementation are initiated in the same manner as the gadgets discussed above.

Returning to FIG. 2, the preferred implementation of the present invention also includes an applications mover 220 which is used by the preferred implementation to move presentations from one location, e.g. the disk drive 105 (FIG. 1), to another location, for example a second disk drive (not shown in FIG. 1). The details of the applications mover 220 will be discussed below with reference to FIG. 22.

The preferred implementation includes a database editor 230 which permits the user to create and manipulate databases for use with presentations. The database editor 230 allows the user to create a database in a standard database format; add, update and delete data records; as well as delete full databases. These operations of the database editor are conventional, however, the method by which the preferred implementation interfaces with the database editor 230 is not conventional and will be described below with reference to FIGS. 17-20.

The preferred implementation also includes an evaluator 240. The evaluator 240 of the preferred implementation of the present invention controls the execution of presentations created with the editors 210, 250, and 280, and the videodisc controller 260. The details of the evaluator 240 of the preferred implementation will be described below with reference to FIGS. 23A-23G.

The object editor 250 of the preferred implementation is used to create display objects for use in a presentation. Display objects are independent visual objects which the user can place on the display screen 122 (FIG. 1). The preferred display objects are: (1) rectangles, (2) polygons, (3) lines, (4) circles, (5) ellipses, (6) text, (7) brushes, and (8) data entry fields. With the object editor 250, the user can create these objects and turn these objects into user input areas that add interactivity to presentations. These input areas are referred to as hit boxes. The functions of the object editor 250 will be discussed below with reference to FIGS. 16A-16M.

The preferred implementation also contains a videodisc controller 260. This controller 260 is used to define video sequences or display selected frames of a videodisc. The videodisc controller 260 permits the user to view video, save frame numbers of a videodisc, and perform other browsing functions of a videodisc. The frame numbers are saved so that they may be used with the video icon (discussed below) to include video in a presentation.

The preferred implementation also includes an expression editor 280. The expression editor 280 is used to define variables and expressions used in a presentation. Variables are useful for storing values in either numerical or in alphabetical (string) form. Variables can then be used in expressions which may be assignment expressions or conditional expressions.

An assignment expression is an expression in which the presentation requests that the preferred implementation assign a value to a variable, for example SCORE=100. In this example of an assignment expression, the variable SCORE is assigned the value 100. In this manner, a presentation can refer to the variable SCORE for the number 100. The conditional expression is generally used to control flow of a presentation. For example, a conditional expression may be SCORE>=100. In this expression, SCORE is greater than or equal to 100 and the preferred implementation understands this conditional expression as meaning "if score is greater than or equal to 100." Further details of the expression editor will be described below with reference to FIGS. 13 and 14.

Finally, the preferred implementation includes a help system 270. The help system 270 provides the user with helpful information which the user requires in order to properly perform selected functions within the preferred implementation. The functions used by the help system 270 of the preferred implementation are conventional and will therefore not be described.

B. Icons (Menus and Submenus) and Relationships

At the center of the preferred implementation is the icon menu which stretches across the bottom of the display screen 122 (FIG. 1) when the preferred implementation is first invoked by the user. The user inputs to the CPU 102 an appropriate command to invoke or begin the processing of the preferred implementation. When the preferred implementation is invoked by the user, the processing of the flow editor 210 begins.

To select an icon from the icon menu, the user positions the cursor, using the mouse 110 (FIG. 1), on the selected icon and clicks the left mouse button 113. The preferred implementation then either displays an icon submenu (FIGS. 5A-5F) or permits the user to drag the selected icon into the flow window 300 for placement in the GRID 310.

FIG. 4 illustrates the main icon menu 400 of the preferred implementation of the present invention. When entering the flow editor 210 (FIG. 2), the main icon menu 400 appears on the bottom of the display screen 122. In addition to a trashcan icon 410, the main icon menu 400 offers access to six submenus of icon commands.

The trashcan icon 410 displayed in the main icon menu 400 is used during an editing session in the flow editor 210 (FIG. 2) of the preferred implementation to throw away or discard unwanted icons.

The control icon 420 offers the submenu of icons illustrated in FIG. 5A, the interrupt icon 430 offers the submenu of icons illustrated in FIG. 5B, the data icon 440 offers the submenu of icons illustrated in FIG. 5C, the wait icon 450 offers the submenu of icons illustrated in FIG. 5D, the AV icon 460 offers the user a submenu of icons illustrated in FIG. 5E, and the module icon 470 offers the user a submenu of icons illustrated in FIG. 5F.

FIG. 6 is a state diagram which illustrates the method used by the preferred implementation to scroll from the main icon menu 400 to the icon submenus illustrated in FIGS. 5A-5F. First, when the user begins an editing session in the flow editor
210 of the preferred implementation, the main icon menu 400 is displayed in the bottom of the display screen 122 (state 610). When the user positions the cursor using the mouse 110 on an icon in the main icon menu 400 and clicks the left mouse button
113 on the selected icon, the user selects one of the icon submenus (state 620) and the selected icon submenu along with the main menu icon (not shown in FIGS. 5A-5F) and trashcan icon 410 are displayed on the bottom of the display screen 122 (state
630).

When the user positions the cursor using the mouse 110 on a icon from the selected icon submenu and clicks the left mouse button 113 on the icon, the selected icon becomes a draggable object. Holding the left mouse button 113 down, the user can then drag a copy of the icon from the icon submenu into the GRID 310 of the flow window 300. The icon remains a draggable object until the user releases the left mouse button 113 (used to drag the icon) when the icon is in the selected space on the GRID
310 (state 640). This process is described below in detail with reference to the processes of the flow editor 210.

Once the left mouse button 113 is released, flow editor operation is returned to the submenu state 630. To return from the submenu state 630 to the main icon menu state 610, the user merely positions the cursor using the mouse 110 on the main menu icon (not shown) which is displayed on the far right in every icon submenu and clicks the left mouse button 113. This informs the flow editor 210 to return to the main menu state 610.

Each of the icons in the icon submenus (FIGS. 5A-5F) represents an action to be performed at the time of the presentation's evaluation (discussed below). Most of the icons perform a general type of action (e.g., playback of animation), but must be individually defined by the user. This definition may include, for example, the selection of the animation file to be played, the number of iterations, the position on the screen, as well as other pieces of information.

The flow window 300 is displayed with the GRID 310 marking the positions icons may be placed. An icon's position, relative to the other icons in the GRID 310, determines how the icons interact. The default traversal of the icon structure is from the top of a presentation in the GRID 310 to the bottom. Icons immediately above/below each other are called sibling icons. Certain icons may be used to group a collection of other icons. These are displayed on the main icon menu 400 (FIG. 4) or submenus (FIG. 5A-5F) with a hollow triangle pointing to the lower right of the icon. When these types of icons are placed in the GRID 310, other icons may be placed below and to the right of them. The triangle is then displayed as a solid, with the marked icon being called the parent and the lower icons called children.

This parenting process allows a presentation to be maintained in a modular manner. When a parent icon is dragged about the presentation and placed in a new position, all of its children are also moved to the new location. When the parent is dragged outside of the GRID 310 and dropped on top of the trashcan icon 410, all of its children are also deleted from the displayed presentation.

In the preferred implementation, there are four basic icon relations: parent icons, child icons, sibling icons, and partner icons. These relations can have a direct effect on the order of execution of a presentation which will be discussed below.

In the preferred implementation, there are nine icons which can function as parent icons: the module icon 551, subroutine icon 552, screen icon 541, loop icon 504, form icon 526, select icon 521, keyboard interrupt icon 511, mouse interrupt icon
512, and grouped wait icon 531, each of which will be described below with reference to FIGS. 5A-5F. As stated, these parent icons are identified by the presence of a hollow triangle in the lower right of the icon. This triangle indicates that the user can place child icons underneath the parent icon. When a parent icon is selected and placed in the GRID 310 and the user selects one or more child icons and places them to the right of the parent icon, the triangle is filled in. On the GRID 310, child icons would be placed beginning one column to the right and one row down from a parent icon.

FIG. 7A illustrates an example of the parent icon-child icon relationship. The module icon 705 of FIG. 7A is a parent icon. This is identified by the triangle in the lower right corner of the module icon 705. In this case, the triangle of the module icon 705 is filled in or appears solid because this module icon 705 has child icons: the screen icon 710, and the brush icon 730. The graphic icon 720 is a grandchild to the module icon 705 and is therefore a descendant of the module icon 705. The operations or acts which would be performed in response to each of these icons will be described below with reference to FIGS. 5A-5F. The screen icon 710 of FIG. 7A is a parent icon with the graphics icon 720 as its child. The parent-child relationship of icons in the preferred implementation is important because the relationship of icons determines the method and order by which the evaluator 240 will execute the operations or acts identified by the icons.

FIG. 7B also illustrates another example of the parent icon- child icon relationship, however, in FIG. 7B, the loop icon 735 is a parent icon which signifies that this parent icon-child icon relationship is one of a parent iterative relationship. This means that the loop icon 735 is used to inform the evaluator 240 to repeat certain operations identified by the child icons of the loop icon 735. The actions of the child icons would be repeated until a condition associated with the loop icon 735
is evaluated as true. In this example, the brush icon 740 and the wait mouse icon 745 would be child icons of the parent loop icon 735 and these child icons may be performed more than once, depending upon the conditions of the loop icon 735.

The preferred implementation also has sibling icons. Sibling icons are icons that are directly above and below each other. The sibling icon may have a partner icon or one or more child icons. FIG. 7C illustrates an example of three sibling icons: an animation icon 750, a speech icon 755, and a wait mouse icon 760, each of which will be described below with reference to FIGS. 5A-5F. As sibling icons, when the evaluator 240 of the preferred implementation executes the operations of the these icons, their operations are performed in a top-down fashion or sequentially.

The forth icon relation used by the preferred implementation is the partner relationship. FIG. 7D illustrates an example of the partner icon relationship. The if-then-else icon 765 requires a partner icon which, in this example, is the brush icon identified by the label brush 1 770. If the expression or condition associated with the if-then-else icon 765 is evaluated during execution as true, then the operations of the partner icon, the brush 1 icon 770, will be executed. Otherwise if the condition of the if-then-else icon 765 is evaluated as false, then the operation of the sibling icon, the brush icon labeled brush 2 775 is executed. In the preferred implementation, if the operation of the partner icon of an if-then-else icon 765 is executed, then the evaluator 240 will continue execution beginning with the next sibling icon immediately following the icon which follows the if-then-else icon. In this example, if the actions of brush 1 770 are executed, then brush 2 775 is skipped and evaluation continues with the icon following brush 2 775 in the presentation which is the wait mouse icon 780.

Returning to FIGS. 4 and 5A-5F, the main icon menu 400 and icon submenus FIGS. 5A-5F will be described. When the user selects an icon from the submenus FIGS. 5A-5F and places the selected icon in the GRID 310 for a presentation, an icon requester must, in most cases, be completed to define the selected icon. Several icons however do not require definition using requesters or the expression editor 280.

Each icon in the submenus FIGS. 5A-5F has a different icon requester. In general, an icon requester is a window (or framed area on the display screen 122) containing information specific to a given icon which must be completed by the user to properly define or describe the attributes for an icon. As will be described in detail below with reference to the operations of the flow editor 210, after the user selects an icon and places it in the GRID 310, the user clicks the left mouse button 113
twice (a double click) to reveal (or to have the flow editor 210 generate on the display screen 122) the appropriate icon requester. The user then completes the icon requester to properly define the icon for later evaluation by the evaluator 240 of the preferred implementation. In cases where no icon requester is used to define an icon the expression editor 280 may be used to define the icon. In other cases, no icon definition is necessary, e.g. call icon or goto icon.

The submenu accessible through the control icon 420 of the main icon menu 400 is used to affect the flow of a presentation through the use of branches and conditional statements. When the user is in the flow editor 210 of the preferred implementation and selects the control icon 420, the main icon menu 400 displayed on the bottom of the display screen 122 is replaced by the control icon submenu 500 (FIG. 5A). In addition to the submenu 500, the trashcan icon 410 is displayed in the far left of the bottom of the display screen 122 and the main menu icon (not shown) which, when selected by the user, returns the main icon menu 400 to the display screen 122, is displayed in the far right of the bottom of the display screen 122. Both the trashcan icon 410 and the main menu icon (not shown) are displayed when the preferred implementation is in the submenu state 630 (FIG. 6) with any of the submenus (FIGS. 5A-5F) displayed on the display screen 122.

The control icon submenu 500 consists of 7 icons. The call icon 501 executes a subroutine which must be defined by the user using the subroutine icon 552 of FIG. 5F. When the user is in the flow editor 210 and selects the call icon 501 and places a copy of the call icon 501 in the GRID 310 (FIG. 3), a referencing placeholder icon (not shown) will appear on the GRID 310 adjacent to the selected call icon which is used to hold the partner icon for the call icon. The partner icon for the call icon 501 is the subroutine icon 552 of FIG. 5F.

A subroutine is a collection of icons with the subroutine icon 552 of the module icon submenu 550 of FIG. 5F as its parent. During evaluation, when a presentation reaches a call icon 501, the referenced subroutine identified in the partner icon to the call icon is performed. During the performance of the subroutine, when either a return icon 554 (FIG. 5F) is encountered or if the subroutine is completed, the presentation will continue starting with the icon following the call.

To select a partner to the call icon 501 placed in the GRID 310, the user double clicks the left mouse button 113 on the referencing placeholder icon adjacent to the referencing icon (e.g., call icon 501). The user is then asked whether he or she wishes to specify the icon to be referenced (e.g., the partner). If yes, then the next icon upon which the user places the cursor on the display screen 122, using the mouse 110, and double clicks the left mouse button 113 initiates the referencing process. The user if then asked if the double clicked icon is the desired referencing partner icon. If yes, then the referenced icon's image replaces the original referencing placeholder icon, and the referencing process is complete. If the user does not wish to reference the selected icon, then the selection process may be continued with another reference icon or aborted.

The conditional-goto icon 502 is used to branch to another part of a presentation on a specified condition. This icon conditionally transfers the flow of logic from one part of the presentation to another. The conditional-goto icon 502 cannot contain children, but requires a partner. The partner is a reference to an icon elsewhere in the presentation. In a manner similar to the call icon 501, when the user selects the conditional-goto icon 502 and places the icon on the GRID 310 for a presentation, a placeholder icon (not shown) appears in the GRID 310 adjacent to the conditional-goto icon which holds the place for the partner icon which will identify where to branch to in the presentation. The user selects a partner for this icon in the manner described above with reference to the call icon 501. Additionally, the user must input an expression, using the expression editor 280 (discussed below), to indicate to the presentation when it is to branch to the identified partner. To invoke the expression editor 280, after placing the conditional- goto icon 502 on a GRID 310, the user places the cursor over the icon and double clicks the left mouse button 113.

Another control icon in the control icon submenu is the goto icon 500. This icon is used for unconditional branching or transfer control within a presentation. The goto icon 503 cannot contain children, but, like the call icon and conditional goto icon 502, requires a partner. Again, when the goto icon 503 is selected by the user from the control icon submenu 500 and placed in a GRID 310, a placeholder appears in the GRID adjacent to the goto icon and the user must specify where the presentation is to branch to when executing the goto statement. This icon represents an unconditional branch in a presentation and therefore does not require definition by using a requester or the expression editor 280. However, the user must select a partner for this icon in the manner described above with reference to the call icon 501.

Another control icon in the control icon submenu 500 is the loop icon 504. The loop icon 504 is used to specify a loop structure within a presentation. The loop icon 504 does not have a partner, but it does require children as described above with reference to FIG. 7B. Children are identified on the GRID 310 by placing icons on the GRID 310 in the column to the right of the parent icon beginning with the row directly below the row upon which the parent icon is placed. The user selects the loop icon 504 to set up a structure to cycle through a group of children icons.

Three types of loops may be constructed with the flow editor 210 and are defined using the loop icon requester and the expression editor (discussed below). They are: the endless loop, the counted loop, and the conditional loop. Each of these loop structures has different exit conditions. The endless loop can be terminated with the loop exit icon 505, which can also be used to terminate the other types of loop structures. The counted loop terminates at the end of the count specified using the loop icon requester and the conditional loop is ended when a selected condition, written in the expression editor, is set to false during the performance of a presentation. During a presentation, when the presentation reaches a loop icon, the actions of the children icons are performed. When the actions of the children icons are completed, the presentation will resume execution from the beginning of the loop. If an exit condition is reached, the loop stops and the presentation moves on to the next sibling of the loop icon.

The exit loop icon 505 ends a loop structure and during a presentation, when an exit loop icon 505 is reached, the presentation continues with the next sibling icon following the loop. The loop exit icon cannot contain children and does not have a partner icon. The loop exit icon 505 does not require definition by an icon requester because when this icon is encountered during execution of a presentation, the current (inner-most) loop executing is exited.

The if-then icon 506 of the conditional icon submenu 500 is used to define a condition which, if true, will cause the action of its partner icon to be performed during a presentation. If the condition is false the partner icon is skipped and the action of the sibling icon following the if-then icon will be performed. In either case, the icon following the if-then icon is always performed. Thus, the if-then icon 506 cannot have children but does require a partner. Again, to set the condition for the if-then icon, the user defines the if-then icon 506 using the expression editor (discussed below) which can be initiated from the flow editor 210 by double clicking the left mouse button 113 while the cursor is positioned on the if-then icon 506
placed in the GRID 310. The partner icon for the if-then icon 506 is selected in the same manner discussed above with reference to the call icon 501.

Finally, the control icon submenu 504 has an if-then-else icon 507. This icon 507 defines a condition for executing one of two separate icons: one if the case specified in the condition, set using the expression editor, is true and one if the condition is false. Similar to the if-then icon 506, the if-then-else icon 507 cannot have children, but requires a partner. During the presentation, if the condition specified for the if-then-else icon in a presentation is true, the presentation performs the actions of its partner icon. It then skips the icon following the if-then-else icon which represents the else part. If the condition is false, then the presentation performs the action of the else part, which is the sibling icon immediately following the if-then-else icon. The if-then-else icon 507 in a presentation is defined using the expression editor (discussed below).

Returning to FIG. 4, the icon main menu 400 also has an interrupt icon 430. When the interrupt icon 430 is selected by the user in the flow editor 210, the interrupt icon submenu 510 of FIG. 5B is displayed on the bottom of the display screen in place of the main icon menu 400. The interrupt icon submenu 510 in the preferred implementation consists of three interrupt icons: (1) the keyboard interrupt icon 511, (2) the mouse interrupt icon 512, and (3) the remove interrupt icon 513. These icons are used to define an action in a presentation that is to be performed during a presentation when the executing presentation is interrupted.

The keyboard interrupt icon 511 allows an interruption to the executing presentation when certain keys are pressed. If one of the specified keys is pressed, the presentation will pause, and the actions of the children of the keyboard interrupt icon will be performed. Thus, the keyboard interrupt icon 511 can have children as well as siblings. The keyboard interrupt icon 511 is defined using the keyboard interrupt icon requester and the object editor 250 (discussed below). To initiate the keyboard interrupt requester, the user double clicks the left mouse button 113 while the cursor is on the icon in the GRID 310. The keyboard interrupt requester has a gadget that, when selected, enables the user to enter the object editor 250.

The mouse interrupt icon 512 interrupts a presentation when a mouse button 112 or 113 is clicked. The mouse interrupt icon 512 defines an interrupt to the presentation flow if the mouse is clicked in a certain area of the display screen 122. If interrupted, the presentation will pause and the actions of children of the interrupt will be performed. The mouse interrupt icon 512 is defined using the mouse interrupt icon requester and the object editor 250 (discussed below). To initiate the mouse interrupt requester, the user double clicks the left mouse button 113 while the cursor is on the icon in the GRID 310. The mouse interrupt requester has a gadget that, when selected, enables the user to enter the object editor 250.

Finally, the remove interrupt icon 513 only disables interrupts in the same column on the GRID 310 of the presentation that have the same parent. This icon 513 does not contain children.

Another submenu of the main icon menu 400 is the data icon submenu 520 illustrated in FIG. 5C. The data icon submenu 520 defines a set of icons used to define variables, define data entry forms, store and retrieve data from a database, and define printed or file output in a presentation. Of the data icons in the data icon submenu 520 there are three icons which exclusively relate to data operations on an existing database: the select icon 521, the read/write icon 522, and the delete icon
523 (FIG. 5C).

The select icon 521 of the submenu 520 can be used to open a database file and select records using one or more fields. The select icon can have other icons as children. One or more of the fields may be key fields. As described more fully below with reference to the database editor 230 of the preferred implementation, a key is made up of one or more fields of the database record structure and is used when searching the data file for a specific: record or a set of records. For example, a database of employee information may contain employee information alphabetically by the last name or by employee ID number. Therefore when creating the database the last name field and employee ID field are specified as key fields. In this way, the user can access data records either by specifying the employee ID or the employee last name. The select icon 521 in a presentation is defined using the select icon requester.

Another data icon in the data icon submenu 520 is the read/write icon 522. This icon 522 reads and writes to database records which were previously selected using the select icon 521. The read/write icon 522 cannot contain children. When using this icon, the user assigns a variable to a field in the database record, and selects the appropriate action (read, insert, or update). The read/write icon 522 in a presentation is defined using the read/write icon requester.

Another data icon in the data icon submenu 520 is the delete icon 523. This icon 523 removes the currently selected record. This icon 523 has no children and is defined using the delete icon requester.

The next icon in the data icon submenu is the variables icon 524. This icon 524 is used to define new global variables, or assign new values to existing variables by evaluating expressions specified by the user. The difference between global variables and local variable is conventional. Global variables can be accessed from anywhere in a presentation and local variables can only be accesses in a particular region of a presentation, e.g., within a subroutine. The variables icon 524 can have no children and is defined using the variables icon requester and the expression editor 280.

Next in the data icon submenu 520 is the output icon 525. The output icon 525 is used to send a single line of output to a disk file or a printer. The output icon 525 cannot contain children and is defined using the output icon requester.

The data form icon 526 follows to the right of the output icon 525 on the data icon submenu 520. This icon 526 defines forms on the screen for data entry by users during the execution of a presentation. The data form icon 526 can have other icons as children. The object editor 250 (discussed below) is used to define all of the data fields for the form.

Finally, the data form exit icon 527 of the data icon submenu 520 is used to exit or abort a data form operation. The form exit icon 527 cannot contain children and this icon 527 can only be used as a child to the data form icon 526.

Returning to FIG. 4, to the right of the data icon 440 is the wait icon 450. When the user selects the wait icon 450, the wait icon submenu 530 illustrated in FIG. 5D replaces the main icon menu 400 on the bottom of the display screen 122. The wait icon submenu 530 consists of five icons.

The first icon in the wait icon submenu 530 is the grouped wait icon 531. This icon 531 is used to combine wait icons. The function of the grouped wait icon 531 is as a parent to other specific wait icons from the wait icon submenu 530. This icon 531 waits for any one or all of its children to be completed.

The next icon on the wait icon submenu 530 is the wait condition icon 532. This icon 532 is used to wait for a specific condition to be true. Once the condition occurs, the presentation continues. This icon 532 cannot contain children and the condition is defined using the wait condition icon requester and the expression editor 250 (described below).

The next icon in the wait icon submenu 530 is the wait keyboard icon 533 which is used to pause the presentation and wait for a desired keystroke. This icon 533 cannot contain children. When the user selects this icon 533, there are two options. The first is to wait for a specific key or keys to be pressed. Second, the user may allow for the presentation to wait for any key to be pressed. A keyboard icon requester and the object editor 250 are used define the condition of this wait icon 533. The display objects and text for the wait keyboard icon 533 are created in the object editor 250 (described below).

The next wait icon in the wait icon submenu 530 is the wait mouse icon 534. This icon 534 is used to pause a presentation and wait for a desired click of a mouse button 112 or 113 (FIG. 1). The wait mouse icon 534 has no children. Similar to the wait keyboard icon 533, the wait mouse icon 534 has two options. First, is to wait for a mouse click in a specific hit box or area of the display screen 122 and the second is to wait for any mouse click. A wait mouse icon requester and the object editor 250 are used define the condition of this wait icon 534. The display objects and text for the wait mouse icon 534 are created in the object editor 250 (described below).

Finally, the last icon in the wait icon submenu 530 is the delay icon 535. The delay icon 535 is used to pause the presentation for a specified number of seconds. It does not require a response from the user. This icon 535 has no children. With this icon, during evaluation (or execution of a presentation), the evaluator 240 does not move to the next icon until a preset time has elapsed. The delay icon 535 is defined by the delay icon requester.

Referring again to FIG. 4, the main icon menu 400 also contains an AV icon 460. When this icon is selected, the AV icon submenu 540 illustrated in FIG. 5E replaces the main icon menu 400 on the bottom of the display screen 122 (FIG. 1). Audiovisual icons are used to perform operations such as playing video, animation, sound, speech, or musical files, and displaying pictures and graphics.

The left-most icon in the AV icon submenu 540 is the screen icon 541. The screen icon 541 is used to define the background screen for presenting any visual information such as pictures. This icon 541 uses an icon requester to specify display parameters, e.g., screen resolution, number of colors, palette, and the size of the picture. The screen icon 541 may also be used to load a bit-mapped image from the disk drive 105 (FIG. 1) to display on the display screen 122. Bit-mapped images are conventional and therefore will not be explained. The screen icon 541 can have other screen icons as children as well as any other AV icons from the AV icon submenu 540.

To the right of the screen icon 541 on the AV icon submenu 540 is the digitized sound icon 542. This icon 542 is used to play a recorded voice or sound that has been previously digitized. This icon 542 cannot have children and is defined using the digitized sound icon requester.

Next in the AV icon submenu 540 is the synthesized speech icon 543. The synthesized speech icon 543 can be used to play back text that the user inputs or text from an ASCII text file. This icon 543 cannot contain children and is defined using the synthesized speech icon requester.

Next to the synthesized speech icon 543 on the AV icon submenu 540 is the music icon 544. The music icon 544 is used to play back musical scores created in music software programs. This icon 544 also cannot contain children and is defined using the music icon requester.

The fourth icon from the left in the AV icon submenu 540 is the graphics icon 545. This icon is used to modify and control the display screen 122 (FIG. 1) and enables the user to place display objects (created using the object editor) on the display screen 122. This icon also lets the user specify color cycling effects. Like most of the AV icons, this icon 545 also cannot have children and is defined using a graphics icon requester.

To the right of the graphics icon 545 in the AV icon submenu 540 is the brush icon 546. This icon 546 is used to overlay a specific picture file on top of the current display screen 122. The brush icon 546 also cannot have children. The brush icon 546 differs from the screen icon 561 in that it does not delete the existing background pictures or graphics on the display screen 122. It also does not modify screen attributes such as resolution. However, the user may specify palette changes in the brush icon requester used to define this icon 546.

The AV icon submenu 540 also contains a video icon 547. The video icon 547 is used to play a segment of video or a single video frame from a videodisc player which may be part of the platform 100 of FIG. 1. The video icon 547 cannot have children and is defined using the video icon requester and may make use of the videodisc controller 260 (discussed below).

To the right of the video icon 547 on the AV icon submenu 540 is the animation icon 548. The animation icon 548 is used to play back an animation file which has been created in a conventional paint or animation software application. The animation icon cannot have children and is defined by the animation icon requester.

At the right hand side of the AV icon submenu 540 is the text file icon 549. The text file icon 549 is used to display text from an ASCII file onto the display screen. The text file icon cannot have children and is defined using the text file icon requester.

Returning to FIG. 4, the last icon on the right of the icon main menu 400 is the module icon 470. When the user selects the module icon 470, the main icon menu 400 is replaced on the display screen 122 (FIG. 1) with the module icon submenu 550
illustrated in FIG. 5F.

The module icon submenu 550 consists of six icons. The first icon in the module icon submenu 550 is the module icon 551. The module icon 551 is used to help organize presentations. The module icon 551 can be used as a parent for other icons or groups of icons including other module icons. Thus the module icon 551 can contain other module icons like itself, as well as other icons as its children. The module icon 541 can also be a child to other parent icons. The module icon is defined using the module icon requester and the expression editor 280. Variables defined in a module of a presentation are local to that module. For example, if the user defines variables for the module icon 705 in FIG. 7A, these variables exist during the evaluation of the module icon's 705 descendants including icons 710, 720 and 730. These local variables would not exist during the evaluation of the module icon's 705 siblings (not shown).

Another icon in the module icon submenu 550 is the subroutine icon 552. The subroutine icon 552 provides another method of structuring a set of actions that the user wishes to use repeatedly in a presentation. The subroutine icon 552 can contain other icons as children, but cannot contain itself as a child. It must always appear in the left-most column of the GRID and therefore, may be a sibling of the very first module icon. The subroutine icon 552 is defined using the subroutine requester and the expression editor 280. Variables associated with a subroutine icon are subject to the same scoping (local) as described above with reference to the module icon 541.

To the right of the subroutine icon 552 on the module icon submenu 550 is the quit icon 553. The quit icon 553 can be used to exit and return to the flow editor 210 when creating a presentation or terminate the execution of a presentation.

The next icon in the module icon submenu 550 is the return icon 554. The return icon 554 explicitly stops the execution of a subroutine and returns control back to the next icon following a call icon 501. This icon 554 cannot have children and it can only appear as a part of the flow of a subroutine. The preferred flow editor 210 will not permit the user to place the return icon 554 outside of a subroutine.

The execute icon 555 is adjacent to the return icon 534 in the module icon submenu 550. The execute icon 555 references an external program and allows the external program to execute as a part of the presentation flow. The execute icon 555
cannot have children and is defined in a presentation with the execute icon requester.

The timer icon 556 of the module icon submenu 550 is used to time specific parts of a presentation. It does not stop the presentation, but merely acts as a stopwatch measuring time in elapsed seconds with up to two decimal places. This icon 556
also cannot contain children and is defined using the timer icon requester.

Finally, the last icon in the module icon submenu 550 is the resource control icon 557. The resource control icon 557 is used to preload and unload resources such as picture, sound, animation and music into memory 135 of the platform 100 (FIG.
1). This icon 557 is used to reduce long waits in the middle of a presentation for the system to load the required information. This icon 557 also cannot contain children and is defined using the resource icon requester.

Returning to FIG. 6 the operational flow of the icon menus may be further explained in the context of a typical editing session as follows. When the flow editor 210 of the preferred implementation is started, the user is presented with an editing screen on the display screen 122 showing a screen header across the top, a panel of icons across the bottom, and an untitled presentation or flow window on the left.

The editing process is started by selecting a specific icon for placement in the empty flow window 300. The icons initially displayed on the panel at the bottom of the display screen 122 consists of the main icon menu 400 (see FIG. 4) which represents the different types of actions the preferred implementation supports. Positioning the cursor on an icon and clicking the mouse (or selecting the icon) instructs the preferred implementation to display on the bottom panel of the display screen
122 the appropriate icon submenu (FIGS. 5A-5F) depending upon which icon from the main icon menu 400 (FIG. 4) was selected. For example, the main icon menu 400 in the preferred implementation has an AV icon 460 which represents the submenu containing all of the audiovisual actions supported by the preferred implementation. These include the icons discussed above with reference to FIG. 5E.

Once the desired submenu has been displayed, any one of the icons in the selected submenu may then be selected for placement in the GRID 310 of the flow window 300 by (1) positioning the cursor on the icon, (2) pressing the left mouse button 113, (3) holding the left mouse button down and dragging the icon with this button depressed, (4) positioning the selected icon being held or dragged by the mouse in the proper position in the GRID 310 of the flow window 300, and (5) releasing the left mouse button 113. The selected icon will now be added to the presentation displayed in the GRID 310.

The definition of icons may be performed in a requester specific to each type of icon. The requester for an icon placed in the GRID 310 is "opened" by double-clicking the left mouse button 113 (FIG. 1). The icon requester is then displayed on the display screen 122. Opening the requester for the first time presents an empty requester with only limited preset attributes (or descriptive information). The requester includes a set of one or more buttons which are regions that the user may activate to modify different attributes. The user can then modify any of the attributes by clicking on the appropriate buttons. Some buttons present further requesters for entering numeric values, selecting a file from a directory on a disk, etc.

Although each icon has specific attributes and therefore a specific icon requester, an example of an icon requester will now be explained with reference to the icon requester for the animation icon 548 discussed above with reference to FIG. 5E.

FIG. 8 illustrates the preferred icon requester 800 for the animation icon 548. The animation icon 548 is used to play back an animation file created using a conventional paint or animation software application. The animation icon requester 800
consists of several input fields in which the user must input information to set the attributes of the animation icon 548 or to define the animation icon 548. The animation icon requester 800 also consists of several gadgets which are used to set icon attributes and reposition the requester 800 on the display screen 122. To initiate each of these gadgets, the user positions the cursor using the mouse 110 on the gadget in the display screen 122 and then clicks the left mouse button 113. This permits the user to alter the attributes associated with an icon using the gadgets of the icon requester.

The first gadget in the animation icon requester 800 is the directory gadget 810 which permits the user to select, using the file requester, the name of an animation file to be played. Alternatively, the user may click the left mouse button 113
on the filename field 815 in the animation icon requester 800 and type in the name of an animation file in the space 815. The animation icon requester 800 also has an override screen gadget 820 which defaults to "on" and uses the screen resolution of the animation file, completely replacing the previous screen. If the user clicks the left mouse button on this gadget 820 to turn it off, then the evaluator 240, during execution of the presentation, will assume the current resolution, i.e., the resolution of the last screen that was displayed.

The palette gadget 825 is used to specify between the current palette and an override palette. A palette is the set of colors specified for display on the display screen 122. The palette gadget 825 is defaulted to the current palette which, if retained by the user, causes the specified animation to be displayed using the current palette of display colors. If the override is selected by the user, then the animation's palette will be used, changing the existing colors displayed on the display screen 122.

The loop gadget 830 is selected if the user wishes to play the animation a specified number of times. The number of loop repetitions is specified in the reps field 835a of the animation icon requester 800 by clicking the left mouse button 113 on the reps gadget 835 which enables the user to enter a number into the reps field 835a. In FIG. 8, the loop gadget 830 has not been activated and therefore the reps gadget 835 is "ghosted" or appears shaded. If the user selects the loop gadget 830, then the reps gadget 835 will be "unghosted" and the user will be permitted to alter the number, e.g., 0, in the reps field 835a.

The left gadget 845 and the top gadget 850 of the animation icon requester 800 are used to specify the coordinates of the top left corner of the picture in the selected animation file. The user merely clicks the left mouse button 113 on either gadget to enter the specific value for these fields 845a and 850a, respectively. The transitions gadget 860 of the animation icon requester 800 is used to specify the screen pattern to be used when switching to the first picture of an animation.

The animation icon requester 800 also has a preview gadget 865, help gadget 870, reset gadget 875, and a cancel gadget 880. The preview gadget 865 is used to see an operation, without running the presentation, while you are creating the presentation. The help gadget 870 initiates the help system 270 of the preferred implementation which provides the user with help information concerning, in this case, the animation icon requester 800. The reset gadget 875 clears all of the attributes previously set by the user in the animation icon requester 800, and the cancel gadget 880 cancels the animation icon requester 800 and returns the user to the point at which she selected the animation icon and double-clicked on the icon to initiate the animation icon requester 800 in the flow editor 210.

The Icon Name field 885 allows the user to give a meaningful name to the icon which will be shown on the GRID 310. It is also useful when searching for a particular icon. The use and contents of this field is completely up to the user. To enter an Icon Name into the icon name field 885, the user merely clicks the left mouse button 113 on the Icon Name field 885.

The Memo gadget 890 allows the user to add a description of the actions of the icon which will be presented only inside the requester. The use and contents of this field is completely up to the user. To enter a memo into the memo field 890, the user merely clicks the left mouse button 113 on the memo field 890.

The Pause gadget 892 allows the user to specify if icons following the animation icon may be started before the animation is completed. If the Pause gadget 892 is selected (remains checked), the animation will be completely presented before its sibling icon is started. If the Pause gadget is not selected (cleared), the animation will be started, and while being presented, the actions of its sibling icon will be performed. Other gadgets in the animation icon requester, e.g., the drag bar gadget 320, have already been described with reference to the flow window 300 of FIG. 3.

After all attributes have been properly set, the requester can be closed, and the information saved, by clicking on the "OK" button 895 which is in all requesters. Subsequent openings of the requester will display the previously set attributes for review or editing.

C. The Presentation Structure

The preferred implementation may be used to generate or create a presentation, to manipulate or edit already created presentations and to execute presentations using the above-described icons. However, each icon is merely an identification of an act to be performed during the evaluation of the presentation and the icon requester is used to define the identified act. As described above, in the preferred implementation icons have familial relationships which are used to determine the order in which the operations of a set of icons in a presentation are to be evaluated. This familial relationship corresponds to the underlying structure of a presentation which is evaluated by the evaluator 240. FIG. 9A illustrates an example of the structure of a presentation which will determine not only how the evaluator 240 would evaluate and execute this presentation structure but would also determine how the flow editor 210 traverses the presentation structure in response to user commands.

A presentation structure created with the preferred implementation is made up of a RootEvent (not shown) and any number events and commands. The RootEvent is a part of every presentation structure and will be described below with reference to the evaluator 240 processes. An event is an icon which may contain children and a command is an icon that may not contain children.

The small block 900 on FIG. 9A shows a sample presentation as viewed in the flow editor. It contains two module icons 901 and 903 (both containing children) and four other icons. The module icons 901 and 903, and all other icons which are displayed with the triangles, may contain zero, one, or more children, as described earlier. These icons are represented internally by event structures. All icons which cannot contain children are defined internally by command structures.

Command and event structures are similar, with event structures being a superset of the command structures. While the complete details of these structures need not be described here, the primary members are the parent list, the child list, the reference list, and the specific data pointer. List structures are defined by the conventional operating system, and are used because of the operating system-supplied routines for fast and easy maintenance of the list contents. Both events and commands contain a parent list and specific data members, but only the event structure contains child list and reference list members. This is illustrated in FIG. 9A. For example, the event structure 901 contains a parent list 950, a child list 951, a reference list 952, and a specific data member 953, and the command structure 902 contains a parent list 955 and a specific data member 956.

The data structures that connect the event structures and command structures of a presentation together are called "LinkNodes." LinkNodes are small sections of memory that comprise the elements of any of these lists. For example, LinkNode 911
connects event node 901 with command node 902. The child list 951 of event structure 901 points to the LinkNode 911 and the LinkNode 911 points to the command structure 902 with a pointer field 960.

An expanded version of a LinkNode also exists which is called a CondNode. CondNodes are used when two icons are displayed on the same horizontal line in the flow editor, and may only be present on an event structure's child list. For example, in FIG. 9A, the CondNode 919 refers to the conditional-goto specifier displayed in the small box 900. Like other LinkNodes, the CondNode 919 contains a pointer field that points to the command structure 905 representing the conditional-goto icon. Since the CondNode 919 is one which refers to another command or event in the presentation structure 910 to be executed when a condition is evaluated as true, the CondNode 919 contains a reference pointer field 970 which points to the referenced command structure 902.

All of these structures can be seen in the example of a presentation structure 910 which represents the structure of the example presentation in box 900.

For each child of a given event, there exists one LinkNode or CondNode on its child list. In FIG. 9B event E1 901 contains two children, a command C1 902, and another event E2 903. Present on El's child list are two LinkNodes 911 and 915 each pointing to one of the children. Likewise, the child list of event E2 903 contains two elements. The first is a LinkNode 921 pointing to the command C2 904, and the second is a CondNode 919 pointing to the command C3 905. Command C3 905 represents the Conditional Goto icon as shown in block 900 of FIG. 9A. Since the action of the Conditional Goto requires a partner to be specified, the CondNode contains a member 970 (FIG. 9A) which points to an event or command defined elsewhere in the presentation. In this example, the referenced icon is the command C1 902.

Referring again to FIG. 9A, each icon which is a child of another has at least one member on its parent list 950, 955 and 957. This list may contain only LinkNodes and is maintained to allow easy tracking of all points in the presentation that refers to each icon. The first element on the list must be a pointer to the event which contains a LinkNode to the child on the event's child list. These are marked as "ActionLink" LinkNodes 912, 914, 918 and 920. This example shows several one element parent lists (event E2 903, command C2 904, and command C3 905).

Any subsequent LinkNodes on an icon's parent list represent non-parental references to the icon and are marked as ReferenceLink LinkNodes. This setting reflects that the event pointed to contains a LinkNode on its reference list referring to this icon. Only the parent list of command C1 901 contains more than one LinkNode, because this is the only icon in the example that is referenced from some other point in the presentation (by the Conditional Goto C3 905). The ReferenceLink LinkNode
913 further marks that the event pointed to has at least one CondNode on its child list that points to the icon 902 containing the ReferenceLink LinkNode 913 on its parent list 955. Only events contain reference list members, which may contain only LinkNodes, each one pointing to the non-child icons referred to by all of the CondNodes on the event's ActionList. For example, as illustrated in FIG. 9A, the event E2 903 has a reference list which contains LinkNode 916 which points to the command C1
902.

Referring again to FIG. 9B, most icons may only be defined by opening their requester in the flow editor 210, specifying the desired information and choosing the OK button. This information is stored either in the Event/Command structure, or in a block of data pointed to by the specific data pointer. This pointer may in fact point to a list of nodes, each storing part of the information specified by the icon. The majority of the information is stored outside of the Event/Command structure to allow these structures to be as small as possible.

The command C1 902 is an animation icon which will show the animation named "Example.AnimFile" in a loop 3 times in paused mode. Likewise the speak icon contains a data block defining all of the attributes of the phrase to be spoken. This is shown in FIG. 9B where the specific data pointer 956 of command C1 902 points to the specific data block 940 which stores information specific to the actions of the animation icon (shown in block 900 of FIG. 9A) as set by the user (described below). Similarly, the specific data pointer 958 of command C2 904 points to the specific data block 941 the action of a speech icon (shown in block 900 of FIG. A).

Both events shown in the diagram contain at least one expression 942, 943, and 944. These are stored in distinct data blocks, marked as expressions, which contain the string version of the expressions. When the module event is encountered by the evaluator 240 during presentation, each of the expressions is evaluated, defining or redefining variables to be used elsewhere in the presentation.

Expression data blocks are also used by all CondNodes which point to commands that define conditional expressions for certain icons (e.g., conditional goto, If-then, or If-then-else). This is shown in FIG. 9B where the CondNode 919 points to the expression data block 945.

D. The Authoring System

The preferred implementation of the present invention provides users with an authoring system in which users can create presentations having a structure of the type described in FIGS. 9A-9B using icons from the submenus (FIGS. 5A-5F) and evaluate the created presentations.

As illustrated in the flow diagram of FIG. 24, the user selects an icon from one of the icon submenus and the preferred implementation receives an indication of the user's selection (step 2410). Based upon the user's selection of a particular one of the icons in the submenus, the preferred implementation then generates a data structure in the memory of the platform 100 (FIG. 1) associated with the selected icon (step 2420). As discussed above, each icon represents an action or operation to be performed by the CPU 102 of the platform 100 during the evaluation process. Additionally, one or more data structures corresponding to selected icons form a presentation.

After the data structure for a selected icon has been generated, the preferred implementation then displays on the GRID 310 in the display screen 122 an image representing the selected icon at the position in the GRID 310 selected by the user (step 2430). After the user selects an icon and places it in the GRID 310 and defines its attributes, the user may evaluate the data structure to perform the action represented by the data structure for each selected icon (step 2440).

E. The Editing Session (Flow Editor)

FIG. 10 illustrates a block diagram of the flow editor 210 of FIG. 2 and the relationship of the three components of the flow editor 210: the icon menu 1010, the edit window handler 1020, and the icon requester handler 1030, to the software components of the preferred implementation identified in FIG. 2 and the computer system components first identified in FIG. 1. In other words, FIG. 10 illustrates that the icon menu 1010 of the flow editor 210 is connected to the edit window handler
1020 and to no other components of the preferred implementation. The icon menu 1010 however is connected to the disk drive 105, the mouse 110, and the keyboard 115 of the platform 100 (FIG. 1). Others skilled in the art may develop other methods of compartmentalizing the functions and operations of the flow editor 210, however, the preferred flow editor has been separated into components 1010, 1020, and 1030 in the preferred implementation to explain easily, the preferred operations of the flow editor. This is not meant to limit the present invention to this particular structure for this flow editor 210.

The icon menu 1010 of the flow editor 210 has been described above with reference to FIGS. 4, 5A-5F, and 6.

The edit window handler 1020, on the other hand is connected to every component of the preferred implementation except the expression editor 280. This means that when the flow editor 210 is executing in the preferred implementation, the edit window handler 1020 cannot access the expression editor 280. However, the edit window handler 1020 can access the applications mover 210, the database editor 230, the evaluator 240, the object editor 250, the videodisc controller 260, and the help system 270. The edit window handler 1020 is also connected to every component as the icon menu 1010 by virtue of them both being a part of the flow editor 210.

The last component of the flow editor 210 is the icon requester handler 1030. The icon requester handler 1030 is used to define or fully describe icons (by creating icon requesters) selected by the user during an editing session. For example, if, during an editing session, a user selects the control icon 420 from the main icon menu 400 (FIG. 4) and enters the control icon submenu 500 (FIG. 5A) and selects the if-then icon 506 to be inserted into the presentation on the GRID 310, the user must also define this icon using, in this case, the appropriate icon requester for the if-then icon. This identifies the condition in which the user wishes the presentation to perform the "then" or partner icon of the if-then icon.

Like the other components of the flow editor 210, the icon requester handler 1030 is connected to the disk drive 105, the mouse 110, and the keyboard 115 of the computer system 100 (FIG. 1). The icon requester handler 1030 is also connected to the evaluator 240, the object editor 250, the videodisc controller 260, the help system 270, and the expression editor 280 of the preferred implementation. That is, when the user has selected an icon from the icon menu and places the icon in a presentation in the GRID 310 and enters the icon requester window of the preferred implementation, the user can, from the icon requester, access the appropriate one(s) of these five components of the preferred implementation. Each of these components, except the help system 270, will be described more fully below.

Referring to FIGS. 11A-11G the operations of the edit window handler 1020 during an editing session will be described.

When the user begins the processing of the preferred implementation, the processes 1100 of the edit window handler 1020 are automatically performed. First, as illustrated in FIG. 10A, the edit window handler 1020 opens the edit screen (step
1101). The edit screen is an area of the display screen 122 which includes zero, one or more flow windows, the icon menus (FIGS. 4 and 5A-5F) along the, bottom of the display screen 122 and the main system menu along the top of the display screen 122
(not shown).

The edit window handler 1020 then displays in the edit screen the main icon menu 400 (FIG. 4) (step 1103) and opens a flow window (step 1105). The flow window is initially untitled. The flow editor then awaits a user action (step 1107) at which point the user may retrieve a presentation previously created from the disk drive 105 (FIG. 1), begin creating a new presentation in the flow window 300 or initiate some other operation of the preferred implementation, e.g., execute database editor. A user action may be initiated by positioning the cursor using the mouse 110 on a selected area of the display screen 122 and then presses the left mouse button 113 to select an operation of the edit window handler 1020. User action may also be implemented using other means, e.g., right mouse button 112 or keys on the keyboard 115, as described below.

When a user action is input the edit window handler 1020, responds by performing the requested function. If the user is in the edit window handler 1020 and clicks the left mouse button 113 while the cursor is inside of the GRID 310 (step 1109), then the operations of the edit window handler 1020 continue with step 1140 in FIG. 11B. If the click is outside of the GRID 310, and not on top of any of the icons in the icon menus, the click is ignored. The edit window handler 1020 is only concerned with user actions inside flow windows containing a GRID 310.

FIG. 11B shows the flow of operations of the edit window handler 1020 used to insert or edit icons in the GRID 310. First, the edit window handler 1020 determines whether it is presently in the collect mode (step 1140). The collect mode is when the user is selecting a group of existing icons for rearrangement in the GRID 310. If the edit window handler 1020 is not in the collect mode (step 1140), then the edit window handler 1020 determines whether the user has clicked on a box in the GRID 310
which already contains an icon (step 1141). If the user has not clicked on a box in the GRID 310 containing an icon (step 1141), then the edit window handler 1020 determines whether any icon in the GRID 310 has been selected (step 1142). If no, then the operation of the edit window handler continues with step 1107 of FIG. 11A. Otherwise, the edit window handler 1120 unselects the current icon (step 1143) and continues with step 1107 of FIG. 11A.

If the user has clicked on a box in the GRID 310 which contains an icon (step 1141), then the edit window handler 1020 determines if this icon is currently selected (step 1144). Selected icons are icons that appear highlighted in the GRID 310. Unselected icons are not highlighted in the GRID 310. If so, and the second click was within a predetermined time period (step 1145), then the icon requester handler 1030 is initiated for the selected icon (step 1146). Once the operations of the icon requester handler (discussed below) are complete, the flow editor returns to step 1107 of the edit window handler 1020 of FIG. 11A. If the user did not double click on the icon within the predetermined time period (step 1145), then the edit window handler 1020 continues operation with step 1107 of FIG. 11A.

If the user has not clicked on the currently selected icon (step 1144), then the edit window handler 1020 unselects the previously selected icon and then selects the currently selected icon from the GRID 310 (step 1147). After the new icon is selected, the edit window handler 1020 determines whether the user wishes to initiated a dragging action of the selected icon (step 1148). A user initiates a dragging action by holding the left mouse button 113 down while the cursor is on top of the selected icon. If no dragging action has been requested (step 1148) then the edit window handler continues with step 1107 of FIG. 11A. Otherwise, if the user wishes to drag the selected icon (step 1148), then the edit window handler 1120 creates a draggable object of the selected icon and permits the user to move the icon until the mouse button is released (step 1149). After the icon is dragged to a new box in the GRID 310, the edit window handler continues operation in step 1107 of FIG. 11A.

Otherwise, if the edit window handler 1020 is in the collect mode (step 1140) then the processes of the edit window handler continue with step 1150 of FIG. 11C. The operations of the collect mode of the edit window handler 1020 begin with first determining the logical minimum and maximum number of icons collectable in both the horizontal and vertical directions and generating in the GRID 310 the collection rectangle with which the user initiates the collection process when the user clicks the left mouse button 113 (step 1150). Then edit window handler 1020 draws and redraws the collection rectangle in response to mouse movements until the left mouse button 113 is released (step 1151). Then edit window handler 1020 presents the user with a requester to verify the collection region specified in the collection rectangle (step 1152). The edit window handler 1120 determines whether the user has confirmed the collection of the icons in the region (step 1153). If yes, the edit window handler creates a module icon (as a parent icon), inserts the module icon as the first icon in the selected group and moves all of the selected icons within the collection region to children icons of the new module icon (step 1154). When the operations of the collect mode are complete or if the user does not confirm the collection (step 1152), then flow of control of the edit window handler 1020 returns to step 1107 of FIG. 10A to await a user action.

Otherwise, the edit window handler 1020 determines if the user has positioned the cursor on the display screen 122 in a predetermined location and has pressed the arrow keys 345 and 350 of the flow window 300 (or the arrow keys on a conventional keyboard) or the scroll bars 335 and 340 of the flow window (step 1111). If yes, the edit window handler 1020 will move the viewable portion of the presentation on the GRID 310 in the flow window 300 in the requested direction (step 1113) and then redisplay the flow window 300 in accordance with the user's request (step 1117). The edit window handler 1020 then returns to step 1107 to await the next user action.

On the other hand, if the user positions the cursor in the flow window 300 on the display screen 122 in the resize window gadget 355 (FIG. 3) and resizes the window (step 1115), then the edit window handler 1020 redisplays the resized flow window
300 (step 1117). In the preferred implementation, the resizing process is performed by the operating system of the platform 100. The edit window handler 1020 then returns to step 1107 to await the next user act:ion.

If the user selects the telescoping action from the main system menu along the top of the display screen 122 (step 1119), then the edit window handler 1020 continues with step 1160 of FIG. 11D. In the telescoping option, the user can condense child icons into their parent icons to conserve space on the GRID 310. To perform the telescoping function, the edit window handler first determines whether the current icon selected by the user is a parent icon (step 1160). If no, then the edit window handler ends the telescoping operation and returns to step 1107 of FIG. 11A.

If the current icon is a parent icon (step 1160), then the edit window handler determines whether the current icon's children are visible (are presently being displayed in the GRID 310) (step 1161). If the current icon has children being displayed in the GRID 310, then the edit window handler 1020 finds the first child icon for the current icon (step 1162) and marks the child icon as non-displayed. This informs the edit window handler 1020 that the marked icon should not be displayed in the GRID 310.

After the child icon is marked, the edit window handler 1020 then determines whether the next icon in the GRID 310 is a sibling icon to the marked child icon involved in the telescoping operation (step 1164). If yes, then this sibling icon is marked as a non-displayed icon (step 1163) and the edit window handler 1020 again continues with step 1164. If the next icon the GRID 310 is not a sibling icon to the child icon (step 1164), then the edit window handler 1020 redisplays the GRID 310 with the child icons of the parent icon selected in step 1160 no longer displayed in the GRID 310 (step 1168). The edit window handler 1020 then continues with step 1107 of FIG. 11A.

Otherwise, if the current icon's child icons are not visible in the currently displayed GRID 310 (step 1161), then the edit window handler 1020 locates the first child icon of the current icon (step 1165) and marks the child icon as displayed (step 1166) in order to display the child icon on the GRID 310. The edit window handler then determines whether the next icon in the GRID 310 is a sibling of the marked child icon (step 1167). If yes, then this child icon is also marked for display (step 1166). If the next icon in the GRID 310 is not a sibling of the marked child icon (step 1167), then the edit window handler 1020 redisplays the GRID 310 (step 1168), completes the telescoping function, and returns to step 1107 of FIG. 11A.

If the user in the edit window handler 1020 selects the function of the edit window handler 1020 to drag an icon from the GRID 310 to the trashcan icon 410 (FIG. 4) to delete the dragged icon (step 1121), then the operation of the edit window handler 1020 illustrated in FIG. 11E to delete the icon is performed.

First, the edit window handler 1020 begins the icon deletion process by removing the selected icon from the presentation structure presently displayed in the GRID 310 (step 1170). Next the edit window handler 1020 determines whether the deleted icon is a parent icon (step 1171). If the deleted icon is a parent icon (step 1171), then the edit window handler 1020 finds the first child icon of the deleted parent icon (step 1172) and then removes that child icon from the presentation structure (step 1173). If this child icon is also a parent icon (step 1174), then the edit window handler returns to step 1172 to find the first child icon of this parent icon. This is a recursive process which is a conventional programming function.

If the child icon is not a parent icon (step 1174), then the edit window handler 1020 determines whether the next icon in the presentation structure is a sibling icon to the deleted child icon (step 1175). If yes, then the sibling icon is removed from the presentation (step 1173) and the edit window handler 1020 determines whether this removed icon is a parent icon (step 1174). If the next icon in the presentation is not a sibling of the child icon that was removed with its parent icon (step 1175), then the edit window handler 1020 determines whether there are any remaining icons visible in the GRID 310 of the flow window 300 (step 1176).

If there are icons remaining in the GRID 310 (step 1176), then the edit window handler redisplays the GRID 310 without the deleted icon (step 1180). Otherwise, the edit window handler determines if there are any icons remaining in the presentation currently being displayed (step 1177). If there are more icons in the displayed presentation, then the edit window handler 1020 changes the GRID 310 position in the presentation to view at least one of the remaining icons (step 1178). If there are no other icons remaining in the presentation, the edit window handler 1020 creates a module icon and adds the module icon to the presentation to duplicate the untitled presentation status (step 1179). After either step 1178 or step 1179, the edit window handler 1020 redisplays the GRID 310 in the flow window 300 and then returns to step 1107 of FIG. 11A.

In FIG. 11A, if the user selects an icon for placement in the GRID 310 (step 1123), then the operation of the edit window handler continues with step 1181 of FIG. 11F. In this case, the edit window handler 1020 first determines whether the selected icon is coming from another GRID 310 or being copied from another part of the same GRID 310 (step 1181). If yes, then the edit window handler 1020 makes a copy of the icon and all of its children, if any, for insertion into this GRID 310 (step
1182).

If the icon is not coming from another GRID 310 or another point in the same GRID 310 (step 1181), then the icon is coming from an icon submenu and the edit window handler 1020 determines whether the user's placement of the icon in the GRID 310
is valid (step 1183). After making a copy of the icon and its children (step 1182), the edit window handler 1020 also determines whether placement of the icon (and its children) is valid (step 1183). In determining whether the placement of an icon is valid, the edit window handler 1020 considers whether the new or copied icon can be a child, mate, or sibling.

If the placement of the new icon is not valid, then the edit window handler 1020 deletes the new icon and any children if the new icon was one which was copied from this or another GRID 310 or selected from an icon submenu (step 1190). Subsequently, the edit window handler returns to step 1107 of FIG. 11A.

Otherwise, if the placement of the new icon is valid (step 1183), then the edit window handler 1020 must consider whether the new icon is being moved from another part of the same GRID 310 (step 1184). Since an icon that is being moved (not copied) from one position to another position in the same GRID 310 and the new position cannot be located in the children, grandchildren, etc. of the original position, the edit window handler 1020 determines whether the new position for the copied icon is a new descendent of the original (step 1185). Once the new position is known, the child list of the original icon is checked to determine whether new position is a descendent of the original (step 1185). If yes, the edit window handler 1020
terminates the operations of FIG. 11F and continues with step 1107 of FIG. 11A. However, if the new position is not inside this region of the presentation, the move is valid (step 1185) and the edit window handler 1020 removes the icon from the original position in the GRID 310 (step 1186). Then, or if the icon being moved is not from another part of the same GRID 310 (step 1185), the edit window handler 1020 adds the new icon at the requested position in the GRID 310 (step 1187).

If the icon being added to the pr