Home
Patent Search
IMT Blog
REGISTER
|
SIGN IN
United States Patent Application
20020194501
Kind Code
A1
Wenocur, Michael L. ; et al.
December 19, 2002
System and method for conducting a secure interactive communication session
Abstract
System, method, signal, operating model, and computer program for electronic messaging. Systems and method for providing security for communication of electronic messages, interactive sessions, software downloads, software upgrades, and other content from a source to a receiving device as well as signals used for such communications. Systems, methods, signals, device architectures, data formats, and computer program structures for providing authentication, integrity, confidentiality, non-repudiation, replay protection, and other security properties while minimizing the network bandwidth, computational resources, and manual user interactions required to install, enable, deploy and utilize these security properties. System, device, method, computer program, and computer program product for searching and selecting data and control elements in message procedural/data sets for automatic and complete portrayal of message to maintain message intent. System, device, method, computer program, and computer program product for adapting content for sensory and physically challenged persons using embedded semantic elements in a procedurally based message file.
Inventors:
Wenocur; Michael L.
(Palo Alto, CA)
, Baldwin; Robert W.
(Palo Alto, CA
)
, Illowsky; Daniel H.
(Cupertino, CA
)
Correspondence Name and Address:
Suite 3400 Four Embarcadero Center
FLEHR HOHBACH TEST ALBRITTON & HERBERT, LLP
San Francisco
CA
94111
US
Series Code:
912855
Filed:
July 25, 2001
U.S. Current Class:
713/201;
713/150
U.S. Class at Publication:
713/201;
713/150
Intern'l Class:
H04L 009/00
Claims
We claim:
1. A computer program product for use in conjunction with a computer system having a server and a client, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism, comprising: a program module that directs the computer system and/or components thereof including at least one or the client or server, to function in a specified manner to provide message communications, the message communications occurring in a computer system hardware architecture neutral and operating system neutral and network transport protocol neutral manner for secure interactive communication sessions, the program module including instructions for: A. sending to a server, by a client, a first message containing a Client-Nonce; B. receiving said first message including said Client-Nonce by said server; C. sending to the client, by the server in response to said received first message and Client-Nonce, a second message containing a copy of the Client-Nonce extracted from the first message, and a value in the form of a Server-Nonce that was chosen by the Server that is not predictable by the Client and is unlikely to have been previously chosen by the Server; the first message and second message having substantially the same content, format and cryptographic processing; D. exchanging third and fourth messages between the client and the server (client to server message) and the server and the client (server to client message) respectively, where the order that said third and fourth messages are sent and received is not material; said third and fourth messages including a content portion that is substantially the same though not necessarily identical and having substantially the same format and cryptographic processing as each other and as with subsequent data transfer messages; the data contents portions of the third and fourth message include a cryptographic transformation of at least the Client-Nonce and Server-Nonce, where the cryptographic transformation is slightly different in the third and fourth messages; and E. each of the server and client examining the respective received third and fourth messages to confirm that they have the expected contents and thus were created by an entity that knew both the Client-Nonce and the Server-Nonce.
2. A hardware architecture neutral and operating system neutral and network transport neutral method for secure interactive communication sessions using less software code and network bandwidth than conventional systems, said method comprising: A. sending to a server, by a client, a first message containing a Client-Nonce; B. receiving said first message including said Client-Nonce by said server; C. sending to the client, by the server in response to said received first message and Client-Nonce, a second message containing a copy of the Client-Nonce extracted from the first message, and a value in the form of a Server-Nonce that was chosen by the Server that is not predictable by the Client and is unlikely to have been previously chosen by the Server; the first message and second message having substantially the same content, format and cryptographic processing; D. exchanging third and fourth messages between the client and the server (client to server message) and the server and the client (server to client message) respectively, where the order that said third and fourth messages are sent and received is not material; said third and fourth messages including a content portion that is substantially the same though not necessarily identical and having substantially the same format and cryptographic processing as each other and as with subsequent data transfer messages; the data contents portions of the third and fourth message include a cryptographic transformation of at least the Client-Nonce and Server-Nonce, where the cryptographic transformation is slightly different in the third and fourth messages; and E. each of the server and client examining the respective received third and fourth messages to confirm that they have the expected contents and thus were created by an entity that knew both the Client-Nonce and the Server-Nonce.
3. The method in claim 2, further comprising after said sever and said client have examined and confirmed that the third and fourth messages were created by entities that knew both the Client-Nonce and the Server-Nonce; F. the Client and Server optionally sending subsequent data messages that have substantially the same format and cryptographic processing as the third and fourth messages.
4. The method in claim 2, further comprising after a last message has been communicated between said client and said server or between said server and said client; (G) terminating the session without a separate session termination message by closing the underlying network connection.
5. The method in claim 3, further comprising after a last message has been communicated between said client and said server or between said server and said client, (G) terminating the session without a separate session termination message by closing the underlying network connection.
6. The method in claim 4, wherein the underlying network connection is a TCP based connection, by closing the TCP socket.
7. The method in claim 5, wherein the underlying network connection is a TCP based connection, by closing the TCP socket.
8. The method in claim 2, wherein the first and second message have no cryptographic processing when the protocol used for the messages is attempting to reuse one or more cryptographic master keys that were established in a previous messaging session, and the first and second messages have substantially the same format, and the Server verifies the existence of a Key-ID from the first message in a server cache of pairs of Key-ID and Master Key values.
9. The method in claim 8, wherein the first and second message have a common header that includes fields for Type, Version, and Content-Length; the first message contents containing a Key-ID and a Client-Nonce; and the second message contents containing the same Key-ID, the same Client-Nonce, and a new Server-Nonce.
10. The method in claim 8, wherein the Key-ID is a cryptographic hash of a previously set up Master Key.
11. The method in claim 10, wherein the cryptographic hash is a MD5 based hash, a SHA-1 based hash, or a SHA-256 based hash.
12. The method in claim 2, wherein the Client-Nonce and Server-Nonce have the same length.
13. The method in claim 2, wherein the Client-Nonce and the Server-Nonce have a length of 8 bytes, 10 bytes, 16 bytes, 20 bytes, 24 bytes, 32
bytes, 64 bytes, 96 bytes, or 128 bytes.
14. The method in claim 2, wherein the first and second messages are cryptographically processed using public key operations and these messages have substantially the same format and cryptographic processing, and the Client and Server verify the certificate chain in the received second and first message respectively.
15. The method in claim 2, wherein the public key operation comprises an RSA operation or an RSA based operation.
16. The method in claim 2, wherein: the first and second messages are created using a Signed-Inside-Enveloped-Data cryptographic primitive; the Client-Nonce is sent to the Server encrypted by the Server's public key in the field of the public key encryption block that is normally associated with a data encryption key or with an OAEP padding seed, and this Client-nonce is used as the encryption key for the Encrypted-Data primitive, and each one contains copy of the message Sender's certificate chain; the Server-Nonce is sent to the Client encrypted by the Client's public key in the field of the public key encryption block that is normally associated with a data encryption key or with an OAEP padding seed, and this Server-nonce is used as the encryption key for the Encrypted-Data primitive, and each one contains copy of the message Sender's certificate chain; and transmission of said Sever-Nonce and Client-Nonce in the field normally used for a data encryption key or an OAEP padding seed enabling a single cryptographic primitive to be used for secure session setup and for secure unidirectional messaging and for other secure protocol applications.
17. The method in claim 16, wherein said cryptographic primitives for Signed-Inside-Enveloped-Data provide transport of a secret key from Sender to Recipient using a public key of the recipient.
18. The method in claim 16, wherein said single cryptographic primitive comprises a Signed-Inside-Enveloped-Data primitive.
19. The method in claim 2, wherein the Data carried in the first message is a Client-Nonce and the data carried in the second message is the Server-Nonce.
20. The method in claim 2, wherein a digitally signed portion of the second message can be precomputed and/or reused with different messaging sessions, and so that the Server need not perform a computationally expense private key operation to initiate a secure session.
21. The method in claim 2, wherein a digitally signed portion of the second message is precomputed for different messaging sessions and no session specific private key operation is performed to initiate a secure session.
22. The method in claim 2, wherein a digitally signed portion of the second message is reused from an earlier session for a subsequent messaging session and no session specific private key operation is performed to initiate the subsequent secure session.
23. The method in claim 2, wherein the cryptographic transformation in the third and fourth messages are the same.
24. The method in claim 2, wherein the cryptographic transformation in the third and fourth messages are different by exchanging the roles of the Client-Nonce and the Server-Nonce.
25. The method in claim 2, wherein the cryptographic transformation is a hash of the concatenation of the client-nonce and server-nonce values.
26. The method in claim 25, wherein the hash is selected from the set consisting of MD5, SHA-1, and SHA-256.
27. The method in claim 2, wherein the cryptographic transformation is an encryption of one of either the client-nonce value or the server-nonce value using the other nonce value as the key.
28. The method in claim 27, wherein the cryptographic transformation encryption is selected from the set consisting of triple-DES, XTEA, RC5, and AES.
29. The method in claim 2, wherein the third and fourth messages are created using an Encrypted-Data cryptographic primitive, and wherein the Encrypted-Data key for the third message is different than the Encrypted-Data key for the fourth message, and both Encrypted-Data keys are derived from a Master Key that is computed with the aid of one or more applications of a cryptographic hash function applied to at least the Client-Nonce and the Server-Nonce.
30. The method in claim 29, wherein the Master Key is computed with the aid of one or more applications of a cryptographic hash function applied to the Client-Nonce and the Server-Nonce and to some or all of the information in the previously send or received messages.
31. The method in claim 30, wherein the Master Key (MK) is computed as the concatenation of at least a portion of the server-nonce, a portion of the client-nonce, and a portion of the first and second messages.
32. The method in claim 30, wherein the Master Key (MK) is computed as a concatenation as follows: MK=HMAC (Server-Nonce.parallel.Client-Nonce, SHA1 (First-Message).parallel.SHA1 (Second-Message)).
33. The method in claim 29, wherein the Encrypted-Data key for the third message equals HMAC (MK, Client-Subject-Name), where a Client-Subject-Name is generated from one or more fields extracted from the Client's certificate.
34. The method in claim 29, wherein the Encrypted-Data key for the fourth message equals HMAC (MK, Server-Subject-Name), where Server-Subject-Name is one or more fields extracted from the Servers certificate.
35. The method in claim 29, wherein: the Encrypted-Data key for the third message equals HMAC (MK, Client-Subject-Name), where a Client-Subject-Name is generated from one or more fields extracted from the Client's certificate; and the Encrypted-Data key for the fourth message equals HMAC (MK, Server-Subject-Name), where Server-Subject-Name is one or more fields extracted from the Server's certificate.
36. A method for conducting secure interactive communication sessions between a server and a client, said method comprising: sending a first message containing a first token chosen by the client; receiving said first message including said first token by the server; sending a second message containing a copy of the first token extracted from the first message, and a second token that was chosen by the server, by the server; exchanging third and fourth messages between the client and the server, said third and fourth messages including a content portion having substantially the same format and cryptographic processing as each other, the contents portions of the third and fourth messages including a cryptographic transformation of at least the first token and second token; and each of the server and client examining the respective received third and fourth messages to confirm that they were created by an entity that knew both the first token and the second token.
37. The method in claim 36, wherein the cryptographic transformation is slightly different in the third and fourth messages.
38. The method in claim 36, wherein the first token comprises a client-nonce and the second token comprises a server-nonce.
39. A computer program product for use in conjunction with a computer system having a server and a client, the computer program product comprising a computer readable storage medium and a computer program mechanism embedded therein, the computer program mechanism, comprising: a program module that directs the computer system and/or components thereof including at least one of the client or server, to function in a specified manner to conduct secure interactive communication sessions between a server and a client, the communications occurring in a computer system hardware architecture neutral and operating system neutral and network transport protocol neutral manner for secure interactive communication sessions, the program module including instructions for: sending a first message containing a first token chosen by the client; receiving the first message including the first token by the server; sending a second message containing a copy of the first token extracted from the first message, and a second token that was chosen by the server, by the server; exchanging third and fourth messages between the client and the server, the third and fourth messages including a content portion having substantially the same format and cryptographic processing as each other, the contents portions of the third and fourth messages including a cryptographic transformation of at least the first token and second token; and each of the server and client examining the respective received third and fourth messages to confirm that they were created by an entity that knew both the first token and the second token.
40. The computer program in embodiment 39, wherein the cryptographic transformation is slightly different in the third and fourth messages.
Description
RELATED APPLICATIONS
[0001] This application claims the benefit of priority under 35 U.S.C. Sections 119(e) and/or 120 and incorporates by reference each of the following U.S. Patent Applications:
[0002] U.S. Provisional Application Serial No. 60/271,455 (Attorney Docket No. P-70322/RMA) filed Feb. 25, 2001, entitled Hardware Architecture, Operating System And Network Transport Neutral System, Method And Computer Program Product For Secure Communications And Messaging;
[0003] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70553/RMA) filed ______, entitled Hardware Architecture, Operating System And Network Transport Neutral System, Method And Computer Program Product For Secure Communications And Messaging;
[0004] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70554/RMA) filed ______, entitled System and Method for Authorization of Access to a Resource;
[0005] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70555/RMA) filed ______, entitled System, Method, and Structure for Generating and Using A Compressed Digital Certificate;
[0006] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70556/RMA) filed ______, entitled Common Security Protocol Structure and Mechanism and System and Method for Using;
[0007] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70557/RMA) filed ______, entitled System and Method for Conducting A Secure Interactive Communication Session;
[0008] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70558/RMA) filed ______, entitled System and Method for Secure Unidirectional Messaging;
[0009] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70559/RMA) filed ______, entitled Secure Certificate and System and Method for Issuing and Using Same;
[0010] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70560/RMA) filed ______, entitled System and Method for Conducting a Secure Response Communication Session;
[0011] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70561/RMA) filed ______, entitled System and Method for Communicating A Secure Unidirectional Response Message;
[0012] U.S. Utility application Ser. No. ______ (Attorney Docket No. A-70562/RMA) filed ______, entitled System, Method And Computer Program Product For Device, Operating System, And Network Transport Neutral Secure Interactive Multi-Media Messaging; each of which is hereby incorporated by reference.
[0013] U.S. patent application Ser. No. 09/627,357, filed Jul. 28, 2000, entitled Method for Cooperatively Executing a Plurality of Code Threads in a Processor Using Instruction Retry upon Resource Constraints;
[0014] U.S. patent application Ser. No. 09/627,645, filed Jul. 28, 2000, entitled Business Method to Generate and Electronically Distribute Rich Media E-mail Messages to People with Physical Disabilities;
[0015] U.S. patent application Ser. No. 09/627,358, filed Jul. 28, 2000, entitled Business Method for Generating and Electronically Distributing Targeted Author-Once Architecture Independent Rich Media Content;
[0016] U.S. patent application Ser. No. 09/628,205, filed Jul. 28, 2000, entitled Method to Generate and Electronically Distribute Highly Targeted Rich Media E-mail Messages;
[0017] U.S. patent application Ser. No. 09/706,661 filed Nov. 4, 2000, entitled Hardware Architecture Neutral Computer Program Language And Structure And Method For Execution;
[0018] U.S. patent application Ser. No. 09/706,621 filed Nov. 4, 2000, entitled System and Method for Autonomous Generation of Customized File Having Procedural and Data Elements from Non-procedural Flat-File Descriptors;
[0019] U.S. patent application Ser. No. 09/706,664, filed Nov. 4, 2000, entitled System and Method for Intelligently Scaling Procedure/Data Sets to Adapt the Procedure/data Sets to Receiver Attributes and Maintain Message Intent;
[0020] U.S. patent application Ser. No. 09/706,609 filed Nov. 4, 2000, entitled Intent Preserving Message Adaptation and Conversion System and Method for Communicating with Sensory And/or Physically Challenged Persons;
[0021] U.S. patent application Ser. No. 09/706,612 filed Nov. 4, 2000, entitled System and Method for Searching and Selecting Data and Control Elements in Message Procedural/data Set for Automatic and Complete Portrayal of Message to Maintain Message Intent;
[0022] U.S. patent application Ser. No. 09/706,617 filed Nov. 4, 2000, entitled System and Method for Adapting Content for Sensory and Physically Challenged Persons Using Embedded Semantic Elements in a Procedurally Based Message File;
[0023] U.S. patent application Ser. No. 09/706,615 filed Nov. 4, 2000, entitled System and Method for Forward and Backward Content Based Version Control for Automated Autonomous Playback on Client Devices Having Diverse Hardware and Software;
[0024] U.S. patent application Ser. No. 09/706,611 filed Nov. 4, 2000, entitled System and Method for Reducing Unauthorized Access by Procedural Messages Executing in a Computer System to Computer System or Memory or Programs or Data Stored Therein;
[0025] U.S. patent application Ser. No. 09/706,614 filed Nov. 4, 2000, entitled System and Method for Self-directed Loading of an Input Buffer with Procedural Messages from a Stream of Sub-files Containing Sets of Logical Files;
[0026] U.S. patent application Ser. No. 09/706,610 filed Nov. 4, 2000, entitled System and Method for Device-Neutral Procedurally-Based Content Display Layout and Content Playback;
[0027] U.S. patent application Ser. No. 09/706,616 filed Nov. 4, 2000, entitled System and Method for Thin Procedural Multi-Media Player Run-Time Engine Having Application Program Level Cooperative Multi-threading and Constrained Resource Retry with Anti-Stall Features;
[0028] U.S. patent application Ser. No. 09/706,613 filed Nov. 4, 2000, entitled System and Method for Streaming Multimedia-Rich Interactive Experiences Over a Communications Channel; and
[0029] U.S. patent application Ser. No. 09/706,606 filed Nov. 4, 2000, entitled System and Method for Cooperative Application-Level Multi-Thread Execution Including Instruction Retry Feature Upon Identifying Constrained System Resource; each of which is hereby incorporated by reference.
FIELD OF INVENTION
[0030] This invention pertains generally to systems and methods for providing security for communication of electronic messages, interactive sessions, software downloads, software upgrades, and other content from a source to a receiving device as well as signals used for such communications; and more particularly to systems, methods, signals, device architectures, data formats, and computer program structures for providing authentication, integrity, confidentiality, non-repudiation, replay protection, and other security properties while minimizing the network bandwidth, computational resources, and manual user interactions required to install, enable, deploy and utilize these security properties.
BACKGROUND
[0031] Numerous security protocols has been proposed in the academic literature and many have been deployed in commercial products. Currently the most popular protocol for secure sessions between a client machine and a server machine is SSL/TLS, which provides an interactive two-way connection that has at least one party authenticated using a digital certificate issued by a mutually trusted third party. Secure browser-based electronic commerce is almost always performed with the help of the SSL protocol. The most popular secure protocols for unidirectional messaging (e.g., e-mail) are S/MIME and PGP, which provide encryption and/or digital signatures based on digital certificates. The most popular protocols for secure downloads and upgrades are Authenticode and Signed JAR files, which also use digital certificates The most popular systems for requesting and issuing digital certificates are PKCS-7&10 and the S/MIME CMS protocol.
[0032] Each of these protocols requires a large amount of software code and data memory to implement and the steps needed to enroll or register to use these systems are time consuming and in other ways annoying to users. A system that needed to implement all of these protocols would be very difficult to implement on a device with limited memory and computing resources, and very annoying to the users.
[0033] These protocols do not provide solutions to the problem of securely authorizing a specific user the right to access a specific resource, such as a web page or software upgrade, in a manner that cannot be spoofed by a third party.
[0034] The need for appropriate security protocols, procedures, and methods are particularly problematic for electronic messaging in general, and for electronic mail or email in particular.
[0035] Electronic mail, commonly referred to as e-mail, is broadly acknowledged as the "killer" application of the Internet and is a major contributor to its growth, but in a number of ways e-mail is stuck in the past. Most e-mail messages, particularly in a business or other commercial environment but also frequently in personal or non-commercial environments as well, have a predetermined intent, goal, or other purpose directed at achieving some particular result or response from the e-mail receiver. Once a message is composed and published, it is generally expected that the intent and quality of presentation of the message will be preserved. In the past, when e-mail was exclusively or primarily symbol or text based, maintaining the goal or intent of the message was relatively strait forward. If the message was well authored so as to present the desired intent and the message was received, it was likely that the receiver would having sufficient intelligence, appreciate the intent of the message. As e-mail has evolved, it may frequently include non-symbolic or non-textual information, for example, digital images or pictures, graphics, digital audio, video, and the like. Usually, these non-symbolic content enhancements are provided as attachments to the basic message. Frequently, the intent of the message or the reason for sending the message will be partially or even entirely lost unless the non-symbolic portion, such as a video attachment, is also viewed by the receiver. Whether the content enhancements are ever seen or heard by the e-mail recipient may be functions of the recipients hardware, software, programmed preferences, sophistication, as well as other tangible and intangible factors. The e-mail author, sender, or forwarder may typically not know these tangible or intangible factors for any particular recipient.
[0036] For these and other reasons that will be described in greater detail herein, conventional procedures for generating and distributing e-mail unfortunately do not typically preserve either the intent of the message or the quality of the presentation when sending messages to a broad range of e-mail client devices (the types and sophistication of which are nearly unlimited) unless concerted efforts are made to maintain the intent and quality. As a result, conventional approaches used to generate and distribute e-mail severely restrict the impact that e-mail could have on recipients and mainstream e-commerce applications.
[0037] One problem, for example, with conventional approaches used to generate and distribute e-mail is related to the fact that content in e-mail messages is typically not adjusted to the hardware capabilities of an e-mail client that will actually receive the content. If the content of the e-mail is not generated to be compatible with the hardware capabilities of a particular e-mail client, the desired intent of the message may be completely lost. Such hardware and/or software capabilities include, for example, audio capabilities, motion video capabilities, microprocessor type, the amount of memory that is available to store and/or execute the e-mail content, display monitor screen size, and display monitor characteristics, which in turn depend on both the logical circuitry (provided by a video adapter) of the display monitor and display monitor screen size, and the like.
[0038] Consider an example where an e-mail publisher sends an e-mail advertisement message that consists of a color motion video of a diamond ring. If the message is received by an e-mail client that does not have required hardware for computing graphical transformations, for example, a graphics accelerator card, the recipient of the message will not be able to view the motion video portion of the message, and a necessary component of the message will have been lost, the motion video.
[0039] Clearly, some client device types will be able to receive, format, and display or present each and every one of the information items included in an e-mail message. Equally clearly, other client device types would be unable to present any but the minimum set of information items, and likely none of the information items unless only the minimum compatible information items was actually communicated. For example, a cellular telephone having only one or a few lines of monochrome display, a low-end Personal Data Assistant (PDA), or the like information appliance having limited display and/or limited multimedia presentation capabilities would only be able to display small amounts of text or limited monochrome graphics. Therefore, while it would be desirable to generate and distribute optimized e-mail messages that include content that is compatible with all e-mail enabled client hardware configurations, this has not been achieved in practice.
[0040] Heretofore, e-mail is not typically authored to take into account the hardware, software, and user preference attributes of the e-mail recipient. Only where a user has subscribed to some service where the content is authored specifically for a particular intended recipient or group of recipients may the content sometimes be tailored to match these attributes. For electronic messages sent to a large number of intended recipients, such as for a mass consumer advertising campaign, where no knowledge of the users' hardware, software, or preference attributes is available, conventional systems and methods do not facilitate providing an optimized e-mail communication that maintains the intent of the message. Therefore, it has been necessary to rely on a least common denominator approach for such e-mailings where the impact of the communication must frequently be sacrificed so that the message may be received and viewed by a maximum number of the intended recipients.
[0041] If the publisher in this example above for the diamond ring generated the e-mail content with a least common denominator approach that incorporated only that content that is compatible with the hardware of all e-mail clients, for example, textual content, the level of quality that may have been desired to show the advertisers products in a positive light would also be lost with respect to an e-mail client that does have the necessary hardware capability to view the motion video. All recipients would merely receive a text message saying for example, "Three Carat Diamond Ring, $1595.00 at Joe's Jewelry Store", rather than at least some potential buyers viewing a multi-media presentation on the ring and other attributes of Joe's Jewelry Store. Therefore, it is also desirable to substantially optimize e-mail to take significant advantage of those respective capabilities and attributes that are known or may be knowable either before sending the message or after the message is received. Related to these ideals is the fact that e-mail messages often include extra information that while compatible with the hardware capabilities of an e-mail client, cannot or will not be used by the e-mail dient.
[0042] For example, there is no need to include color image data in a message that is being sent to a device that only has a monochrome monitor. A monochrome monitor cannot display a color image no matter how fancy a video card the device may have. To make matters even worse, there are a number of undesirable side effects of sending such extra information. For example, the extra information may take up a significant amount of limited memory resources of the receiving device, and/or, depending on the communication channel connection characteristics of the client device, may slow down the speed at which the message is received by the device. In addition, in spite of the fact that a user's device may be capable of receiving a rich-media message, the user may simply prefer not to receive advertisements or other e-mail having multi-media or rich media content.
[0043] Another problem with conventional techniques for generating and exchanging e-mail, is that e-mail messages are not typically generated such that an e-mail client's network connection characteristics are considered. As a result, the presentation of the e-mail message may be compromised. Such network connection characteristics include, for example, nominal speed or bandwidth of network connections, latencies, throughput, and other contemporaneous communication link/channel attributes. This is a problem because, even though a client device may be capable of receiving a very rich message, if the then prevailing communication channel is only supporting low speed or low bandwidth communication, the conventional systems and methods do not provide procedure to reduce the richness of the message while maintaining the goal or intent of the message. In fact, conventional streaming techniques for rich media tend to do just the opposite, that is to permit any reduction in quality so that the content is received within a real-time or near-real-time time constraint. In some instances, the content may be so degraded as not to offer any useful information at all.
[0044] Another problem with conventional techniques for generating and exchanging e-mail, is that e-mail messages are typically generated in a manner that is insensitive to individual user preferences. Such preferences include, for example, preferred language, security level, physical disability requirements, content layout, demographic information, and the like. For example, a user may be a predominantly Spanish-speaking individual who prefers to receive information, for example, text and audio, in the Spanish-language where possible, rather than in for example the English language. If a message is generated in a language that is not understood by the recipient, the recipient will not be able to understand the message without additional assistance, for example, with assistance by a language interpreter. Even if the message might be understood by the recipient, it may fail to make the desired impression on the recipient. Additionally, if the message does not comply with the recipient's physical disabilities, for example, blindness or deafness, the recipient also may not be able to fully understand the message without additional assistance, for example, having the message translated into a Braille or an audio format. As illustrated in both of these example, if the e-mail is generated in a manner that is insensitive to individual user preferences, the full impact and intent of the message is generally lost.
[0045] To complicate matters, an e-mail client device that has received an e-mail may forward the e-mail to additional e-mail enabled devices, and they in turn may forward the message to other e-mail clients, and the like. Each of these additional e-mail clients may have similar, narrower, or broader hardware capabilities, network connection characteristics, and corresponding user preferences as compared to the capabilities, characteristics and preferences of a forwarding e-mail client. Desirably, e-mail messages are generated in a manner such that the respective content of the e-mail is optimized and compatible with the respective hardware capabilities, connection characteristics, and user preferences associated with all e-mail clients, regardless of whether the e-mail client received the message directly from the publisher or from an intermediary by way of forwarded e-mail.
[0046] Yet another problem with conventional e-mail is that it provides poor navigational and procedural control for e-commerce applications, and conventional e-mail has little or no capability for rich graphics, audio, video, or interactive controls. As a result, conventional e-mail severely restricts the ease of use of e-mail and the impact that e-mail could have on recipients and mainstream e-commerce applications. Such applications include, for example, business-to-consumer (B2C) e-commerce and business-to-business e-commerce (B2B). This problem becomes more apparent every day, because increasingly, communications between suppliers and customers is being accomplished via e-mail. Customers are inquiring about products and orders via e-mail, and suppliers are alerting existing and potential customers about new products and services.
[0047] To illustrate this problem, refer to Table 1, where there is illustrated a targeted promotion in the form of an e-coupon from an on-line business or retailer (sometimes referred to as an retailers) to a consumer (this is an example of a business to consumer or B2C transaction) that offers the consumer a gift certificate.
[0048] To take advantage of the retailer's targeted promotion, a recipient must perform an number of time consuming navigational and procedural steps. For example, at step 1, the recipient must point her browser to the on-line retailer's web site on the world wide web (www). At step 2, the recipient must select the items of interest and be sure not to use a particular payment method (1-click), but instead place the selected items in the shopping cart. At step 3, the recipient must select a "checkout" button. Finally, at step 4, the recipient must wait until prompted by the retailer's web site to type in the numbers of the provided gift certificate claim code to generate an order form to complete the transaction. These procedures are time consuming and require complicated navigation for the recipient of a targeted promotion to generate an order in response to the promotion.
1TABLE 1
EXAMPLE OF AN E-COUPON FROM AN ON-LINE RETAILER To: dan_i@pacbell.net Amount: U.S. $10.00
From: on-line retailer.com Claim code (YOU'LL NEED THIS WHEN ORDERING!): 2AUH_RX8A7G_RE73YL Expiration date: December 3, 1999
Using your gift certificate is easy. Just follow these steps: 1. Visit our Toys & Video Games store at http://www.on-line retailer.com/toys. 2. Select the items you want. Please use our Shopping Cart rather than our 1_Click .sup.SM ordering to pay for your order with a gift certificate. 3. Hit the `Proceed to Checkout` button.
[0049] To make matters even worse, the recipient of a targeted promotion must be connected to the internet to respond to the promotion. Often an e-mail recipient will download e-mail from an internet connected device to a non-internet connected device for example, a handheld PDA, for later perusal at a location that may not have convenient internet access. However, it can be appreciated from the foregoing discussion, that to perform the procedural and navigational steps required for the recipient to respond to the promotion, the recipient must be connected to the internet because there are no procedures for the recipient to navigate the steps outlined in the promotion without connecting to the retailer's web site.
[0050] Desirably a targeted promotion would include interactive controls and content that is generated such that it is optimized and compatible with the respective hardware capabilities, connection characteristics, and user preferences associated with all e-mail clients. Such interactive controls would allow a recipient of a targeted promotion to respond to it without needing to undertake time consuming navigational and procedural steps either to generate an order or to obtain additional information that relates to the promotion. Additionally, it is desirable to have a procedure which will allow the recipient to respond to the promotion without having to respond from a device connected to the internet.
[0051] There are a number of problems that must be solved to overcome the above discussed limitations of traditional procedures used to generate and distribute e-mail. For example, it is rare that an author knows the respective hardware capabilities, connection characteristics, and user preferences of each e-mail enabled device to which a message is targeted. Even if the author did know of such capabilities, characteristics, and preferences, the author would typically be required to perform a number of laborious, time consuming procedures to generate such messages. For example, for each respective device, the author would typically need to manually compose each respective message based on each respective e-mail client's respective capabilities, characteristics, and associated preferences. But, as discussed above, these labors will be moot if the targeted message is forwarded to a device that has different such capabilities, characteristics, and preferences than the device for which the original e-mail message was composed. It is also advantageous that the message be composed automatically without human intervention, and that the message ultimately received by a recipient substantially match hardware, software, and user preference attributes of each individual client device and user.
[0052] Additionally, if an author desires to compose a message, for example, with a similar intent but that is targeted to a different audience than a prior targeted message, the author would typically be required to generate individual messages that not only conform to the different audience, but that also conform to the such capabilities, characteristics and preferences discussed above. For example, it may frequently be desirable to alter the content of an e-mail message to take advantage of a particular cultural context or to avoid particular language or stereotypes that may be detrimental to the intent of the message. For example, if it is known that the receiver identifies themselves with the Armenian-American community it may be advantageous to frame an advertisement so that it is well received by that member of the Armenian-American community and uses for example video images showing Armenian-American's enjoying the product and Armenian music as the background. By the same token, when marketing the same products to an individual identifying himself or herself with the Irish-American community, it may be advantageous to show Irish-Americans enjoying the product and traditional Irish music in the background.
[0053] In light of the above, what is needed is a procedure for generating and exchanging optimized e-mail that conveys the intent of the e-mail publisher across a wide variety of audiences within the boundaries of the hardware capabilities, and connection characteristics of all e-mail enabled devices. Ideally, such optimized e-mail will be generated in a manner that is sensitive to any user preferences of an end user for whom the message is directed. Desirably, a receiver of an e-mail message would be able to access and respond to the message with interactive graphical user interface controls in a manner that does not depend on whether the e-mail client is on-line or off-line. It is also desirable that the e-mail not only be optimized for the user's normal hardware, software, communications channel and other attributes if such are known to the e-mail author, but most desirably to the actual attributes at the time the e-mail message is received by the recipient.
[0054] Also needed are system architectures and program and data structures coupled or used together with appropriate security protocols, procedures, methods, and that provide the desired functionality in a secure manner and desirably do so in an architecture-neutral operating-system neutral, and transport layer neutral environment.
SUMMARY
[0055] The invention provides numerous innovations and enhancements over conventional systems and methods, and where implemented in whole or in part as a computer program (for example, as software, firmware, a combination of software, firmware and/or hardware) also provides computer program and computer program product as well as various articles of manufacture.
[0056] In one aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for authorizing a specific user the right to access a specific resource such as an e-mail message or a promotional coupon. in another aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for representing a digital certificate that enables at least encryption and digital signatures using substantially less storage and bandwidth than conventional digital certificates.
[0057] In another aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for implementing two or more security protocols such as 1) secure interactive sessions, 2) secure unidirectional messaging, 3) secure software downloading, 4) secure software upgrading, and 5) secure issuing of digital certificates, using a common set of data formats, algorithms, subroutines, and methods.
[0058] In another aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for secure interactive sessions using less software code and network bandwidth than conventional systems.
[0059] In another aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for secure unidirectional messaging using less software code and network bandwidth than conventional systems.
[0060] In another aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for secure certificate issuing using less software code and network bandwidth than conventional systems.
[0061] In another aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for secure response session using less software code and network bandwidth than conventional systems.
[0062] In yet another aspect, the invention provides a system, device, method, computer program and computer program product for a hardware architecture neutral and operating system neutral and network transport neutral method for secure unidirectional response message using less software code and network bandwidth than conventional systems.
[0063] The invention provides numerous innovations and enhancements over conventional systems and methods, and where implemented in whole or in part as a computer program (for example, as software, firmware, a combination or software, firmware, and/or hardware) also provides computer program and computer program product as well as various articles of manufacture. Furthermore each of the innovations provides and/or supports one or more business models and methods of during business particularly when the innovations contribute to a generated revenue stream (either directly or indirectly) and fosters relationships between consumers and/or businesses.
[0064] For example, the invention provides a system, device, method, computer program, and computer program product for a hardware architecture neutral computer program language and structure and method for execution.
[0065] The invention further provides a system, device, method, computer program, and computer program product for autonomous generation of customized file having procedural and data elements from non-procedural flat-file descriptors.
[0066] The invention further provides a system, device, method, computer program, and computer program product for intelligently scaling message procedural/data sets to adapt the procedural/data sets to receiver attributes and maintain message intent.
[0067] The invention further provides a system, device, method, computer program, and computer program product for an intent preserving message adaptation and conversion system and method for communicating with sensory and/or physically challenged persons.
[0068] The invention further provides a system, device, method, computer program, and computer program product for searching and selecting data and control elements in message procedural/data sets for automatic and complete portrayal of message to maintain message intent.
[0069] The invention further provides a system, device, method, computer program, and computer program product for adapting content for sensory and physically challenged persons using embedded semantic elements in a procedurally based message file.
[0070] The invention further provides a system, device, method, computer program, and computer program product for forward and backward content based version control for automated autonomous playback on client devices having diverse hardware and software.
[0071] The invention further provides a system, device, method, computer program, and computer program product for reducing unauthorized access by procedural messages executing in a computer system to computer system or memory or programs or data stored therein.
[0072] The invention further provides a system, device, method, computer program, and computer program product for self-directed loading of an input buffer with procedural messages from a stream of sub-files containing sets of logical files.
[0073] The invention further provides a system, device, method, computer program, and computer program product for device-neutral procedurally-based content display layout and content playback.
[0074] The invention further provides a system, device, method, computer program, and computer program product for thin procedural multi-media player run-time engine having application program level cooperative multi-threading and constrained resource retry with anti-stall features.
[0075] The invention further provides a system, device, method, computer program, and computer program product for streaming multimedia-rich interactive experiences over a communications channel.
[0076] The invention further provides a system, device, method, computer program, and computer program product for cooperative application-level multi-thread execution including instruction retry feature upon identifying constrained system resource.
[0077] These and other aspects of the system, device, method, computer program, and computer program product are provided by the invention and each may be utilized separately or in various combinations to provide a broad range of structures, functions, and capabilities.
[0078] In still another aspect, the invention provides various signals, such as signals in the form of digital bit sequences, for providing such communication either with or without security features.
BRIEF DESCRIPTION OF DRAWINGS
[0079] FIG. 1 is a diagrammatic illustration showing a block diagram that illustrates aspects of an exemplary system, according to one embodiment of the present invention;
[0080] FIG. 2 is a diagrammatic illustration showing block diagram that illustrates aspects of an exemplary sender/publisher of content, according to one embodiment of the present invention;
[0081] FIG. 3 is diagrammatic illustration showing an enumerated list that illustrates aspects of an exemplary Extensible Markup Language (XML) document from a sender/publisher, according to one embodiment of the present invention;
[0082] FIG. 4 is a diagrammatic illustration showing block diagram that illustrates aspects of an exemplary sending story server, according to one embodiment of the present invention;
[0083] FIG. 5 is a diagrammatic illustration showing block diagram that illustrates aspects of an exemplary story enabled client, according to one embodiment of the present invention;
[0084] FIG. 6 is a diagrammatic illustration showing block diagram that illustrates aspects of an exemplary procedure, according to one embodiment of the present invention;
[0085] FIG. 7 is a diagrammatic illustration showing block diagram that illustrates aspects of an exemplary procedure, according to one embodiment of the present invention;
[0086] FIG. 8 is a diagrammatic illustration showing block diagram that illustrates aspects of an exemplary Story Compiler implemented on a computer, according to one embodiment of the present invention,
[0087] FIG. 9 is a diagrammatic illustration showing block diagram that illustrates aspects of an exemplary procedural layout of rectangles on a virtual display screen, according to one embodiment of the invention.
[0088] FIG. 10 shows an exemplary embodiment of a Message ID according to the invention; and,
[0089] FIG. 11 is a diagrammatic illustration illustrating steps for creating an embodiment of a message tag from a message ID.
DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION
[0090] Aspects of the inventive system, system architecture, and method are now described so that the security features which may advantageously be used with such system, system architecture, and method will be more readily understood. It will be apparent to those workers having ordinary skill in the art in conjunction with the description provided herein, that the inventive security apparatus, data structures, instructions, codes, methods and other aspects may be utilized with StoryMail.TM. type features as well as with other non-StoryMail systems and methods. Exemplary system architectures and methods are therefore described first, followed by a more detailed description of other security features of the invention. Other aspects of the invention are described in the related applications which are hereby incorporated by reference. While the term storymail or StoryMail may be used to conveniently describe certain types of structures, files, or operations, it will be appreciated that structures, files, or operations that do not formally or exactly satisfy the Storymail criteria but that provide Storymail-like or would otherwise operate with the inventive element may also or alternatively be used.
[0091] Exemplary System Architecture and Method Embodiments
[0092] We first provide a top-level description of some of the key technology components of the invention called a story or other content and systems and methods for authoring, communicating, securing, and rendering such content, along with a description of some of the advantages provided by stories. This description is then followed by several sections that describe the manner in which certain functional and procedural capabilities and/or advantages are achieved in the inventive system. Section headers when provided are provided merely as a convenience to the reader as a guide to portions of the description addressing certain aspects of the invention; however, it will be appreciated that various aspects of the invention are described throughout the description and certain aspects are best described in several portions of the description rather than in a single portion to that relationships may be better understood. Therefore, the description should be considered as a whole with respect to the characteristics or attributes of any structure, system, device, method, procedure, computer program, or other aspect of the invention.
[0093] For purposes of an initial working definition and in somewhat simplified terms, a story as the term is used in this description generally refers to a single, author once, play everywhere file or data/command structure that is interactive either on-line or off-line and that can be used to distribute rich multimedia messages or other rich-media content to all e-mail enabled clients. (More complete as well as alternative definitions of "stories" are described elsewhere in the detailed description.) Next, aspects of an exemplary system to generate, transfer and play stories, according to one embodiment of the present invention, are described. Once this top level description has been provided, the detailed operation of the respective business or operating models and methods of the invention will be described and more readily understood.
[0094] The term e-mail is used here because it represents a form of electronic communication that is known in the art, but it will be appreciated that the inventive system, method, software, business and operating model pertain to much more than what is normally envisioned for conventional e-mail systems and methodologies. The inventive e-mail enhancement, extension, or replacement contemplates some generalized electronic content that is directed to one, a plurality, or a multitude of recipients.
[0095] Recall that in greatly simplified terms, a story is a single, author once, play everywhere file or data/command structure that is interactive either on-line or off-line that can be used to distribute rich multimedia messages or other rich-media content to all e-mail enabled clients. Stories can be used to distribute and coordinate e-commerce transactions, order fulfillment, meeting scheduling, advertisements, catalog item descriptions, customized catalogs and brochures, holiday greeting cards, electronic storybooks, driving directions, vacation slide and picture shows, surveys, real-estate walk thru, medical care pamphlets, pharmaceutical information pamphlets, recipes, business presentations, party invitations, instructional manuals, entertainment, and numerous other applications, particularly where the message consists of more than merely a text or symbolic message. Several of such exemplary applications include, for example, surveys, forms, contracts.
[0096] Story content creation is advantageously automated and dynamically adaptive, because a story is optimized over a plurality of variables to selectively communicate elements of an e-mail message to e-mail client devices and users. Such variables include, for example, client device hardware capabilities, network connection characteristics and user preferences. This is accomplished from a standpoint, for example, of CPU speed, display type, screen size, the existence of and or attributes of audio and/or video capabilities, data scalability, language, use of or not use of audio or visual content, nominal speed or bandwidth of all of the communication links and protocols, and the like.
[0097] In preferred though not all embodiments, a final story is not generated until substantially all such relevant e-mail client information is determined during the time of connection of the client device. In a sense, the system and procedure of the present invention is contrary to other prevailing trends (which attempt to pre-form content so that is available as early as possible) in that StoryMail actually delays composition of the final message until it is ready to be received. For example, if it is determined that an e-mail client cannot view motion video but can display text and play audio, the story will be generated such that it does not include motion video, but rather textual and/or audio elements that communicate the intent of the e-mail publisher within the capabilities of the e-mail client.
[0098] In yet another example, even though a client device may be capable of receiving and rendering a very rich message, if the then prevailing communication channel is only supporting low-speed or low-bandwidth communication, a story is generated such that the richness of the message is reduced so that the message is optimized for the attributes of the client device and the user preferences at that moment in time.
[0099] Sometimes, the message may be optimized or nearly optimized to be received within any time constraints that may be imposed, however, unlike systems and methods that must satisfy real-time or near real time constraints, the story need not provide real-time delivery, as it is intended to be a messaging and communication system, method, and operating model, rather than a real-time rich-media broadcast or streaming system. In this regard, a story is a fully aware e-mail message that is optimized to substantially deliver the intent of an e-mail publisher across the broad range of all e-mail client architectures.
[0100] A story may further be optimized to comply with a predefined set of user defined preferences, making each story beneficially configurable for physically challenged individuals. This is because for every logical element (either text, sound, images, video, or the like logical elements) there is an underlying textual description of that logical element. In addition, there are contextual logical elements included as may be needed to insure that the intent of the message may be easily understood in text or audio only representations. An example of such contextual logical element would be a text element that provides an overview of what is on the screen to be rendered as text or audio in cases where some or all of the screen's visual elements can not be seen by the recipient on the receiving device.
[0101] In a preferred embodiment, all logical elements have corresponding semantic information so that it can be known or determined which elements to use under varying circumstances. For example, the aforementioned contextual logical text element would have associated semantic flags packaged with it inside a story indicating that the element contains text providing an overview of the elements displayed on a screen for use when it is known that the recipient cannot view the screen. Such a case might be when a story player application is used to render and control a rich media message for someone whose only means of communication to the rich media message playing application is over a voice only telephone connection. In other embodiments, an audio representation, either recorded or generated by a text to speech engine may provide audio information backup--contextual information, or semantic information rather than text. In this manner an individual can read text and the text can automatically be articulated for a blind individual.
[0102] In one embodiment, the inventive system, method, and operating model are designed to interface with a peripheral device that generates a Braille or other tactilely sensible indicia corresponding to the story. This peripheral device may either be linked to a conventional client device, such as a computer, or integrated within the device. Using semantics, there is always an alternative sensory presentation mode.
[0103] Stories are self contained and lightweight, meaning that stories have relatively small memory and processor requirements and can be played on client devices the types and sophistication of which are virtually unlimited. A story is self contained because in at least one embodiment, a story is actually a single file that is made up of a number of component logical files. Each component file encapsulates, for example, one or more of computer program instructions, control information, user input forms, validation procedures, and/or multimedia content. Each component logical file is respectively compressed and all of the component logical files are combined, packaged, compressed again to generate the single story file.
[0104] A story is lightweight not only because when it is executed, or played, a story's contents are selectively and sequentially decompressed. But also because a story only includes those elements that are optimized and compatible with the e-mail client's hardware capabilities and network connection characteristics, making stories lightweight (thin) enough to run on inexpensive information appliances or other devices. In fact one of the great advantages of the StoryMail system is its ability to support the hardware capabilities and network connection characteristics of virtually any client device. In fact, a story can even be played on a client device that is not multimedia enabled because a story always has a set of text that describes, or narrates any non-textual element of the story. The story also contains semantic flags indicating the circumstances under which to render all text or non-textual elements.
[0105] A story according to embodiments of the invention is reliable because it is played in a novel run-time environment, wherein, unlike an HTML Web page where there may be links to other servers to provide further information, a story is a self-contained unit. The novel run-time environment is largely deterministic because of the self contained cooperative multitasking system employed in the playback engine and the explicit input buffer coding instructions with fixed size memory buffers. So if it runs correctly one time on one device it will almost certainly run correctly most of the time on all devices.
[0106] A run-time environment such as this is more reliable than, for example a pre-emptive multitasking system using the device's threading mechanism, or an architecture which allows for variable size buffering. Also in story messaging all content is present on the target device before the story is run. So unreliable connections to other devices or content on a network are unnecessary and part of a story cannot be missing since they are packaged together in a single logical file.
[0107] Because a story is self contained and reliable, creation of story content can be completely automated, devices made today will be able to handle future content without upgrades. This provides for intelligent content specific scaling and compression, it is easily stored and exchanged between e-mail clients as a single file, for example, that can be: embedded in a Web page, embedded in an e-mail attachment, stored in ROM, streamed from a server, run as a MIME type, run as an ActiveX component, run as a plug-in, and/or run as an ActiveX component.
[0108] Most story enabled devices will run or play a story in a window, or in a non-windowed operating environment such as occur on in basic or thin client devices, on a display device screen. Such devices include, for example, a desktop computer, notebook computer, personal data assistant (PDAs), telephone, set-top box, movie marquee, informational kiosk, Internet e-mail appliances, billboard, microwave oven, point-of-sale displays, gasoline pump, vending machine, instructional appliance, automobile display device, global positioning system (GPS), point-of-sale display, and myriad of other device types are supported. In fact, a story can even be played on a client device that is not multimedia enabled because preferred embodiments of the inventive story always have a set of text that describes, or narrates any non-textual element of the story, along with semantic information describing the role of each logical element. In one embodiment, a device may play a story entirely with voice commands and automatically articulated responses.
[0109] It is noted that although applicant describes embodiments of the inventive structure, method, computer program, operating model, and structure and organization of content used in or in conjunction with other aspects of the invention, the underlying inventive concept and indeed many embodiments of the invention do not require all features described here. Many such structures and procedures though advantageous and desirable are optional. Including text behind each logical element of the story is a preferred embodiment. Therefore, with respect to the structure and content of a story described here, it should be understood for example, that not all stories must contain underlying text behind each logical element of the story.
[0110] These optimizations make a story very flexible, scalable, and powerful. Unlike some conventional systems and methods, a story maintains a focus on the intent of the message and preserves that message intent in spite of its ability to selectively communicate elements to client devices and users.
[0111] For example, in conventional video streaming systems the primary goal has been to maintain real-time transmission of the video stream and to relax quality to the point where almost all picture quality has been lost if necessary to maintain continuous operation. For an advertiser promoting a high-end product, such as example a diamond ring, it is very important to maintain the quality and clarity of the product image. If the transmitted image(s) of the diamond ring make the ring appear undesirable, the entire purpose for the advertisement is lost. Therefore, attempts should be made to customize composition of the message so that where possible the bright high-resolution image of the diamond ring is presented to the receiver, and if such presentation is not possible then to provide an alternative possibly textual description of the ring which creates the same desire to own product as the bright clear image would. This particular example really illustrates the notion of selecting or substituting content to maintain the intent all of the StoryMail.TM. message independent of the device hardware capabilities or network connection characteristics and even to some extent independently of user preferences.
[0112] The inventive structure and method may be applied to on-line auctions as well and provide significant benefits here. For example, a story message provides rich product descriptions complete with BID forms; bi-d limit exceed notifications providing a bi-dder a chance to upgrade a bi-d from a form embedded in the message without requiring the bi-dder to go to the action web site; and, bi-d accepted notification with transaction completion automation.
[0113] Traditionally, on-line auctions require composing a product description that may not scale up and down depending on the device. Traditional on-line auctions typically require repeated visits the site to determine if a bi-d is accepted. Furthermore, traditional on-line auctions generally require further visits to a Web site or the placement of a phone call to complete a transaction.
[0114] It can be appreciated that stories can be used at point of sale to provide looping demonstrations and/or advertisements of a product. For example, a story can be embedded in read-only-memory (ROM) of microwaves, stereos, set top boxes, and the like. Playback of such a story can be in the store that displays the story 180 enabled product for sale. The manner in which the story is played back may be modified by each viewer according to view preferences. For example the underlying content may have English, French, Spanish, and Russian audio and text content that may be selected by the viewer. Such input may be buttons on the playback device, a touch screen device, voice input, or other input devices as are known in the art. Additionally, story enabled devices, for example, soda machines, can be implemented to play media rich advertisement stories that can be updated using only a phone line to upload a different story. The content of such story may be communicated, for example overnight to a large variety of different device types, yet will be playable by all such device types.
[0115] There are other exemplary applications for stories, for example, stories can also be used for meeting scheduling, advertising, catalog item descriptions, holiday greeting cards, electronic storybooks, driving directions, vacation slide and picture shows, surveys, real-estate walk throughs, medical care pamphlets, pharmaceutical information pamphlets, cooking or production recipes, business presentations, instructional manuals, entertainment, and numerous other applications where the message consists of more than merely the text message.
[0116] We now describe aspects of an inventive next generation e-mail system that is used to generate, distribute, and play stories. In one embodiment, a story that is sent as a message from a server to a client device is called StoryMail. Referring to FIG. 1, there is a block diagram that illustrates aspects of an exemplary embodiment of a StoryMail system 300. StoryMail System 300 (also referred to simply as system 300) is a distributed client/server system with server peering.
[0117] Sender/publisher 310 is connected across I/O interface 312 to user interface 314. Sender/publisher 310, for example, can be a general-purpose computer, provides at least a subset of the information and content used to generate and transmit a story to sending story server 302. In other words, parts of a story may reside on any server anywhere or computer that can be addressed, that is connected to network 306. In this case, sender/publisher 310 provides links, for example, a Uniform Reserve Locator (URL) address of the document or other resource to be included in the story. Sender/publisher 310 includes a number of components which are described in greater detail below in reference to FIG. 2.
[0118] I/O interface 312 can be any type of I/O interface, for example, a peripheral component interconnect (PCI) bus interface, a SCSI interface, or the like. Sender/publisher 310 is also connected across I/O interface 308 to network 306. As an alternative to 312, I/O interfaces 308 and 309
can be used if information is passed through network 306. I/O interfaces 308 and 309 can be any type of I/O interface, for example, a modem connected to a public telephone network, a leased line, or a wireless radio wave or optical interface. Network 306, for example, can be a local area network (LAN) or a wide area network (WAN).
[0119] Network 306 is connected across I/O interface 304 to sending story server 302. Sending story server 302, for example, is a general-purpose computer or device for generating and transmitting stories to client devices, such as conventional e-mail server 332, story enabled client 336, conventional e-mail client 340, and story enabled device 344. A greater detailed description including aspects of an exemplary embodiment of sending story server 302 is provided below in reference to FIG. 4. I/O interfaces 304, 308, 309, 324, 326, 330, 334, 338, and 342 can be any type of I/O interface, for example, a modem connected to a public telephone network, a leased line, or a wireless radio wave interface.
[0120] In one embodiment, the system of the invention includes receiving story server 328, for example, is a general-purpose computer or device for transmitting stories to client devices, such as those client devices listed above. One difference between receiving story server 328 and sending story server 302, for example, is that sending story server 302
is able to generate stories and distribute stories, whereas receiving story server 328 is not able to generate stories but is able to distribute already generated stories. Receiving story server 328 is beneficial because it may contain functionality which can be used to eliminate the need for providing that same functionality in story enabled clients 336 and story enabled devices 344. This is advantageous because the computation and/or memory capacity of such devices is normally more limited than that of the servers 328. In addition, since there are likely to be many more story enabled clients 336 and story enabled devices 344, the implementation costs are lower if the functionality is contained on the servers 328 rather than on the story enabled clients 336 and story enabled devices 344. Examples of such functionality include proxy server functions, placing stories into in-boxes, and security features such as decryption, authentication and digital signature verification.
[0121] In one embodiment, network 306 is connected to conventional e-mail server 332 which is a traditional e-mail server used by a number of machines connected to network 306 to distribute and collect e-mail messages. Procedures for a machine to distribute and collect e-mail messages are known in the art. Conventional e-mail server 332 provides story messages to both non-story enabled devices, for example, conventional e-mail client 340, as well as story enabled clients and devices, for example, story enabled client 336 and story enabled device 344. As will be described in greater detail below, the presence of conventional e-mail server 332 is not necessary for story enabled client 336 or story enabled device 344 to receive stories However, the presence of conventional e-mail server 332 is necessary for conventional e-mail client 340 to receive a story enabled message. In one embodiment, a story enabled message will not include a story, but rather includes information indicating that a richer message, or story underlies the story enabled message. This embodiment is described in greater detail below in reference to FIG. 6 and FIG. 7.
[0122] Story enabled client 336 includes, for example, computer program applications and data for playing a story received from a story server, for example, sending story server 302 and/or receiving story server 328
Story enabled client 336 is, for example, a general-purpose computer, a notebook computer, a personal digital assistant, a telephone, a set-top box, an Internet e-mail appliance, a movie marquee, an informational kiosk, a billboard, a gasoline pump, a vending machine, an instructional appliance, an automobile display device, a GPS system, a point-of-sale display, and the like. Story enabled client 336 starts life as a conventional email client 340. It becomes story email client 336 when story enabling software is downloaded or installed from a network or direct connection to another device. Story device 344 has the story enabling software built in by the manufacturer.
[0123] Conventional e-mail client 340 is a typical e-mail client, for example, a general-purpose computer that is not able to execute, or play a story. However, conventional e-mail client 340 is able to receive e-mail messages that include information indicating that a richer content message, or story is behind the e-mail message. In one embodiment, besides including information that a story underlies the e-mail message, the e-mail also includes, for example, an e-mail message that delivers the publisher's 310 message in a traditional e-mail format. Such traditional e-mail formats include, for example, text, HTML and/or attachments. Such an embodiment is advantageous for a number of reasons. For example, while conventional e-mail client 340 will not be able to play a story without upgrading its computer program applications, it will still receive content that corresponds to publisher's 310 message or promotion. Additionally, the message can be forwarded to another e-mail client device, for example, story enabled client 336, wherein the richer message will be available to the other client device
[0124] In one embodiment, conventional e-mail client 340 upgrades its capabilities to enable it to play a story. In a situation where conventional e-mail client 340 upgrades its computer program applications to enable it to play a story, conventional e-mail client 340 would become a story enabled client 336. In one embodiment, conventional e-mail client 340 can perform such upgrades, for example, by downloading a story player from a web site or an FTP site, or by loading a story player from a CD-ROM or diskette. In a preferred embodiment, conventional email client 340 upgrades by responding to a link provided in the email message, wherein the link points to a download image or site.
[0125] Story enabled device 344 is manufactured with story functionality built in. Such devices include networked household appliances, cell phones, smart cards, and pagers.
[0126] Each client device 336, 340, and 344 includes, for example, an e-mail program (not shown) that respectively receives and/or delivers e-mail respectively from/to one machine connected to network 306 from/to another machine connected to network 306. To facilitate such reception and delivery, an email program utilizes Internet email protocols, for example, known POP3 or IMAP protocols. In one embodiment, such an e-mail program is a conventional e-mail program, such as Microsoft Outlook Express.RTM.. In another embodiment, the e-mail program is a special e-mail program designed specifically to receive and/or transmit stories to another client or device across network 306.
[0127] Referring to FIG. 2, there is a block diagram that illustrates aspects of an exemplary sender/publisher 310, according to one embodiment of the present invention. Sender/publisher 310 includes processor 142
connected across local bus 144 to memory 146. Processor 142 is used to execute computer program applications 148 and fetch data 150 from memory 146. Local bus 144 can be any type of bus, for example a peripheral component interconnect (PCI) bus, as long as local bus 144 has a set of signal lines that can be used by processor 142 to transfer information respectively to and from memory 146.
[0128] Data 150 includes, for example, database 152 representing any combinations of textual information, motion video, audio, forms, automation scripts, a story recipient list and any other message content, communication, or the like, that may be sent in an electronic format. A form can be any type of form or document, for example, a purchase order form, a registration or an application form. Typically a form provides an inquiry and provides some instructions for answering or responding to the inquiry. Database 152 is a standard database that can be created and managed using any of a number of conventional database tools.
[0129] In one embodiment, database 152 includes, for example, textual descriptions in more than one language of a number of products, digital or binary images of the products, motion videos to advertise and illustrate the products, product identification numbers, audio clips to advertise and describe the products, and/or recipient information, such as a list of e-mail addresses to which to send a story. Desirably, for every non-textual item of data in database 152, a textual description of that item of data is available. For example, if database 152 includes a color photo of a particular toy, there will be a corresponding text description of that toy.
[0130] In a preferred embodiment, a digital or binary image can have a set of scaled and color depth versions of the binary image. For example, if database 152 includes a 300 dots per inch (dpi) 24-bit color binary image of the cover of a book, database 152 will also include a 1-bit black and white representation of the image, an 8-bit and 16-bit gray scale representation of the image, and various resolutions of each of the resolutions, such as 100 bit and 200 bit resolutions.
[0131] In a preferred embodiment, scaling of logical story elements can occur at three different times: (1) when generating the message; (2) when executing the procedural elements of the message; and, (3) while the message elements are being rendered by the hardware specific functions (e.g., the HAL functions) that connect a portable story playback engine to actual device specific hardware.
[0132] For example, in one preferred embodiment, sending story server (see FIG. 1) scales the story content when generating the message to conform to the story enabled clients' 336 hardware capabilities, network connection characteristics, and specified user preferences at the time that such information are determined (see FIG. 7, step 228). In yet another preferred embodiment, story player 194 (see FIG. 5) scales the content of the story when the procedural elements of the story are executed, or played. For example, a digital image may be scaled from 300
dpi to 200 dpi while the digital image is being displayed. In yet another embodiment, story player's 194 HAL may scale the story to fit into a particular display screen size and/or add scroll bars to the display so that an entire story can be viewed.
[0133] Document 154 is author once information created by using a number of structured document languages, for example, extensible markup language (XML), and Excel spreadsheet format, database records extracted with SQL, and the like. In a preferred embodiment, Document 154 is an XML document. Document 154 can be created in a number of different ways. For example, Document 154 can be created using any of a number of known XML Editors, Word processors, device drivers, and the like.
[0134] Referring to FIG. 3, there is a block diagram that illustrates aspects of an exemplary Document 154 used by sending story server 302
(see FIG. 1) to generate a message/promotional story 180, according to one embodiment of the invention. FIG. 3 uses a structured document syntax pseudocode that does not conform to any one particular structured document syntax, but is rather used only for purposes of illustrating the invention. In a preferred embodiment, XML document 154 includes a tag that identifies a particular storyteller 172 (see FIG. 4) and a unique identifying attribute of the particular storyteller 172.
[0135] The pseudocode describes a set of tags that each respectively in turn describes an element, wherein each tag is followed by an equals sign ("=") and a corresponding textual description that defines some other property of the element. The property can be either an absolute description string, an embedded document, or a string that includes a URL and a document name. If a descriptive property is a URL and document name, the URL will be accessed and the identified document downloaded when document 154 is parsed by story server 302 (see FIG. 4) during one time processing of document 154, as described in greater detail below in reference to FIG. 4.
[0136] Line 400 includes a tag that identifies a "STORYTELLER ID" element, which is followed by an attribute of the element, "ecoupon 5". "Ecoupon 5" identifies a unique storyteller 172 (see FIG. 4) in story server 302
(see FIG. 1). In this example, ecoupon 5 storyteller 172 will be used to generate a form and a user interface to be used by a sender/publisher 310
(see FIG. 1) to generate and distribute one or more ecoupon stories 180
(see FIG. 4) to distribute to one or more customers as dictated by sender/publisher 310 (see FIG. 1). Storytellers 172 are described in greater detail below in reference to FIG. 4.
[0137] Line 402 includes a tag that identifies a "PRODUCT VIDEO" element, which is followed by an attribute of the element that identifies a particular MPEG motion video, "BOOKRETAILER.COM.backslash.PROMO24.backsla- sh.ISBN12980.MPG" that is to be distributed in a story 180 (see FIG. 4). In this example, the motion video is identified by a URL link to the author's database 152 (see FIG. 2) and a corresponding motion video document.
[0138] Lines 404 and 406 include tags that identify respective product picture elements, wherein each respective tag identifies a specific binary image (or other digital image or graphic) that has a respective different pixel resolution. For example, line 404 includes a tag that identifies a "PRODUCT PICTURE 100DPI" element, which is followed by an attribute of the element that identifies a 100 dpi binary image, such as the JPEG image "BOOKRETAILER.COM.backslash.PROMO24.backslash.ISBNL2980
100DPI.JPG". Whereas, line 406 includes a tag that identifies a "PRODUCT PICTURE 200DPI" element, which is followed by an attribute of the element that identifies a 200 dpi binary image, such as the JPEG image "BOOKRETAILER.COM.backslash.PROMO24.backslash.ISBNL2980 200DPI.JPG". Both binary image files are identified by respective URL links to the author's database 152 (see FIG. 2) and a corresponding JPEG document.
[0139] Lines 408 and 410 include tags that identify respective audio file elements, wherein each respective tag identifies a specific audio file that is implemented in a different language. In particular, line 408
includes a tag that identifies a "PRODUCT AUDIO ENGLISH" element, which is followed by an attribute of the element that identifies an audio file that is implemented in English ("BOOKRETAILER.COM.backslash.PROMO24.backs- lash.ISBNL2980 ENG.WAV"). Whereas, line 410 includes a tag that identifies a "PRODUCT AUDIO SPANISH" element, which is followed by an attribute of the element that identifies an audio file that is implemented in Spanish ("BOOKRETAILER.COM.backslash.PROMO24.backslash.ISBNL2980 SPAN.WAV"). Both audio files are identified by respective URL links to the author's database 152 (see FIG. 2) and a corresponding WAV document. These tags are merely illustrative and not exhaustive of the type of tags, file elements, and/or identifiers that may be used.
[0140] Lines 412 through 418 include tags that identify respective text file elements, wherein each respective tag identifies a specific text file with analogous intent written in a different language. In particular, line 412 includes a tag that identifies a "PRODUCT TEXT ENGLISH" element, which is followed by an attribute of the element that identifies an ASCII text file that is implemented in English ("BOOKRETAILER.COM.backslash.PROMO24.backslash.ISBNL2980 ENG.TXT").
[0141] Whereas, line 414 includes a tag that identifies a "PRODUCT TEXT MANDARIN" element, which is followed by an attribute of the element that identifies a unicode text file that is written in Mandarin ("BOOKRETAILER.COM.backslash.PROMO24ISBNL2980 MANDARIN.UNI") and the like. Each text file of these examples is identified by respective URL links to the authors database 152 and a corresponding text or unicode document.
[0142] Line 420 includes a tag that identifies a respective "PRODUCT SKU" (stocking unit) number element, which is followed by an attribute of the element, in particular an absolute value that identifies the promotion's targeted product's SKU. Line 422 includes a tag that identifies a respective "FULFILLMENT SERVER URL" element, which is followed by an attribute of the element, in particular a URL for the promotion's fulfillment server. A procedure for using such a fulfillment server is described in greater detail below in reference to FIG. 7.
[0143] Lines 424-428 includes tags that identify story 180 (see FIG. 4) recipient or customer information. For example, Line 424 includes a tag that identifies a "FIRST NAME" element, which is followed by an attribute of the element, in particular, the name "DAVE". Line 426 includes a tag that identifies an "EMAIL ADDRESS" element, which is followed by an attribute of the element, in particular an e-mail address, such as for example to "someone @ somewhere.com" that identifies the recipient's e-mail address, and the like.
[0144] Line 430 includes a tag that identifies a respective "MASTERDATABASE ID" that is used by sending story server 302 (see FIG. 1) to identify those portions of a master parts database to use for a particular message/promotion. In one embodiment of the invention, sending story server 302 returns the message/promotion ID 430 to sender/publisher 310 (see FIG. 1), such that the message/promotion ID 430 is unique to any other message/promotion IDs in a master parts database. Such a message/promotion ID can be used by publisher 310 to modify and/or delete the information that corresponds to a message/promotion in a corresponding master parts database. Such a master parts database is described in greater detail below in reference to FIG. 4. In one embodiment, such a message/promotion ID is used by publisher 310 to send a corresponding message/promotion to recipients in batches, each batch job referencing the message/promotion ID.
[0145] It can be appreciated that document 154 can include any number of user defined elements and respective attributes of such defined elements. As will be discussed in greater detail below, recipient information, for example, that information illustrated in lines 424428, can be supplied to sending story server 302 (see FIG. 1 and FIG. 4) at any time through a number of different mechanisms.
[0146] In a preferred embodiment, for at least a subset of the non-textual data in Document 154, a textual description of that non-textual data is identified in Document 154. In yet another embodiment, for every textual description, there is a corresponding text description identified in more than one language, for example, English and Spanish text descriptions. In yet another embodiment, if Document 154 identifies an audio file in a particular language, Document 154 also identifies other audio files that have analogous content to the audio file in different languages. It may also provide a textual transcription and/or a summary of the audio files for presentation when the receiving device does not provide audio playback or the recipient chooses not to receive the content in an audio format. In yet another embodiment, if document 154 includes a binary image (either embedded or via a URL) having a particular resolution, document 154 also includes other resolutions of the binary image. Including such multiple resolutions of a binary image is beneficial for the reasons discussed in greater detail above. Furthermore, not only may the binary or digital images be different resolution, they may be different types of files, such as for example, a bit-mapped image (*.bmp), a TIFF format image (*.tif), a JPEG compressed image (*.jpg), or the like.
[0147] Applications 148 includes, for example, one or more of the following computer program applications (a) a Web browser (not shown) such as Netscape Navigator.RTM. or Microsoft Internet Explorer.RTM., for accessing a Web page served from sending story server 302; (b) any of a number of commercially available XML Editors for creating document 154. Other applications may also be stored or provided, for example, multimedia authoring systems, story mail applications, templates for other applications such as spreadsheets, multimedia and/or XML database managers.
[0148] Sender/publisher 310 also includes, for example, a database stored or referenced which includes at least a subset of the content necessary to represent the information and data in a story.
[0149] Referring to FIG. 4, there is a block diagram that illustrates aspects of an exemplary sending story server 302, according to one embodiment of the invention. Server 302, includes processor 162 connected across local bus 164 to memory 166. Processor 162 is used to execute computer program applications 168 and fetch information from data 170. Local bus 164 can be any type of bus, for example, a peripheral component interconnect (PCI) bus, as long as local bus 164 has a set of signal lines that can be used by processor 162 to transfer information respectfully to and from memory 166.
[0150] There may be any number of sending story servers 302 and receiving story servers 328 (see FIG. 1). In such a system 300, each server 302 and 328 will respectively communicate directly with another respective server 302 and 328, or with one or more conventional e-mail servers 332 (see FIG. 1) using one or more communication protocols, for example, SMTP/ESMTP/MIME/HTTP communication protocols. (For purposes of this description, wherever SMTP is used, ESMTP is also applicable). Sending story server 302, using information that is provided both by sender 302
and story enabled client 336, generates and distributes stories 180 as e-mail, or StoryMail. Such information can be provided to sending story server 302 through a number of different mechanisms. For example, the information may be provided if sender/publisher 310 (see FIG. 1) sends document 154 across I/O interface 308 to server 302. (The contents of document 154 are described in greater detail above).
[0151] In one embodiment, sending story server 302 also serves one or more documents on the World Wide Web (WWW) identified by a unique Uniform Resource Locator (URL) that allows a user of sender 302 to input information through network 306 into server 302 that will be translated into document 154. There are a number of known computer programs that are used to translate information into a structured file format, for example, XML. Aspects of an exemplary procedure used by sending story server 302, sender/publisher 310, and story enabled client 336 to exchange information to generate, distribute and play story 180 are described in greater detail below in reference to FIG. 5 and FIG. 6.
[0152] Applications 168 includes, for example, composition engine 170, storyteller 172, e-mail engine 173, and other applications 174. Each of these applications 168, and in particular, composition engine 170, storyteller 172, and e-mail engine 173 work cooperatively to build story 180. Composition engine 170 provides, for example, a framework of data structures, a run-time model, a compiler, an application programming interface (API), and conventions for building an almost endless variety of different stories 180 that conform to a story run-time model. The story run-time model is designed such that a story playback engine on a story client can be simple in complexity and fast. The run time model provides a lightweight cooperative multitasking multimedia and central application framework. (Such a run-time model described in greater detail below).
[0153] Composition engine 170 passes information provided by sender/publisher 310 (see FIG. 1), such that the information is represented in a procedural data format that is not a flat data format. Advantageously the technologies are designed for the procedural content to be fully computer-generated, that is, without manual user intervention. (Manual building is possible but it is not preferred or even desirable.) In one embodiment of the invention, industry standard XML interfaces are used to completely automate one time processing of such provided information, such that existing authoring tools and content formats, for example, JPEG, AVI, MPEG, MP3, and the like, are supported through a simple yet powerful transcoding mechanism of the invention.
[0154] To accomplish this, composition engine 170 performs one-time processing of the provided information such that the resulting procedural format of the information for example, is a sequenced set of data, for example, computer program instructions or operation codes (op codes), control information, parameters and media parts. The phrase "sequenced set" means that the data is organized into a time line that dictates the rendering and navigational characteristics of a story 180. This time line may include procedural tests, branches, jumps, conditional statements, and the like so that the rendering may not ultimately be perfectly linear or sequential.
[0155] For example, such a sequenced set of data may include a first set of computer program instructions to display a graphic. The first set of computer program instructions is followed, for example, data used by a story player to display navigational buttons on the story receiving devices display. Desirably, each media part is assigned an absolute priority that controls when and if a particular media part will be rendered.
[0156] The computer program instructions specify operations to render graphical user interface (GUI) components, media parts, and provide procedural control to user interaction with the GUI components. The control information, for example, provides offsets into the sequenced set of data that indicate where particular media parts are located. In one embodiment, control information also provides a set of semantics and flags for each logical element of a story to maintain the intent of the message on all receiving devices.
[0157] In yet another embodiment, control information, for example, includes an array of hot spots, one hot spot for every logical element. Such logical elements include, for example, button controls, text input controls, bitmaps, areas wherein motion video will be displayed, text boxes, decorative elements, and the like Each hot spot is associated with a rectangular region of the receiving devices' screen display (if one is available). The rectangular region facilitates event identification. Such event identification is associated with user instantiated events. For example, if a user selects, for example, with a mouse device, a region identified by the rectangle associated with a particular hotspot, the operating system will generate a button click event which, as will be described in greater detail below is processed by a story player in the context of the logical element selected.
[0158] Each hot spot is further identified as being either active or inactive. An active hotspot is a hotspot that generates an event when a user selects a region within the rectangular area associated with the hotspot. In contrast, an inactive hotspot does not generate an event when a user selects a region within the rectangular area.
[0159] In a preferred embodiment, each hotspot area is implemented as a bitmap. Aspects of an exemplary procedure for a story player to use an array of hot spots to play a story is described in greater detail below in reference to FIG. 6.
[0160] In addition to areas the hotspot array may also contain semantic and alternative rendering element identifiers (ids) for logical elements other than areas. For example, a hotspot's semantic flags may indicate that there is overview test available that describes the overall purpose of a screen of information, and the hotspot may also contain the id of the overview text element of the story.
[0161] Aspects of control and control information include memory buffer creation, memory buffer loading, branching, condition or searching, layout, subroutines, linkage between different sequences of instructions, decompression and compression and file packaging, e-mail access for sending messages, requests for subfiles.
[0162] In one embodiment, each opcode, parameter and offset is a 32-bit word. This is beneficial for a number of reasons. For example, portability and adaptability are supported by the use of fixed size 32-bit words. A 32-bit fixed size word is advantageously used for representing a large dynamic range of value and is highly compressible because both instructions and parameters are designed to have mostly small integer values. The fixed size makes things very scalable and processor words are always aligned along the word boundary.
[0163] Because of this suitably chosen fixed size, the playback code, or the story 180 is also small and reusable. Parameters and opcodes can be processed by the same code and operation, for example, addition operations can be performed without the need for size conversion of the code. An additional advantage is that the opcodes and data are aligned in memory for fast access. Aspects of an exemplary procedure to use such a procedural data layout to play story 180 are described in greater detail below in reference to FIG. 5 and FIG. 6.
[0164] Such one-time processed information is stored by composition engine 170 as a set of master parts data into master parts database 178. Desirably, each set of master parts data is identified by a unique identifier that can later be used by sender/publisher 310 to access, modify, and delete the contents of a particular set of master parts data. in master parts database 178. The set of master parts data can be used by sender/publisher 310 (see FIG. 1 and FIG. 2) to generate and distribute any number of stories 180 to targeted e-mail enabled clients.
[0165] In one embodiment, composition engine 170 is eminently portable, meaning that it may also be embedded in other devices besides sending story server 302. For example, composition engine 170 may be embedded, for example, into a digital camera. A single global data structure allows the implementation of composition engine 170 code as a set of C++ objects, composition engine 170 code is reusable and can be instantiated more than one time. An additional advantage of this is that applications including composition engine 170 will be easy to build. Furthermore sizes of all program variables are explicitly defined and there is built-in support for little-endian and big-endian systems. A thin hardware extraction layer (HAL) and the ability for all text to be represented in ASCII or Unicode also supports portability. In combination, all of these aspects make a story quickly and easily portable to a broad range of devices, able to handle nearly all the computer programming instruction sets or languages.
[0166] Story teller 172 includes, for example, a set of programmed logic that will select at least a subset of a particular set of master parts data in master parts database 178 to build story 180. Because composition engine 170 represents the provided information in a procedural format, a story 180 is just one big procedural language/data/environment. In a preferred embodiment, a story 180 is part of the same procedural language including the content, decompression, rendering, layout, hotspot responses and navigation. In some aspects, a story 180 may be viewed as a self-contained ultra-low overhead multi-threaded run-time system. For example, a story 180 generates video frames by executing sequences of instructions. This allows for mixing of different video decompression/reconstruction algorithms within a single frame. For example, a motion compensation vector equivalent for a whole frame can be applied using a single instruction which moves rectangular parts of one picture into another.
[0167] In one embodiment, storyteller 172 builds a story 180 from the master parts database 178 in response to a message from StoryMail enabled client 336 (see FIGS. 1 and 4). (Such a message is described in greater detail below in reference to FIGS. 5 and 6). In this embodiment, the message will include a unique identifier, such as the unique identifier discussed above, to determine which set of master parts data to use to build a story. The particular master parts that a storyteller 172 will select to piece together story 180 together depends on the purpose of storyteller 172 and the particular hardware capabilities, network connection characteristics, and user preferences associated with a targeted story enabled client 336 (see FIG. 1 and FIG. 4). Aspects of an exemplary procedure to send server 302 such capabilities, characteristics, and preferences are described in greater detail below in reference to FIG. 5 and FIG. 6.
[0168] The purpose of storyteller 172 can include any one of the exemplary applications of a story 180 that were discussed in greater detail above or other purposes. In one embodiment, sending story server 302 includes any number of pre-configured storytellers 172, wherein each storyteller 172 will have a unique such purpose. For example, a first storyteller 172-1 may be used to build an e-coupon story 180, a second storyteller 172-2 may be used to build a parts catalog story 180, and the like.
[0169] In yet another embodiment, the invention contemplates that sending story server 302 will serve a Web page interface (not shown) whereby publisher/sender 310 creates and modifies storytellers 172. For example, in one embodiment, such a Web interface provides a set of button controls that when selected by a user allows the user to: (1) add logical story elements, for example, an MPEG file, to master parts database 178; (2) select portions of such logical story elements, for example, a user selects a particular picture and a particular video to include in a story 180; (3) specify the dimensions of portions of the story, for example, a user may specify that the dimensions of a particular sequence of logical story elements are to be of a particular width and height; (4) order the logical story elements on a time line, and take into consideration any use