Electronic Proceedings of the
ACM Workshop on Effective Abstractions in Multimedia
November 4, 1995
San Francisco, California

Synchronization Aspects of an Hypermedia Presentation Model with Composite Nodes

Guido Lemos de Souza Filho
DIMAp, UFRN
Campus Universitário, Lagoa Nova
59072-970- Natal-RN, Brasil
(055)(021)610-2690
guido@dimap.ufrn.br

Luiz Fernando Gomes Soares
Depto. de Informática, PUC-Rio
R. Marquês de São Vicente 225
22453-900 - Rio de Janeiro, Brasil
(055)(021)529-9523
lfgs@inf.puc-rio.br
http://www.inf.puc-rio.br/staff/professores/lfgs.html

Marco Antonio Casanova
Centro Científico Rio, IBM Brasil
Caixa Postal 4524
20001-930 - Rio de Janeiro, Brasil
(055)(021)271-2334
casanova@vnet.ibm.com

ACM Copyright Notice


Abstract

This paper outlines the synchronization aspects of a hypermedia presentation model, called Nested Context Presentation Model (NCPM), which has three major features. First, it allows authors to specify temporal synchronization relationships among events within hypermedia documents using the usual concepts of anchors and links. Second, it allows asynchronous elements, such as user interaction, to be combined with synchronous elements, like video exhibitions, in a single hypermedia document. Third, it separates the presentation specification from the structural definition of the hypermedia document. The synchronization facilities of NCPM can be applied to any hypermedia model with composite nodes and, with minor simplifications, even to those conceptual models without composites. In particular, the synchronization facilities can be applied to those models that can be translated to MHEG objects.

Table of Contents


Introduction

Structural hypermedia conceptual models treat documents as essentially passive data structures. However, hypermedia systems must provide tools for the user to access, display, manipulate and navigate through the structure of the documents. This functionality is captured by presentation models.

This paper outlines the synchronization aspects of a hypermedia presentation model that has three major features:

  1. It allows authors to specify temporal synchronization relationships among events within hypermedia documents using the usual concepts of anchors and links;
  2. It allows asynchronous elements, such as user interaction, to be combined with synchronous elements, like video exhibitions, in a single hypermedia document; and
  3. It separates the presentation specification from the structural definition of the hypermedia document.

The discussion is hear phrased in terms of the Nested Context Model (NCM), but the major ideas apply to any hypermedia conceptual model offering nested composite nodes. NCM is the conceptual model of HyperProp, a hypermedia prototype system that provides an environment for the construction of hypermedia applications. HyperProp is in fact organized as a library of classes that reflect the concepts of NCM.

< -- Table of Contents

Nested Contex Model

The class hierarchy for the NCM [SoCR94] [SoCR95], wich a basic conceptual model for hypermedia documents that supports composition is illustrated in Figure 1.


Figure 1. Class Hierarchy for a Basic Conceptual Model

An entity is an object which has as attributes at least a unique identifier, an access control list and an entity descriptor. For each entity attribute, the access control list has an entry that associates a user or user group to his access rights for the attribute. The entity descriptor contains information determining how the entity should be presented to the user.

A node is an entity which has as additional attributes an anchor list and a content. A terminal node is a node whose content and anchor list are both application dependent. The model allows the class of terminal nodes to be specialized into other classes (text, audio, image, etc.). A composite node C is a node whose content is a set L of nodes and links. We say that an entity E in L is a component of C and that E is contained in C. We also say that an entity A is recursively contained in B iff A is contained in B or A is contained in an entity recursively contained in B.

In a node, each element of the anchor list is called an anchor of the node and is an entity which has as additional attribute a region. A region of a terminal node is either the special symbol ß, representing the entire node content, or a set of marked information units. The exact notion of information unit and marked information unit is part of the definition of the terminal node. For example, an information unit of a video node could be a frame, while a information unit of a text node could be a character. For a composite node C, a region is either the special symbol ß or a subset of the content of C containing only nodes (ß again represents the entire node content). Any subset of the information units of a node may be marked.

The anchor list acts as the external interface of a terminal node N, in the sense that an entity can access regions of the content of N only through the anchor list of N. Hence, anchors shield other entities from changes to the content of N. As an example, consider a text node with just one anchor whose region points to the second paragraph of the text. Assume that a link indirectly refers to the second paragraph of the text by identifying the appropriate anchor. Then, any changes to the text must be reflected only in the anchor region and do not affect the link.

A link is an entity with two additional attributes, the source end point set and the destination or target end point set. The values of these attributes are sets whose elements, called end points of the link, are pairs of the form <(Nk,...,N1),A> such that N1 is a node, Ni+1 is a composite node and Ni is contained in Ni+1, for all i ([1,k), with k>0, and A is an anchor of N1. The node Nk is called a base node of the link. We also require that, for every link l contained in a composite node C, for every end point <(Nk,...,N1),A> of l , node Nk must be either C or a node contained in C.

< -- Table of Contents

The Presentation Model: Synchronization Aspects

As in the previous section, we will discuss the presentation model through a description of its classes and their functionality. The class hierarchy of the NCPM is shown in Figure 2.


Figure 2. Class Hierarchy of NCPM.

Descriptor Objects

In the Nested Context Presentation Model (NCPM ) [SoSC95], the user's interaction with a hyperdocument is modeled in conformance to the Dexter Model (DM). The fundamental concept in the runtime layer of DM is the instantiation: a presentation of the component to the user. The instantiation of a node also results in instantiation of its anchors. An instantiation in NCPM is called a representation object, discussed in detail in Representation Objects. Representation objects have descriptors among their components, the topic of this section.

A descriptor consists of a collection of event descriptions and a start-up specification. The event concept adopted in the NCPM is based on the similar concept introduced by Buchanan and Zellweger [BuZe92].Each event description contains a region description, an operation list and a specification of the event type.

Each event description may define a synchronization point at which the display of the associated terminal or composite node can be synchronized with events in the same terminal or composite node or in others. An event description may also define a synchronization point at which the behavior of the representation of a terminal or composite node can be altered.

The region associated to an event is defined by an anchor. The anchor then defines the region within the content of the node that is associated with the event. So, the NCPM uses the anchor concept to bind the regions in the content of the nodes with the specification of the document's presentation.

The type of the event is defined by its trigger condition. The events are triggered by the selection of its region, or by the exhibition of its region to the user. The exhibition of the region associated to an event to the user defines a presentation event. The selection by the user of a region defines an interactive event. The presentation events can be synchronous events, whose temporal placement relative to other event are known in advance; or asynchronous events, whose time of occurrence can only be determined at runtime. The interactive events are asynchronous events. Each descriptor has a distinguished synchronous event, marking the beginning of the display of the associated terminal or composite object. The termination of the display of the associated object can be a synchronous event, if we know in advance the duration of presentation of the associated object, or an asynchronous event, otherwise.

Operations lists in events are used to control the display of a document. An operation list consists of an ordered list of operations. An operation is composed by a set of actions and a set of conditions (that must be satisfied in order to trigger the actions). Note that we use the terms action and condiction as defined in MHEG [ Pric93] .

A start-up specification contains information to initiate the presentation of an entity. In particular, they define methods for exhibiting or editing entities, and channels. The methods can be any program and, in particular, any editor. The channels identify the resources (hardware and software) that will be used to present node's content. A start-up specification also has an ordered list of operations that must be executed when a node is selected, before a node begins to be exhibited. A start-up specification must define all parameters needed to create a representation object, as defined in the next section.

< -- The Nested Context Presentation Model

Representation Objects

A representation object is created by the runtime display controller (the equivalent to the Dexter instantiator) from the NCPM objects. It is created from a descriptor, a node, methods and a set of channels. We say that the object thus created is a representation of the identified node. Representation objects are responsible for detecting and signaling the occurrence of events.

The descriptor can be defined in an attribute of the node from which the representation object (a version of the identified node) is created. Links may also have a set of descriptors, as in the Dexter model, which contains information for the presentation model indicating how the representation objects of the referenced nodes should be created. Composite nodes also have, for each of its component nodes, an attribute where a descriptor for creation of the representation object of the component may be defined.

When presenting a node, the descriptor defined explicitly by the user bypasses the descriptor defined by the link used to reach the node; this in turn bypasses the descriptor defined by the composite node that contains the node, which bypasses the descriptor defined within the node. Each node type has a descriptor default, that is used if none of the descriptor presented above was defined.

The methods of the NCPM objects, wich are similar to the Firefly procedures [ BuZe92], operate on the terminal or composite objects and its events and are invoked when the operation lists are processed. The Behavior specification methods implement the interface to create or edit the events in the representation objects . Time analysis methods control the occurence of the events in time.Control methods are used to start, stop, pause and resume the presentation of the nodes, and to change the layout of the presentation, for example,modifying the volume, color etc. Control methods are also used by the system to speed up or slow down the exhibition rate, while the document is being displayed.

In addition, the representation object have specifications of the channels, wich are abstractions of the multimedia resources that, in turn, are used to exhibit the nodes. For example: display, audio device etc. The channels of the NCPM is a simplification of the channels used in the CMIFed [ RJMB93]. The NCPM channels are used specify only the device and the space in the device that will be used to present the content of the nodes, and are then similar to the MHEG virtual channels [ Pric93].

< -- The Nested Context Presentation Model

Reinterpretation of Links

In the presentation model, a link is reinterpreted as a logical assertion about a collection of events. An event occur when the user selects a region associated with an anchor (interactive event), or when a region associated to an anchor is exhibited to the user (presentation event). Besides a set of source end points and a set of destination, or target, end points, a link may also contain two special attributes: one defining a descriptor for each of its target end points, and another, called the meeting point, discussed below.

A hypermedia link or hyperlink is expressed as a relation between a set of events in one or more nodes, represented by the source end points, and one or more events in the same or other nodes, represented by the target end points. At least one of the events of the source end points must be an asynchronous event defined by a user interaction.

A synchronization link or synclink is expressed as a relation between a set of presentation events in one or more nodes, represented by the source end points, and one or more events in the same or other nodes, represented by the target end points.

The relations associated with a link (synclink and hyperlink) are expressed through its meeting point. The meeting point attribute consists of an ordered list of operations, composed by a set of conditions and a set of actions. The conditions are based on the source end points. Whenever a condition is satisfied it triggers the associated actions. The actions of a meeting point may trigger either the execution of operations in the target representation objects or the avaliation of the operation lists of the the target end points by the representation objects that contains this endpoints.

< -- The Nested Context Presentation Model

Script Objects

In the NCPM the user can trigguer the execution of a script in response to the occurrence of an event A script object encapsulates a program whose instructions (operations), when executed, generate messages invoking the activation of valid methods in the NCPM representation objects
< -- The Nested Context Presentation Model
< -- Table of Contents

An Example of a Document Presentation Specifyed with the NCPM

To illustrate the utilization of the NCPM we take, as an example, the document shown in Figure 3. The structure of the document is described in the section structural model. The document has two composite nodes, four terminal nodes (two texts, one graphical animation and one audio) and three links. The NCPM specification of the presentation of the document is listed in the section NCPM specification.


Figure 3. An Exemple of a NCM Document.

In the example we can observe the three major features of the NCPM model. First, as we can see, the NCPM allows the authors to specify temporal synchronization among events of interest (the exhibition or the selection of information units in different nodes) using the NCPM links. Second, the concepts of presentation and interactive events provide the authors with a uniform way to combine the synchronous elements with the asynchronous ones in the specification of the hyperdocument's presentation. Third, the NCPM separates the presentation specification from the structural definition of the document, facilitating the definition of alternative presentations for the same hyperdocument.

Another point, wich can be observed in the example, is that, using the NCPM, in a single presentation specification, the author may define alternative presentations to the components of the hyperdocument. The choice of the alternative presentation that will be used is made in runtime and depends on how the user navigates on the hyperdocument to reach the component. In the example, we can see three distinct presentation formats that can be used to exhibit the node E. The first presentation format is used when the node is reached from the browser showing a map view of the context node C that contains E. In this map, if the user selects the graphical element representing the node E, this node is shown with the format defined by the descriptor desc5. The second presentation format, defined by the descriptor desc6, is adopted if the node E is reached by the navigation of the link l3. The link l3 is traversed when the region associated to the anchor t in the node D is exhibited to the user. The third alternative exhibition of the node E is defined by the descriptor desc7, and is used when the link l1 is traversed. This occurs when the user to select the region associated to the anchor i in the node B. This feature is based in the notion of the perspective used to see a node, wich is better explained in the reference [CaSS95].

< -- Table of Contents

Structural Model of a Document

CONTEXT NODE A:	Nodes: {C, B, F}
		Links: {l1, l2}
		Descriptor: desc1

CONTEXT NODE C:	Nodes: {D, E}
		Links: {l3}
		Descriptor: desc2

TERMINAL NODE B: Content: Graphic-animation
		 Anchors: {i, k}
		 Descriptor: desc3

TERMINAL NODE F: Content: Audio
		 Anchors: {m}
		 Descriptor: desc10

TERMINAL NODE E: Content: Text1
		 Anchors: {j, u}
		 Descriptor: desc11

TERMINAL NODE D: Content: Text2
		 Anchors: {t}
		 Descriptor: desc12

LINK l1:	Source: {<(B), i>}
		Target: {<(C, E), j>}
		Meeting Point: {<(C, E), j> = <(B), i>}

LINK l2:	Source: {<(B), k>, <(F), m>}
		Target: {<(B), ß>, <(F), ß>}
		Meeting Point: {<(B), ß> = <(F), m> and (not <(B), k>),
				<(F), ß> = <(B), k> and (not <(F), m>)

< -- Table of Contents

NCPM Specification

DESCRIPTOR desc1: Start-up: Play-content
		  Events:  {(Einitial | ß | exhibition | play C, B, F parallel)}
		  Link l1: {(<(C, E), j>, Play with desc7) = <(B), i>}
		  Link l2: {(<(B), ß>, spped-up) = <(F), m> and (not <(B), k>),
			    (<(F), ß>, speed-up) = <(B), k> and (not <(F), m>)}

DESCRIPTOR desc2: Start-up:  Browse-content
		  Channels:  chan1
		  Events:    {(E1 | D | selection | play with des4),
			      (E2 | E | selection | play with des5),}
		  Link l3:   {(<(E), u>, Play with desc6) = <(D), t>}

DESCRIPTOR desc3: Start-up: Play-content with Animator-Pro
		  Channels: chan2
		  Events: {(E1 | i | selection | signal l1),
			   (E2 | k | exhibition | signal l2),}

DESCRIPTOR desc4: Start-up: Play-content with Word-for-Windows
		  Channels: chan3
		  Events:   {(E1 | t | exhibition | signal l3)}

DESCRIPTOR desc5: Start-up: Play-content with Word-for-Windows
		  Channels: chan4

DESCRIPTOR desc6: Start-up: Play-content with WordPerfect
		  Channels: chan4

DESCRIPTOR desc7: Start-up: Play-content with synthesizer
		  Channels: chan5

CHANNEL chan1: 	type: display
		position: <50,100,300,400>

CHANNEL chan2: 	type: display
		position: <100,0,500,700>

CHANNEL chan3: 	type: display
		position: <0,0,200,300>

CHANNEL chan4: 	type: display
		position: <400,0,600,300>

CHANNEL chan5: 	type: audio
		volume: 60%
< -- Table of Contents

Conclusions

This paper presented the Nested Context Presentation Model throught the definition of a set of basic objects that can be used to model all the relevant aspects regarding temporal and spatial synchronization in the presentation of a hypermedia document. The idea behind NCPM design was to provide high level abstractions to define synchronism in the document’s presentation, considering the possible use of the syntactic structures provided by the MHEG objects. A full implementation of an environment that uses the NCPM as presentation model is under development as part of the HyperProp project.
< -- Table of Contents

End Notes

[1] We define an event as the presentation of a marked state in a media object or a user interaction with the document as, for example, the selection of an anchor.

Acknowledgements

The work reported in this paper was partially financed by a grant from the HyperProp-PROTEM project of the CNPq-Brazil.
< -- Table of Contents

Bibliography

[BuZe92]
Buchanan, M.C.; Zellweger, P.T. "Specifying Temporal Behavior in Hypermedia Documents", Proceedings of European Conference on Hypertext, ECHT'92. Milano. December 1992.
[Pric93]
Price, R. "MHEG: An Introduction to the future International Standard for Hypermedia Object Interchange", Proceedings ACM Multimedia’93. Anaheim, CA. August 1993.
[RJMB93]
van Rossum, G.; Jansen, J.; Mullender, K.S. and Bulterman D. "CMIFed: A Presentation Environment for Protable hypermedia Documents". Proceedings ACM Multimedia’93. Anaheim, CA. August 1993.
[SoCR94]
Soares, L.F.G.; Casanova, M.A.; Rodriguez, N.L.R. "Nested Composite Nodes and Version Control in Hypermedia Systems" Proceedings of the Workshop on Versioning in Hypertext Systems, in connection with ACM European Conference on Hypermedia Technology, Edinburgh, September, 1994.
[SoCR95]
Soares, L.F.G.; Casanova, M.A.; Rodriguez, N.L.R. "Nested Composite Nodes and Version Control in an Open Hypermedia System" IEEE Transaction on Information Systems; Special issue: Multimedia Information Systems, (Accepted for publication).
[SoSC95]
Souza, G.L.; Soares, L.F.G.; Casanova, M.A. "Synchronization Aspects of a Presentation Model for Hypermedia Documents with Composite Nodes" Research Report Departamento de Informática, PUC-Rio. Rio de Janeiro, Brasil. ., October 1995.
[CaSS95]
Casanova, M.A.; Souza, G.L.; Soares, L.F.G. "Anchors and Links for Composite Nodes" Research Report Departamento de Informática, PUC-Rio. Rio de Janeiro, Brasil. ., October 1995.
< -- Table of Contents