MetaWeb


a general purpose interface to shared interactive applications in the World-Wide Web


position paper
for the Workshop "Programming the Web - a search for APIs"
of the fith international World Wide Web Conference (Paris, May 1996)


Thomas Koch
GMD, March 1996

Introduction

Today the World-Wide Web (WWW) is changing from a distributed hypermedia system towards a general purpose access point to existing applications. However due to the nature of the Web (uses a simple stateless request/response protocol) there are several restrictions to web-based applications that must be overcome in order to build powerful application interfaces in the future.
This paper summarizes the evolution in use of the web during the last years and sketches the current limitations as well as already proposed solutions. Finally it presents an architecture that can be used to build web-based applications without the current restrictions and sketches how an existing web-based application can be extended by using this architecture.

History of the Web

The Web was developed as a "personal hypertext system used for keeping track of personal information on a distributed project" [Berners-Lee94]. Invented in 1990 at CERN the Web introduced two public and simple standards on which it was based : HTTP and HTML. According to this the Web consists of several independent servers and clients. The servers provide information described in the hypertext markup language (HTML). The clients access these information (called web pages) from a server using the hypertext transfer protocol (HTTP). The following figure illustrates the client/server architecture of the Web :

figure 1 - simple client/server architecture of the Web

The success of the Web is based upon several reasons. First most of the software is public domain and clients (called browsers) are available for different platforms. Moreover the Web integrates existing internet services like FTP, News and Gopher. But probably the biggest advantage of the Web is its extensibility. In fact the Web can be used as an interface to existing information systems (or any kind of application).
While in the above situation the client only request for a document stored on the servers local file system, the client is also able to send a set of parameters (usually generated from user input) to the server. In this case the server executes the specified application which uses the passed parameters to generate the requested information. This information (usually represented in the HTML format) is then passed to the client by the server. This situation is summarized in figure 2. The used interface between the web server and the executed application is called Common Gateway Interface (CGI).

figure 2 - the Web as an access point to existing applications

Although this method allows easy access to existing applications there are some limitations to it that follow from the used network protocol HTTP and the page description language HTML. First there is no powerful support for user interface design in HTML. This is not very surprising since HTML has been designed as "a common basic language of interchange for hypertext" [Berners-Lee94] and not as an user interface description language. This is usually not too important, since the existing user input elements (like checkboxes, textfields and buttons) are sufficient for most applications (like database search engine frontends) and HTML is growing currently (new input elements like file browsers have already been proposed).
More important is the drawback that results from the used network protocol : since HTTP is a simple stateless request/response protocol the client will always get a "snapshot" of the actual state of the addressed application. This means that there is no way for the application to tell the client that there has been a change in the application's data. Obviously there are many applications which don't suffer from this, since their data won't change or their normal application interfaces don't face this problem either. An example for this situation could be a user's query to a database : after the query is done another user could change the data, but the first user is aware about this problem and wouldn't expect to be notified about this.
On the other hand there are several applications that can only be used in a very restricted way through a web-based interface. One example is an application that provides on-line stock market data : a user accessing this application will always get the current data, but must perform a new request to the corresponding Web server to get the data updated each time he wants to see the actual values.
Fortunately there are several solutions to this problem. Most of them are based on client-site computation. In contrast to the above situations where all computation is done on server site (the client is only responsible for displaying the result as a Web page), in this situation there is also some computation done on client site. Therefore the client (or the machine the client is running on) executes another application (here called client-site application). Technically this can be an already installed application on the local machine (usually called helper application) or an application that is loaded over the network by the browser as needed (for example a Java script or applet [SUN95]).
This client-site application can then directly connect to the server-site application (using an application specific network protocol), request a synchronous update of the fetched application data and control the client (browser) to display the updated data. This situation is sketched in figure 3.

figure 3 - client-site applications for synchronous update

One of the major drawbacks of this method is that for each server-site application there has to be a special client-site application and an application specific network protocol designed to work with the server-site application. Obviously there is a need for a general purpose interface on client-site as well as on server-site.
The following section introduces such interfaces as part of an event-based architecture.

MetaWeb - bridging the gap between WWW and applications

As seen above there is need for a general architecture that allows applications to be smoothly integrated into the Web. This section presents such an architecture based on the notion of events and sessions. In this approach an event is used to describe any type of information exchange. An event may be application-generated or user-generated. Users as well as applications may specify their interest in special events. Any part of the system that produces or consumes events is designated as an event-module. Each client-site consists of a set of event-modules connected to an event-client. An event-client is responsible for sending and receiving events from the system and delivering them to the affected event-modules. Each event-client is connected to an event-server. An event-server receives events from other event-servers or event-clients and forwards them to the affected event-servers and event-clients. All events are addressed to sessions. The event-server maintains a list of all active sessions and stores al l incoming events in the addressed sessions. All event-clients and event-servers together form a web with event-servers as internal nodes and event-clients as external nodes. This web is placed above the World-Wide Web and is therefore called the MetaWeb.
A simple architecture overview is given in figure 4.

figure 4 - MetaWeb architecture

MetaWeb can be seen as a toolkit for web-based application development. It provides :
Basic components of the MetaWeb architecture are event servers, event clients and event modules - presented in more detail in the following sections.

event servers

Event servers form the backbone of the MetaWeb. They perform the event transmission and storage and maintain consistency between the connected event clients through event ordering. Each event server maintains a set of sessions which are hosted by it. Event servers are connected to other event servers as well as to event clients. Event servers can be mutually connected in order to transport events between different user groups in geographically dispersed settings. This is extremely useful to reduce network traffic and to allow to access locally installed applications (intranet-based) from distant sites - see figure 5 for a short outline.

figure 5 - connecting event servers to bridge wide distances

event clients

The event clients define an interface (API) for event modules to the MetaWeb and are part of the toolkit. All an application designer has to deal with is the development of appropriate event modules. In contrast to common client/server architectures where most of the work is done from the servers, in MetaWeb the clients play a more important role. An event client connects to an event server and registers itself with the name of the user or application that started the event client. Then it delivers all events generated by the connected event modules to the event server and receives events generated by other event modules (at other sites) from the event server.

event modules

Event modules build the interface between the MetaWeb and the user (or application) connected to it. An event module defines the set of events if produces as well as a set of events it is interested in and therefore wants to consume. After connected to the event-client the event module generates events (according to user input or application specific behaviour) and sends them to the event-client. On the other hand it has to define, how to handle incoming events, delivered by the event-client. This allows to define open protocols [Roseman93] which can be interpreted by different event modules. An event module can be used from an application to propagate a state change to some interested users. On the other hand an event module can be used to visualize events propagated through the MetaWeb (e.g. representing an applications state change) to a user. Therefore different event modules can be provided with different event representation. MetaWeb will probably provide a "handy" event module, which offers basic func tionality for "browsing" MetaWeb (e.g. who-is-on, session-browser, simple paging service etc.).
As described above, in MetaWeb most work is done on client site. In consequence, facilities like access control or registration will be realized by the event modules. Different event may therefore provide different kinds of registration (open-door, facilitator-controlled etc.) and access control (floor control, locking etc.).
Event modules can be used to realize distributed applications without the need for a web-based application connected to them. By this way a set of users connected to the MetaWeb can use special event modules for synchronous collaboration (ranging from a simple textual chat or shared whiteboard to elaborated joint-editing tools).
Although event modules will usually be coupled to a web browser (since MetaWeb is designed to allow a stronger coupling between the Web and web-based applications) this is not needed. For example the "handy" event module could be used as a stand-alone application (means not executed from or inside a web browser) in order to let a user glance at the MetaWeb without browsing the Web.

All basic components of the MetaWeb (event clients, event servers and event modules) act on two basic objects of the MetaWeb : events and sessions - described in the following sections.

events

Although there are a set of pre-defined events (for session management etc.), application-specific events can be defined by the event modules. Usually events run through several stages :

Events can be seen as transportable containers of application data. Each application defines what data to store in an event, what events to produce/consume and how to visualize event reception (means how to "execute" incoming events). How to do this is very application-dependant and should therefore be left to the application developer. For example a web-based application can decide to generate an event named "Update-Event" each time the application data has changed and the application's event module on client-site can then force the browser (web client) to update the already loaded web page that contains part of the application data. On the other hand a application can define much more events (e.g. one event for each kind of possible state change) or provide more specific information about what happened to the application in an event's body. This allows application developers to define the degree of event granularity. Besides these application-specific information, each event contains at least its originato r (an event module), its target (a session) and its type (defined by the event module for event identification).

sessions

While the Web provides asynchronous interaction through static pages, MetaWeb provides synchronous interaction through active sessions. A session can be described as a period of synchronous interaction. In MetaWeb sessions are the "doors" through which events muss pass at the event-server. Sessions are created and deleted by event-modules through appropriate events. Session management (maintaining session data, performing user join or leave operations) is controlled by the event-server that hosts the session.
Joining an active session can be done explicitly (initiator-based or joiner-based) or implicitly (artefact-based) [Edwards94]. MetaWeb provides both methods through different event modules. Sessions can be used to specify the "scope of activity" of a web-based application. This implies the coupling between the Web and the MetaWeb. While most of the existing systems provide a strict page-based approach, MetaWeb will provide different levels of coupling (dependant on the used event-modules) :

The following section investigates an interesting field for web-based applications - Groupware. It shows how far the web is useful for building groupware systems on top of it and how far it restricts them. Finally it gives an example of how to use MetaWeb to overcome these restrictions for a given system.

Groupware and WWW

Since the World-Wide Web has been developed to "allow collaborators in remote sites to share their ideas" [Berners-Lee94] it's obvious to use the web for collaborative information sharing systems. These systems play an essential role in the research of Computer Supported Cooperative Work (CSCW). An almost accepted term for those systems is Groupware - described as "computer-based systems that support groups of people engaged in a common task (or goal) and that provide and interface to a shared environment" [Ellis91].

The BSCW shared workspace system

The BSCW shared workspace system [Bentley95] is an example for such a system that is built using the Web. The BSCW system is a document storage and retrieval system with features for collaborative information sharing. It maintains a number of workspaces, accessible from different platforms using standard Web clients. Each workspace contains a set of workspace members and a number of shared information objects (documents, links, folders etc.). A member of a workspace can access a workspace and all objects it contains and add other users or objects to the workspace. Each time a member accesses a workspace the system presents and overview of the workspace contents and recent activities by other users (like "user A has read Document X at time t").
The BSCW system has turned out to be a valuable collaborative system for asynchronous collaboration.
On the other hand there is a lack for synchronous collaboration and awareness in the system. Awareness can be defined as an "understanding of the activities of others" [Dourish92]. The BSCW system provides asynchronous awareness, since users accessing a shared workspace at different times are aware of other users past activities. But what about on-going activities ? If two users access a shared workspace at the same time they won't be aware of each others presence or activities in the workspace. Under the assumption that the system would provide such a synchronous awareness it would then be desired to have support for synchronous collaboration (e.g. let the two users talk to each other).
The following section sketches how MetaWeb can be used to extend the BSCW system with facilities for synchronous awareness and synchronous collaboration.

Using MetaWeb to extend BSCW

The BSCW system consists of a system kernel that handles all incoming requests (from a Web server as well as from an e-mail system). Any incoming request is handled by updating a persistent object store accordingly. The result is then passed back (to the Web server or e-mail system). In order to connect the BSCW system to the MetaWeb the BSCW kernel must be configured to trigger a special BSCW event module for each request. In this setting the BSCW event module produces special BSCW events for each action performed by the BSCW kernel and sends them to the corresponding session on a BSCW event server. Therefore the BSCW event server maintains one session for each shared workspace. Figure 6 presents an outline of the suggested architecture.

figure 6 - attaching event modules to the BSCW system

A member of a shared workspace browsing this workspace with a web browser can use the browser event module (provided by the BSCW system) for immediate visualization of other members actions. Moreover the browser event module can be used to provide information about the user's browser actions (like switching to another workspace) to the MetaWeb. For example this information can be used by other browser event modules to display who is actually accessing a shared workspace.

Providing synchronous awareness with MetaWeb

One of the major tasks in groupware is to provide an appropriate interface that allows the user to become aware of what other users are working on, this awareness component requires information about other users work. MetaWeb can be used to provide this information by encapsulating it in events and delivering it to the interested system components. To extend the BSCW system with synchronous awareness it needs two different event modules : a BSCW event module and a Browser event module. The BSCW event module creates a session for each workspace and defines the set of events it produces (like document_read, document_write). On user activities (like a user adding a document to a workspace) the BSCW kernel triggers the BSCW event module to produce an according event. This event will then be delivered to all Browser event modules that consume events belonging to this shared workspace (session). The Browser event module (loaded into the browser when first entering a shared workspace) produces events on a browser a ction like accessing a shared workspace (e.g. join_workspace and leave_workspace). It also defines to consume these browser events (generated by other users in the system) as well as BSCW events generated by the BSCW kernel.
Consider two members "Jonathan" and "Thomas" accessing a shared workspace "papers" at the same time :
Jonathan adds a paper "position paper" to the shared workspace causing the BSCW kernel to update its persistent object store and to trigger the BSCW event module to generate an according event. The BSCW event module then produces the BSCW-event "document_add", saying that member "Jonathan" has added the document "position paper" to the workspace "papers". This event is then propagated through the event server to member Thomas' BSCW event module. This event module then visualizes the event information according to the member's preferences.

Providing synchronous collaboration with MetaWeb

While special event modules can be defined to provide basic functionality for synchronous collaboration (simple text chat, shared whiteboard etc. - as described above), MetaWeb is not designed to provide high-level tools for synchronous collaboration like audio-/video-conferencing. However, MetaWeb can be used to launch such tools. Therefore the necessary information (needed by the collaboration tools for connection set-up) are stored as attributes to each user. A special collaboration event module waits for events describing an other user's wish to establish a connection for synchronous collaboration. In reaction to such an event, the collaboration event module then launches the corresponding collaboration tool, passing the needed parameters to it.

Conclusions

We have shown how far the Web provides interfaces for development of Web-based applications and what restrictions exist. These restrictions are caused by the simple stateless protocol used in the Web. It can not be expected that this basic layer of the Web will change in the future since the provided functionality has turned out to be very useful for what it was designed for : serving documents in a distributed hypertext system.
However this leads to the fact that there is a need for a common interface for Web-based applications which provides easy information exchange between different parts of a distributed Web-based application. This can be any of the existing interfaces for distributed applications (like OMG's CORBA) as well as special Web-aware application interfaces developed in the future.
We have presented a general purpose interface to shared interactive applications in the World-Wide Web, based on an event propagation model. We claim that this interface is especially useful for the development of Web-based groupware applications. We demonstrated how an existing web-based application - the BSCW shared workspace system - can be easily extended by using the MetaWeb event propagation model.
It has to be pointed out that there is no need to change any existing web technology (including web client and web server) in order to use MetaWeb in concert with existing (or newly build) web-based applications.
MetaWeb will probably be based on the Java programming language [SUN95]. Our current plans contain implementation of event client and event server as well as the development of an appropriate event representation and event transport protocol. In the second phase MetaWeb will be used to extend the BSCW shared workspace system as described above. Although MetaWeb will be used within the scope of the BSCW project it is also planned to investigate other possibilities for using MetaWeb (by development of according event modules). Therefore MetaWeb will provide a useful and general API that allows development of Web-aware applications on a high level.

References

[Bentley95] Bentley R., Horstmann Th., Sikkel K. and Trevor J. (1995) "Supporting collaborative information sharing with the World-Wide Web : The BSCW Shared Workspace System". In Proceedings of the 4th International WWW Conference, Boston, December 1995
Also available from URL http://orgwis.gmd.de/~bscw/papers/boston-95/BOSTON.html
[Berners-Lee94] Berners-Lee T., Cailliau R., Luotonen A., Nielsen H.F., Secret A. (1994) "The World-Wide Web". In Communications of the ACM, Vol.37, No.8, August 1994, pp. 76-82
[Dourish92] Dourish, P. and Bellotti, V. (1992). "Awareness and Coordination in Shared Workspaces". In Proceedings of CSCW i:92, Toronto, Canada : ACM., pp. 25-38.
[Edwards94] Edwards W.K. (1994)."Session Management for Collaborative Applications". In Proceedings of CSCW '94, Chapel Hill, USA. ACM, pp. 323-330
[Ellis91] Ellis, C.A., Gibbs, S.J. and Rein, G.L. (1991). "Groupware : Some issues and experiences". In Communications of the ACM, Vol.34, No.1, 1991.
[Roseman93] Roseman, M. and Greenberg, S. (1993)."Building flexible groupware through open protocols." In ACM Conference on Organizational Computing Systems, California, October 1993, ACM Press.
[SUN95] SUN (1995). "The Java language : A White Paper". From URL : http://java.sun.com/1.0alpha3/doc/overview/java/index.html