Prof. Dr. Helmut Schauer, Prof. Dr. Beat Schmid,

INFORMATIK ? INFORMATIQUE 5/1998 1 Impressum Herausgeber ? Editeur

SVI/FSI Schweizerischer Verband der Informatik-organisationen / Fédération suisse des organisations d’informatique

Postfach 71, 8037 Zürich

vertreten durch ? représenté par

Prof. Dr. Helmut Schauer, Prof. Dr. Beat Schmid,Prof. Dr. Carl August Zehnder

Redaktion ? Rédaction

Fran?ois Louis Nicolet

Schwandenholzstr. 286, 8046 Zürich

T elefon 01 371 73 42, T elefax 01 371 23 00E-Mail

Beirat ? Comité consultatif

Prof Dr. Boi Faltings (EPFL Lausanne), Prof. Dr. Ste-fan Klein (Universit?t Münster), Dr. Ernst Lebsanft (Synlogic AG, Binningen), Prof. Dr. Rudolf Marty (IFA Informatik, Zürich), Prof. Dr. Moira C. Norrie (ETH Zürich), Prof. Dr. Hans-J?rg Schek (ETH Zürich), Dr.Hans Stienen (Synscape, Binningen), Helmut Thoma (ITpro AG Basel), Dr. Reinhold Thurner (Herrliberg),Rudolf Weber (Mummert+Partner AG Zürich) English proof-reading:

Michael Hird, Alasdair MacLeod

Druck, Versand ? Impression, expédition Stulz AG, 8942 Oberrieden

Inseratenverwaltung ? Régie des annonces Jean Frey Fachmedien, Postf., 8021 Zürich T elefon 01 448 86 34, T elefax 01 448 89 38E-Mail

Jahrgang ? Année de parution

5. Jahr ? 5ème année

Erscheinungsweise ? Parution

J?hrlich 6 mal ? 6 fois par an

Au?age ? Tirage: 4’500 Exempl.

Die Zeitschrift sowie alle in ihr enthaltenen Beitr?ge und Abbildun-gen sind urheberrechtlich geschützt.

La revue ainsi que toutes les contributions et ?gures qu’elle con-tient sont protégées par la loi sur les droits d’auteur. Bezugsm?glichkeiten, Abonnements Possibilités d’obtention, abonnements

Die Zeitschrift I NFORMAT I K/I NFORMATIQUE ist of?zielles Organ der folgenden I nformatikorganisationen ? La revue INFORMATIK/INFORMATIQUE est l’organe of?ciel des organi-sations d’informatique suivantes

SI Schweizer Informatiker Gesellschaft

Société Suisse des Informaticiens

Schwandenholzstr. 286, 8046 Zürich Tel. 01 371 73 42, Fax 01 371 23 00

E-mail WIF Wirtschaftsinformatik-Fachverband

Hans Huber-Str. 4, Postfach, 8027 Zürich Tel. 01 283 45 30, Fax 01 283 45 50

E-Mail

INFORMATIK/INFORMATIQUE kann von den Mitgliedern fol-gender I nformatikverb?nde über ihren Verband bezogen wer-den ? Les membres des associations suivantes peuvent obtenir la revue INFORMATIK/INFORMATIQUE auprès de leur asso-ciation

GRI Groupement Romand pour l’Informatique

Case postale 90, 1000 Lausanne 21

GST Gesellschaft zur F?rderung der Software-Tech-

nologie, Fritz K?ser-Str. 10, 4562 Biberist SIK Schweizer Informatik-Konferenz

Postfach 645, 4003 Basel

SVD Schweiz. Vereinigung für Datenverarbeitung

Postfach 373, 8037 Zürich.

INFORMATIK/INFORMATIQUE kann nur über die Mitgliedor-ganisationen des SVI/FSI bezogen werden (Ausnahme: ?ffentli-che Bibliotheken über SV I /FS I ). INFORMATIK/INFORMA-TIQUE ne peut être obtenue que par les organisations af?liées àla SVI/FSI (exception: les bibliothèques publiques auprès de la SVI/FSI).

ISSN 1420-6579 Fr. 15.– 2Editorial – Moira Norrie, Guest Editor 3Approaches to Accessing Databases through Web Browsers – Antonia Erni and Moira Norrie 8Integrating Globally Distributed Heterogeneous Information Sources – Michael Rys 14Collecting and Querying Medical Information on the Internet – Epaminondas Kapetanios, Moira C. Norrie, Julian Schilling 18CEPIS Position Statement regarding Electronic Commerce 19Business Round Table on Global Communications 20Theorie der Turing-Maschinen – Buchbesprechung von Thomas Bühlmann 21Einladung zur Konferenz Electronic Commerce und zur Gereralversammlung der SI 21Invitation à la conférence Electronic Commerce et à l’Assemblée générale de la SI 22Wahlen / Elections 23Statuten?nderung 23Amendement aux Statuts 23Beschlussprotokoll der Generalversammlung der Schweizer Informatiker Gesellschaft vom 24. September 199724Procès-verbal de l’Assemblée générale de la Société Suisse des Informaticiens du 24 septembre 199724Ada in Switzerland 25CHOOSE 26Security 26SGAICO 27SIPAR 27Software Ergonomics 28Bericht von der 43. Generalversammlung – Roland Wettstein 29Planungssitzung 1998 – Ugo Merkli und Dieter Spahni 30Informationsobjekte - alles Design? Informatiker und Gestalter im Dialog – Ugo Merkli 31Ein Vorstandsmitglied stellt sich vor – Dieter Spahni 32Der WIF gratuliert …Gobal Information Systems Mosaic Mitteilungen der Schweizer Informatiker Gesellschaft (SI)Communications de la Société Suisse des Informaticiens (SI)Mitteilungen des Wirtschaftsinformatik-Fachverbands (WIF)

Visit the INFORMATIK/INFORMATIQUE Web site: http://www.access.ch/sinfo/

Global Information Systems

2INFORMATIK ? INFORMATIQUE 5/1998Editorial

Four years ago, a lecture course entitled “Global Information Systems” was offered for the ?rst time by the Institute of Infor-mation Systems at ETH Zurich. At that time, everyone was talking about the World-Wide Web, but people were only beginning to explore this global information space and business sites were few and far between. Questions were raised about the likely impact of this new technology on society and whether there could ever be any real commercial bene?ts.

As database researchers, we had always considered ourselves at the centre of the information management business. Sudden-ly, it seemed that our world of structure and organisation had been overtaken by a world of chaos. What was emerging was a global network of information producers and consumers in which anyone was free to publish whatever information they chose in a completely unstructured format. How should we react? One possibility was to bury our heads in the sand and hope that it would go away. The other was to jump on the band-wagon and try to be part of this brave new world. We chose the latter approach. In whatever direction this new community would go, it was clear that there would be problems of manag-ing information on the global scale – and there we could con-tribute.

That ?rst course offering for “Global Information Systems”was very much like the Web itself in terms of its exploratory nature. The Web was evolving so quickly that within the time span of the course there were many new and exciting develop-ments. So what was the content of that course? The emphasis was on the need to adapt and integrate existing information technologies to deal with a dynamic and open world. Concepts from information retrieval, knowledge representation and data-base systems could be used to tackle the problems of ?nding, accessing and processing information represented as HTML documents.

Over the past four years, the course content, like the Web, has changed greatly. No longer do we question the commercial potential of the Web and each year new technologies are intro-duced. Nowadays, any serious Web developer has to know about everything from Internet programming in its various guises to database and agent technologies. Web development tools are one of the largest growth sectors of the software mar-ket. However, the underlying message of our course remains the same – the development of an effective information system requires a careful selection and integration of appropriate tech-nologies. It is important to understand not only the available technologies, but also the characteristics of the application and its clients to guarantee a system that will meet its requirements in terms of functionality, presentation and performance.

For this special issue, we have selected three papers that illustrate the variety of technical, architectural and application issues involved in developing global information systems. The link between information systems and the Internet arises out of both the desire to allow users to access databases over the Internet and the need to use database technologies to manage information published on the Web. We start by examining the different approaches used to provide access to database sys-tems via standard Web browsers. It is important to appreciate the relative advantages and disadvantages in selecting the best database Web technologies for an application system or under-standing where the performance problems arise.

Next, we look at how a “database view” of the Web can be realised. Database people are used to locating and processing information by means of querying and a number of research projects are investigating how the concept of querying can be brought to the Web. Representative of this work is the TSIMMIS project at the University of Stanford and a member of this project group, Michael Rys, kindly agreed to write an overview of this project.

To conclude, we present an Internet information system developed as a part of a collaborative project between the Insti-tute of Information Systems at ETH Zurich and the Institute of Social and Preventive Medicine at the University of Zurich. This serves to show some of the architectural issues involved in developing a particular Internet service and how “thinking gen-erally” can result in, not only more ?exible and powerful sys-tems, but also the ability to reuse systems when similar services are required in further projects.

Moira Norrie, Guest Editor

Global Information Systems

INFORMATIK ? INFORMATIQUE 5/1998 3

Approaches to Accessing Databases through Web Browsers

Antonia Erni and Moira Norrie

Access to databases through Web browsers is now commonplace with some form of Web server being provided by all major vendors of database software. In this paper, we review the various approaches and architectures for Web access to databases discussing their relative advantages and disadvantages. In particular, we compare an approach based on CGI programming with one based on special Web DBMS components implemented in Java.

Introduction

All major vendors of database management systems (DBMS) now provide some means of accessing a database sys-tem through standard Web browsers such as Netscape Commu-nicator and Internet Explorer. The demand for such access arises not only from the desire to provide simple, universal interfaces to shared information services, but also from the problems of managing complex Web sites. Increasingly, data-base systems are being used to manage information published on the Web and Web documents are being generated dynami-cally.

Here, we review the various approaches and architectures currently used for providing Web access to databases. Speci?-cally, we compare in detail an approach based on Web access to remote database servers via standard Web protocols and CGI programs, with that of implementing an Internet-based client-server DBMS in which components of the system are imple-mented in Java and execute on the client side. While both ap-proaches are general and not dependent on the nature of the un-derlying D BMS, the systems we describe are both object-oriented database systems (OODBMS) with general graphical object browsers, thereby enabling a more direct form of com-parison between the approaches.

We begin in section 2, we describe the main approaches to accessing database systems via the Web and discuss their rela-

tive advantages and disadvantages. In section 3, we describe the commercial system O2 Web which is an example of a sys-tem that provides Web access via CGI programs. Section 4presents Internet OMS developed at ETH Zurich as an example of an OODBMS with a client-server architecture for the Web.Concluding remarks are given in section 5.

Approaches

In the main, there are three different technologies that can be distinguished for providing Web access to databases. To date, the most widely used is that of access via Common Gate-way Interface (CGI) programs [Gundavaram 96] which is the standard means of providing Web access to application servic-es. Two other approaches based on Java are becoming increas-ingly popular. The ?rst relies on the ability to access database systems from Java programs using standardized Java DataBase Connectivity (JD BC) interfaces. The second is to actually develop Web components of the DBMS which can be down-loaded as Java applets and execute on the client machine. We review each of these approaches in turn.

Currently, most database vendors provide CGI-based Inter-net access to databases. This includes the major relational ven-dors such as Oracle [Oracle 96] and OODBMS vendors such as O2 [O2]. One of the main goals of such interfaces is to support publication of information on the Web by means of dynamical-ly built HTML pages.

As stated previously, the main feature of CGI program access is that it uses the standard HTTP and HTML protocols for all communication between the client and the server. Users supply query parameters by means of HTML forms and these param-eters are then sent using the HTTP protocol’s GET or POST methods to the server which calls a CGI program to interpret the request and establish a connection to the requested database (possibly indirectly through a special DBMS Web Server). The query results produced by the database are encoded in HTML documents and returned to the client. There are also available general Web intermediary database servers which use special HTML extensions to encode SQL statements in HTML docu-ments and can connect to any D BMS supporting a standard SQL-based interface such as ODBC.

1

2

Global Information Systems

4 INFORMATIK ? INFORMATIQUE 5/1998

The major advantage of such a technology is that no special requirements are placed on the client side. There is no need to install special software on the client side, nor are there any requirements that the client Browser must support special plug-ins, Java or Java Script. Further, since all communication uses standard HTTP protocol, any system allowing general Web ac-cess can access the database and there are no special problems of security restrictions especially in the case of networks con-trolled by ?rewalls. Due to these advantages, we also provide such an interface as part of the OMS database development suite [Norrie 93, Wuergler 95, Leidi et al. 98] to facilitate uni-versal Web access to OMS databases.

The disadvantage of this approach is that the entire process-ing of requests is on the server side. This includes the processes of handling HTTP requests, constructing queries to be executed and communicating with the DBMS server and generating the presentation of results in terms of HTML documents. If a serv-er has many clients, this can result in severe problems of server overload and slow access. Additionally, since no processing is performed on the client side, there is no validation of form input data before the request is sent to the server. System per-formance is further restricted by the fact that access is often “stateless” meaning that there is no record of access sequences and user browsing may result in the same queries being issued and executed repeatedly without exploitation of any local or remote caching.

An example of a system using this approach is O2 Web based on the commercial OODBMS O2 and we describe the opera-tion of this system in detail in section 3.

The use of Java and JavaScript enables code to be download-ed and executed on the client machine, thereby making it pos-sible to move some of the database system functionality to the client. JDBC is a standard for database connectivity proposed by Sun that allows Java applications or applets to access data-bases [Reese 97]. Web access to one or more databases can therefore be provided by means of a Java applet provided that each DBMS provides a JDBC driver implementing the appro-priate libraries. One of the main aims of this approach is to make applications independent of speci?c D BMSs in that access through a standard interface allows different DBMSs to be accessed in the same manner and a DBMS to be changed without effecting the application. At present, most JDBCs pro-vide an SQL-level application programming interface. Efforts are underway to de?ne similar JDBC interfaces speci?cally for OODBMS. In the context of OMS, we have also developed a JDBC driver which may either work through SQL or through the OMS query language AQL [Füst?s 95].

While JDBC provides general Web client program access to remote databases, there is no database functionality as such in these Web clients - unless the application programmer codes it.Another approach is to implement general DBMS components in Java and execute these components on the client side. Typi-cally, the DBMS component executing on the client side will be concerned with data input and result presentation. This is a similar distribution of functionality to that found in typical client-server DBMS architectures. Thus, one can consider that rather than simply providing access to a database via the Web,

an Internet-based client-server architecture is developed with functionality such as database browsing being provided through a Web browser.

D eveloping such a Web-based client-server architecture yields performance gains in that validation of data input and processing of results can be done on the client side, thereby re-ducing server load. Further, it is possible to have better control of caching on both the client and server sides. There is a trend nowadays in this direction and we use our system, Internet OMS [Erni et al. 98, Erni/Norrie 97], to explain the approach and its bene?ts in section 4.

O2 Web

In this section, we describe the architecture and operation of system O2 Web [O2]. The system uses CGI programs to pro-vide Web access to the OODBMS O2 and we show the general O2 Web architecture in ?gure l.

At the bottom of ?gure 1, we have the O2 System waiting for requests to the database. The OODBMS O2 has been specially designed to store and manage large amounts of complex and multimedia data. The O2webserver

on top of it accepts requests of clients accessing the information via the Web. For each re-quest, the O2webserver invokes the speci?ed query and gener-ates HTML pages with the results encoded. The information stored in the HTML pages is default view of the objects in the database. This is the generic mode of O2 Web which allows cli-ents to browse through hypermedia information stored in O2databases without having to implement any special kind of Web interface.

In ?gure 2, we show the default presentation of a person ob-ject of a contacts database containing information about people and organisations. The generic mode presents only the at-tributes of objects to the user and not the methods associated with the object. This restricts the forms of database information that are visible and severely reduces the browsing functionality.For application programmers interested in a speci?c presen-tation of data, O2 Web allows total control of HTML document generation for objects of each class. Information about how to present an object of a class can be stored as a class method in

3

Global Information Systems

INFORMATIK ? INFORMATIQUE 5/1998 5

the database itself. In this way, the generic presentation meth-ods of the O2webserver are overridden and the objects are no longer presented in the generic mode. Storing object presenta-tions in the database classes allows application programmers to build their own speci?c Web applications. However, by closely coupling objects and presentations, they allow for only one style of presentation and this limits data reusability.

In the middle of ?gure 1, the main components for establish-ing the connection between clients requesting data over the Web and the O2 database are shown. The HTTP Server accepts requests arriving as a URL cgi-bin request with encoded query parameters. The O2webgateway is a CGI script responsible for connecting to the requested database system and passing to it the constructed query expressed in the O2 query language OQL. To ?gure out to which of the O2 databases the query should be forwarded, the O2webgateway connects to an O2webdispatcher running on the local area network. The O2webdispatcher tells the O2webgateway which O2webserver to connect to. The query is then sent to the O2webserver which executes it and returns the results as HTML pages to the O2webgateway. The HTML pages are then returned to the cli-ent via the HTTP server.

As the client user continues to navigate through the HTML pages by clicking on links, further queries are sent to the data-base. What happens when the same query is requested twice by the same client? In sending a URL containing a call to a CGI script, the Web browser does not recognize that the result will

be the same as that already stored in its cache. The browser therefore retrieves again the same data over the Internet. In not having a cache on the server side, the O2webserver re-executes the query and rebuilds the HTML page containing the query result. The positive aspect of this technology is the fact that the resulting HTML pages always contain the newest data from the database. Such a technology is of interest for databases often changing data and for clients always interested in the newest and updated information. However, the effect is that the system is unnecessarily slow in cases where the data is relatively stable and the user merely browsing as occurs in many applications.There exist other Internet database systems using CGI scripts, but caching the result HTML pages on the server side for faster access to the desired data. Such systems are for exam-ple Oracle Web [Oracle 96] or OMS Web [Leidi et al. 98]. Internet OMS

The Internet OMS [Erni et al. 98, Erni/Norrie 97] system is an example of an alternative approach based on the idea of extending the DBMS with Web components that execute on the client machine and are accessed through a general Web brows-er. The system was built to investigate means of providing fast and effective Web access to databases and is based on our own OODBMS called OMS [Norrie 93, Wuergler 95]. The architec-ture of Internet OMS is shown in ?gure 3.

The architecture can be viewed in terms of its server and cli-ent sides with the browser part considered as the user’s entry point to the Internet database. On the server side there is the OMS System with the OMS Server on top, waiting for requests to the OMS system. The OMS server forwards the requests to the OMS system and stores the results in a format specially de-?ned for Internet OMS, containing query result data together with metadata information.

For each OMS database, a Database Agent

is installed on the server side as shown in ?gure 3. This agent is a Java-Stand-alone program and is responsible for sending the client re-

4

Figure 2:

Generic presentation of an O2 Object

Global Information Systems

6 INFORMATIK ? INFORMATIQUE 5/1998

quests, if necessary, onto the OMS server and storing the query results in a Global Cache . Queries already requested by other clients are retrieved directly from the global cache. It is impor-tant to realize that, in typical browsing modes, there are a lim-ited number of entry points and so queries at the top-levels are often already evaluated and in the cache and therefore there are signi?cant bene?ts in terms of access times.

A client requesting OMS data over the Internet ?rst requests an HTML page from the HTTP Server shown in the middle of ?gure 3. This HTML page contains a Java applet, the Front Agent , which is transferred to the client machine and which is responsible for the presentation of the OMS query results. The front agent is effectively a component of the DBMS, running on the client machine. The connection between the front agent and the database agent is established using raw sockets rather than HTTP protocol which also leads to performance improve-ments.

The front agent initially opens an OMS Client window which enables the user to start browsing the database by either dou-ble-clicking on entries of the list of available object collections or by directly entering OMS queries. Such an OMS client win-dow is shown top left in ?gure 4.

Before sending a query to the OMS database, the front agent checks if the query result is already stored in its own local cache Memory , see ?gure 3. If the result is already there, the front agent interprets the query result, dynamically generates the appropriate presentation format of the OMS Query Result object, see ?gure 3, and presents it to the user. Figure 4 shows two examples of query objects. On the bottom left there is a collection object containing all persons stored in the database.On the right side, a person object itself is presented. Internet OMS query results contain not only attributes of objects, but also methods.

The presentation of database objects in the multi-windows format, shown in ?gure 4, imitates the object browsing style of

object-oriented databases making it very convenient for users to work with object-oriented database systems, such as OMS,via the Web.

If the query requested by the Web client is not in the memory of the front agent, that means the query was not yet requested,it is sent over to the database agent for further processing. If the query has been previously requested by another client, the re-sult is stored in the global cache and can be retrieved directly from there. Otherwise, the result is sent over to the OMS data-base server. Thus, we have a two-level caching scheme.

To ensure that query results in the cache are up to date, the database agent is responsible for reacting to changes in the database. If such changes occur, the agent reruns the queries related to the data changed and replaces the query results of the global cache with the newest versions. Since clients also store query results in their own local cache, communication between front agents and the database agent is necessary. The database agent keeps track of all clients accessing the database and uses this information to notify clients of relevant changes to the database and send updated versions of data to the clients con-cerned.

Advantages of such a Web component architecture as shown on the Internet OMS example, are gains in performance and reusability. The generation of query results in a special format containing both data and metadata enables presentation to be dealt with on the client side and further enables many different presentations according to user and application preferences.Having a front agent applet running on the client side, a local application controlled memory cache can be utilized, rather than relying solely on general caches of the browser. Further,validation of user inputs -including constraint checks and also query syntax checking can be performed on the client side.In using raw sockets for Internet connection, instead of the ‘stop and go’ HTTP protocol another gain in performance is achieved. Unfortunately, not all network systems allow such direct socket connections to other machines. For example, net-works using a ?rewall-technology to connect to the outside world do not accept such connections, and therefore do not allow Internet OMS applications to run within their network.For these cases, we offer the CGI means of accessing OMS da-tabases [Leidi et al. 98]. Further the storage of query result of all clients in the global cache, again enables performance gains such that clients can pro?t of query already requested by other clients.

For further improvements of performance, Internet OMS was extended to support users regularly accessing the database.These users may obtain access to additional services by regis-tering with the database agent and installing a personal assist-ant locally. The personal assistant has a user pro?le which it uses to pre-fetch data into the local cache based on predicted user information requests. We refer to this as active caching

since it means that data may be cached, not only as the result of a user-initiated data transfer, but also at the initiation of either a personal assistant or a database agent. Detailed information about this Internet OMS extension can be found in [Erni et al.98].

Figure 4: Generic presentation of an Internet OMS Object

Global Information Systems

INFORMATIK ? INFORMATIQUE 5/1998 7

Conclusion

We have described in some detail two basic approaches to providing Web access to databases. The ?rst approach of using CGI programs has the general advantage of being universal in that it requires no special Browser versions or installations on the client machine and has no problems operating over restrict-ed networks.

The second approach of moving some of the database func-tionality to the client machine enables a client-server architec-ture to be achieved over the Web with the additional advantages of improved knowledge and control of operation, thereby facil-itating cooperative client-server caching schemes. The penalty is a certain loss of universality due to user, network and brows-er restrictions on Java applets that sometimes prevent the oper-ation of this form of Web interface. The current solution is to provide both forms of interface and allowing application devel-opers and/or users to select the most appropriate.

References

[Erni/Norrie 97] A. Erni and M. C. Norrie. Agent Based Internet

Database Services. In 4th Doctoral Consortium CAiSE’97, Bar-celona, Spain, June 1997.

[Erni et al. 98] A. Erni, M. C. Norrie, and A. Kobler. Generic Agent

Framework for Internet Information Systems. In Proc. IFIP WG 8.1 Conference 98, Beijing, China, 1998.

[Füst?s 95] T. Z. Füst?s. OMS Internet Integration using Standard

Interfaces. Master’s thesis, Institute for Information Systems,ETH Zurich, 1995.

[Gundavaram 96] S. Gundavaram. CGI Programming on the World-Wide Web. O’Reilly & Associates, 1996.

[Leidi et al. 98] T. Leidi, A. Erni, and M. C. Norrie. OMS Web. Tech-nical report, Swiss Federal Institute of Technology (ETH), 1998.[Norrie 93] M. C. Norrie. An Extended Entity-Relationship Approach

to Data Management in Object Oriented Systems. In 12th Intl.Conf. on Entity Relationship Approach, pages 390–401, Dallas,Texas, December 1993. Springer-Verlag, LNCS 823.[O2] O2 Technology Inc., Versailles, France. O2Web.

[Oracle 96] Oracle Corporation, California 94065. U.S.A.Oracle

WebServer 2.0, 1996.

[Reese 97] G. Reese. Database Programming with JDBC and Java.

O’Reilly & Associates, 1997.

[Wuergler 95] A. Wuergler. Object Model System: An Object Data-base Management System for the OM Data Model. Master’s the-sis, Institute for Information Systems, ETH, 1995.

5

Global Information Systems

8 INFORMATIK ? INFORMATIQUE 5/1998

Integrating Globally Distributed Heterogeneous Information

Sources

Michael Rys

In an age of globalisation of business and information, the integration of heterogeneous, globally dispersed information sources is a crucial part of successful information management. Stanford University’s T SIMMIS prototype system provides the ?exibility, scalability, and abstraction mechanisms required by this integration process. This article gives an overview of how T SIMMIS is able to integrate distributed, heterogeneous information sources using a mediation approach. It describes the component-based architecture and how T SIMMIS supports the (semi)automatic generation of the components, and illustrates the different aspects with examples.

Introduction

The improvements in databases, Web-access, world-wide connectivity, and ubiquitous computation have made our world rich with resources which contain potentially useful informa-tion. However identifying, extracting and combining useful information becomes a dif?cult task, as the information sources become more complex in scope, more widely dispersed, and more perse and heterogeneous due to their autonomous development.

Applications that want to access these information sources are confronted with a wide variety of different communication protocols, data models and source capabilities. The integration of these sources into a single information system is a highly complex task. Providing access requires a persity of knowl-edge about the location and status of the sources, their contents,semantics, access protocols, and formats. Reusing the access component of such an integrated application in other applica-tions is dif?cult.

As these information systems grow and evolve, their mainte-nance cost increase. The list of required functions and their information resources grows as applications expand. Complex systems are hard to scale, even when based on a client-server architecture, due to their n:m interactions between clients and servers. Guaranteeing the availability of, and adding function-ality to, information sources for all existing clients increases maintenance cost considerably.

Furthermore, the availability of data from many, heterogene-ous, and possibly relevant sources overwhelms decision-mak-ers and their applications, resulting in what is called informa-tion overload . It turns almost impossible to deal correctly with all the many crucial details needed to integrate, explore and utilize the information sources. There often is an impedance mismatch between the clients’ needs and the services made available by the information sources.

By introducing a mediation middleware layer, we are able to provide scalability, maintainability, and value added services that are closer to the customer needs [Wiederhold 92]. A medi-ator is a software module that exploits encoded knowledge about selected sets or subsets of data to create information for a higher layer of application. It should be kept small and sim-ple, so that it can be maintained by one expert, or at most, a small and coherent group of experts. Mediators provide inter-mediary services, linking data resources and application pro-grams. Their function is to provide integrated information without the need to integrate the data resources.

To solve the problem of accessing and integrating many het-erogeneous and distributed information sources, the mediation approach incorporates the concepts used by previous approach-es, such as multi-database systems, federated databases, wrap-pers of non-database sources, and knobots. It provides addi-tional value-added services, such as active integration of data and further processing, and abstraction of the data to increase the information content.

The integrated sources do generally not use the same data model or schema or speak the same query language. To avoid confusion of babylonian proportion, the data models, schemas and query languages must be translated into a lingua franca used throughout the mediation level and provided to the client applications as the data model, the schema and the query lan-guage of the mediated information system.

We present T SIMMIS [T SIMMIS

, Garcia-Molina et al. 97], an information integration prototype system based on the media-tion architecture that integrates distributed heterogeneous

1

Global Information Systems

INFORMATIK ? INFORMATIQUE 5/1998 9

information sources containing semistructured data (e.g. sourc-es on the World-Wide Web or ?les) and structured data (e.g.relational databases).

For more detailed general discussions of T SIMMIS and related work see [Garcia-Molina et al. 97] and [Chawathe et al. 94]. The Object-Exchange Model O EM

To support the integration effort successfully, the com-mon data model must be more ?exible than the models com-monly used for database management systems. In particular, it has to support a rich collection of structures including nesting,it must be able to handle missing or widely differently struc-tured data (so-called semistructured data ), and it must convey the information about the structures themselves and about the meaning of the terms used in the data.

T SIMMIS uses the “lightweight” object model O EM (Object-Exchange Model) as the common data model [Papakonstanti-nou et al. 95a]. O EM is a simple self-describing (i.e., schema-less ) object model which allows nesting of objects in an object graph structure similar to X ML [X ML ]. It is “lightweight” be-cause it requires no strong typing of the objects, and therefore is ?exible enough to express missing or differently structured data in an elegant way. These features simplify the data and information transfer among the mediation components.

O EM ’s fundamental concept is an object consisting of the four components:

? object ID . It may be constructed by the mediators to describe the origin of the object, such as a pointer to the original object in the information source. It may also be a pointer to an object in the workspace used to answer the query. Unlike object ID s in object-oriented database systems, O EM oids may be local to a query and need not be persistent.

? Label

tells what the object represents. Since there is no explicitly given schema, the label takes the role of conveying all the information about the object which otherwise is done by a schema. However, objects with a given label are not required to conform to a particular set of subobjects.

?Type of the object’s value, either set, or an atomic type like string or integer .

?Value of the object, either an atomic value or a set of objects.Figure 1 shows an example of a collection of O EM objects (without their object ID ). The topmost object is labelled library and is of type set (indicated by the curly brackets).The set contains a single publication object of type set . Its set consists of six author subobjects of type set and the four string typed objects title , location , page , and year . Each of the author objects contains a singleton set with a name object of type string .

The Query Language MSL

Using a common query language allows the seamless integration of new mediators and sources in an existing media-tion system. This greatly facilitates the modular construction and extension of integrated information sources.

T SIMMIS provides an object-oriented, logical query language called M SL (Mediator Speci?cation Language) to query the O EM data model. Its main contribution in comparison to related languages, such as D ATALOG or O QL , is that it allows to query both unstructured and structured data. A complete speci?cation of M SL can be found in [Papakonstantinou et al. 96a]. In a nut-shell, an M SL query consists of rules composed of a head and a body, separated by the implication sign :-. The head describes the answer objects, and the body describes the conditions that the queried objects must satisfy. In general, head and body are based on patterns of the form . The object ID ?eld is optional in a body pattern when its existence at the source is irrelevant; when missing in a head pattern, it means that a unique id has to be generated for the answer object.

The following is an example query over the object structure presented in Figure 1:

:- }> }>@DBLP

The angular brackets associate labels with their values, while curly brackets group members of a set which would be the val-ue of some object of type set . The query retrieves all publica-

2

3

Global Information Systems

10INFORMATIK ? INFORMATIQUE 5/1998

tion titles of the year 1997 from a source called DBLP which can be either a mediator or wrapper. It is applied against the root object library and matches the pattern in the body of the query against the object structure of the source by looking for paths in the object structure (such as library.publication.title ) that agree with the nested structure of the query. In our example, there is exactly one path which has the correct path and satis?es the condition on the year subob-ject. X denotes the variable to which each value of title will be bound if the year is 1997. The result of the query is a set containing for each binding of X exactly one subobject with the label pubtitle and the value bound to X .

The M SL rule is actually a query. However the same rule could also specify a wrapper or trivial mediator: one that ex-ports 1997 publications from the DBLP source. We will discuss this in more detail in Section 4.2.

In addition to M SL , T SIMMIS offers an O QL -based query lan-guage for the O EM model called L OREL which serves as an alternative end-user language for T SIMMIS . L OREL is also the query language of L ORE (Lightweight Object REpository), a D BMS for semistructured data using the O EM data model [L ORE ].

The Mediation Architecture

The various mediation services within the mediation middleware layer are modular components which can interact with each other using the common data model and query lan-guage. Figure 2 depicts the T SIMMIS mediation architecture consisting of wrappers that provide the mapping between the source models and the mediation model, and the mediators that perform the integration. In the current prototype implementa-tion of T SIMMIS , each component is implemented as a C ORBA object that use O EM as data model and supports M SL as the que-ry language.

One main aspect of a mediation system is that its mediation components provide domain-speci?c functionality. This requires that the components have to be built by human experts and cannot – in the general case – be created automatically.However, many tasks of wrappers and mediators, such as com-munication, post-processing and query planning and execution,are common to all wrappers and mediators. Thus, one of the main goals of T SIMMIS is to provide toolkits and libraries that help to automate the generation of such generic parts in order to facilitate the manual generation of the mediation compo-nents, so that the component builder can concentrate on the do-main speci?c issues.

4.1 Wrappers

In T SIMMIS , wrappers provide access to the integrated infor-mation sources. A wrapper converts the underlying data objects of a source to O EM and transforms the (implicitly or explicitly given) schema of the source into the view it provides to the mediators. It also translates the M SL queries it receives from the mediators into queries understood by the sources query processing facility.

The goal is to provide access to information sources of vastly different nature, such as simple text ?les, world wide web sources, and relational databases. This means that the wrapper may translate the M SL query to SQL, but it can also mean that the M SL query is translated into the input of a CGI script in order to query a Web-based source.

Unlike standard database system interfaces, where all queries over the database schema and content can be executed,wrapped information sources normally only allow a limited set of queries. For example, a bibliographic search engine may respond to queries asking for the title written by a certain author, or the publication dates of these titles, yet be unable or unwilling to respond to a query asking for all titles published in a given year.

A T SIMMIS wrapper therefore ?rst decides whether its under-lying source can answer the query, i.e., whether the query is directly supported . If so, it transforms the query into a source query. The wrapper then transforms the answer into the appro-priate O EM objects and returns the result. If the query is not directly supported, the wrapper determines if the query can be answered by applying some built-in local ?ltering to the result of a directly supported query. For example, the source might not support selections over a speci?c attribute but can provide the full data. The wrapper can then perform the selection itself if the selection is built-in.

In many cases, the wrapper needs to extract data from unstructured or weakly structured sources such as Web pages,text documents or simulation reports. This extraction process is often achieved by a wrapper subcomponent called extractor . Its purpose is to extract the required data and to transform it into the required data model. T SIMMIS provides such an extractor that extracts and transforms data from Web pages using a tem-plate that speci?es the extraction and transformation process using simple pattern matching rules of the following kind:

4

Global Information Systems

INFORMATIK ? INFORMATIQUE 5/199811

[source object, pattern 1, object 1, pattern 2,

object 2, … ],

[object 1, pattern 11, object 11, … ], …

The ?rst of the above rules applies the pattern 1 to the text represented by the source object, and stores part of the data in the target object 1 as speci?ed by the pattern. If this pattern does not match, the next patterns are applied until one or none of the pattern matches (in the last case the source object becomes empty). The second rule applies the pattern 11 to the object 1 and extracts data to object 11. Unless otherwise speci?ed, the target objects become subobjects of the rule’s source objects in the resulting O EM object graph. For more information on the extractor and its capabilities see [Hammer et al. 97a, Hammer et al. 97b].

In order to simplify the construction of wrappers, T SIMMIS provides a template-based tool for generating wrappers [Papa-konstantinou et al. 95b, Hammer et al. 97c]. It takes a speci?-cation of the O EM objects provided by the wrapper, the map-ping of the provided views to the source actions, and three user-written functions that are needed to connect to the source and execute queries, and combines them with the standard compo-nents used to perform the communication, ?ltering etc. (Figure 3).

For example, the O EM “schema” of a bibliographic source might look like

pub_schema 1 publication 2 author

3 url dblp_url

3 name author_name 2 title title

2 location location 2 page pageno 2 year year *

where the ?rst line gives the name of the O EM “schema” and the following lines give the structure and the mapping from the source data to the output objects. The ?rst column indicates the nesting level of the objects, the second column indicates the label of the object from the source and the optional third indi-cates the label as exported from the wrapper.

This “schema” de?nition is then used in the wrapper tem-plate to specify the views over the source and their mappings to the source. For example the wrapper speci?cation

pub_schema:

X :- X: <&O publication {}> AND title_contains(O,$S)

// $$ = "getpublication -contains " ^ $S //.

provides the view of all publications where the title contains a speci?c substring. The M SL rule gives the view de?nition, and the part between the // indicates the source query (in this case it is a P ERL program which executes the web extractor with a source speci?c extraction template). Since the source objects may not provide all the subobjects, the M SL rule only uses the labels required to de?ne the view. Note that T SIMMIS wrappers allow the use of additional built-in predicates such as compari-son operations by a query to a wrapper. These predicates per-form further ?ltering of the result and don’t have to be speci?ed in the view de?nition, nor do they have to be mapped to the sources.

There is the opportunity to answer certain queries that match no pattern by deducing a logically equivalent combination of several queries. For more information about extending the wrapper rules to handle such queries and for an example see [Garcia-Molina et al. 97].

4.2 Mediators

Mediators , the main components of the middle layer of a mediated architecture, integrate multiple heterogeneous infor-mation sources. They provide signi?cant added functionality over the wrapped source capabilities and transform the data provided by one or several (wrapped) sources to information needed by an application. Such transformations require knowl-edge of where the data are, speci?cations about the data repre-sentations, as well as an understanding about the level of detail versus the users’ expectations [Wiederhold 92]. While wrap-pers convert the source data models into the mediation data model, mediators perform the schema or general domain model conversions as required by the integration.

The T SIMMIS project focuses on mediators that provide inte-grated O EM views over underlying data sources, and on how to generate the generic parts automatically.

In order to generate a mediator, a M SL speci?cation is given to the Mediation Speci?cation Interpreter (MSI) [Papakonstan-tinou et al. 96a] to de?ne the view provided by the mediator and the rules on how to execute the view over the integrated sourc-es, which are either wrappers or some other mediators. The body of the M SL rules describe the objects which must be present at the source(s) in order for the de?ned object to appear in the view, and the conditions these objects must satisfy (see Section 3). The MSI is thus provided by T SIMMIS as a general query execution engine specialised to the mediation domain through the views speci?ed by the mediator builder using the M SL rules.

Queries to the mediator are then speci?ed over the exported views. At runtime, the mediator collects and integrates the nec-

Global Information Systems

12INFORMATIK ? INFORMATIQUE 5/1998

essary data from the sources, based on the received query and the view’s speci?cation. This process is analogous to expand-ing a query against a conventional relational database view.Thus, these mediators perform dynamic integration, in contrast to the data warehousing approach, where these views are pre-materialized [W HIPS ].

The mediators in the current T SIMMIS implementation pro-vide the following functionality:

?routing: The mediator offers a view which abstracts from the actual source location and passes the query and answer between the client and the source.

?source abstraction: The mediator provides further ?ltering and restructuring of a single source. While this functionality is best moved as closely to the source as possible (i.e., incor-porated into the wrapper), there are cases when a wrapper cannot be altered and a new application needs to abstract the source further.

?source integration: The mediator provides the functionality to combine and integrate the data from different b9b62c3a376baf1ffc4fad78mon ways to integrate source data include union over sets of source data, join between them, and fusion of the inpidual source objects into integrated objects.

The following M SL rule is an example for a join between two bibliography sources.

<&f1(I,D) pub {