IDLE: Unified W3-Access to Interactive Information Servers

Abstract

This paper focuses on the problem of accessing interactive information servers ("interactive sessions" in W3-terminology, e.g. an existing library information system) directly with a W3-browser instead of opening an additional viewer (e.g. a terminal emulation program). IDLE is a language designed to describe the user interface of interactive information servers in a way that allows automatic translation of W3-requests to server-commands. Interactive information servers described with IDLE can be accessed by any W3-browser via a special translation server. Describing the user interface of an interactive information server is typically much easier than designing a fully featured gateway for each service.

Keywords:

World-Wide Web (W3), translation servers, interactive sessions, user interface description language (IDLE).

1. Introduction

In this paper we present another approach: Translation servers [Perrochon 94b] handle the translation from a stateless protocol like HTTP (Hypertext Transfer Protocol) into a stateful one that can be used by interactive information servers (e.g. VT-100, 3270). Translation servers can also transform hypertext-links to database queries and the query results back to hypertext. To communicate with an interactive information server, the translation server accesses the same user interface as any human does [Perrochon 94b] (Fig. 1).

Fig 1. Translation server as intermediary.

Generic translation servers are able to access different interactive information servers. To achieve this, the user interface of an interactive information server is described in a formal way. This paper proposes a simple user interface description language (IDLE, Interface Description LanguagE) useful to describe a whole range of servers. Information servers described with IDLE can directly be accessed by W3-browsers using HTTP and a translation server. The language IDLE allows the programmer to concentrate on the hard part of the task: automating the user interaction required to fetch useful data from an interactive information server. All low level aspects of communication with both the server and the browser are completely hidden from the IDLE programmer.

The structure of this paper is as follows: in Section 2 we will define the notion of states and some other terms and we also will refer to related work. In Section 3 IDLE is introduced. Section 4 discusses some implementation aspects, while Section 5 presents a short example. The paper ends with a summary and some future-oriented thoughts. An EBNF-like description of IDLE can be found in the appendix.

2. Definitions and Related Work

2.1 Definitions

Browser (or client): Similar to a terminal emulation program that accesses an interactive information server, a browser accesses a non-interactive information server. Mosaic is the most famous browser used to access W3-servers.

State: [Comer et al. 94] defines: "Information that a server maintains about the status of ongoing interactions with clients is called state information. Servers that do not keep any state information are called stateless servers; others are called stateful servers". State information includes all stored state variables and their values. An individual set of values defines a state. A server is said to change its state whenever the values of its state variables change.

A stateless server always accepts the same set of requests from clients and its reaction solely depends on the commands given. A stateful server can assume several different states; it changes its state depending on the previous commands. In addition, requests are always interpreted depending on the current state. Whether a server is stateless or stateful depends also on the communication protocol used. A stateless server cannot function with a protocol that makes the meaning of a message depend on a previous message. Typically, an interactive information server is stateful. Examples: W3-servers are stateless (apart from collecting statistical information), library information systems and similar information servers accessed with telnet are stateful servers.

"Interactive information server" and "stateful information server" are synonyms. The former focuses on the interaction with the user, the latter on the server itself. We prefer the term interactive information server, because it is close to "interactive service", another synonymous notion that is frequently used in the W3-context.

2.2 Related Work

3. IDLE - A Simple Interface Description Language

There exist many scripting languages for manipulation character strings, such as Perl or awk. IDLE's regular expressions provides the same string matching functionality. In addition is has special functions that handle communication with both the information server and the browser.

Because IDLE programs are neither very large nor very complex, no support for modularization and abstract data types or object-oriented programming is given. An EBNF like definition of the language can be found in the appendix.

3.1 Variables and Functions

For manipulating string lists and single strings, many functions are defined:

var := expr

Assignment: The expression expr is evaluated and the resulting string list is copied to var. If var already exists, its previous contents are destroyed.

ADD (expr1, expr2)

The string list of expr2 is appended at the end of the string list of expr1. As we are handling lists, duplicates are not removed.

DEL (expr1, expr2)

All strings of expr2 which also are in expr1 are deleted from expr1. (For sets, this is also called "asymmetric set difference".)

FIRST (expr)

Returns the first string of expr. If expr is an empty list, FIRST returns an empty string.

LAST (expr)

Same as FIRST but returns the last string of expr.

CONCAT (expr, str)

The string str is appended to all strings in expr.

RIGHTOF (expr, str)

All strings of expr are searched for str. If str matches a string of expr, all characters right of the occurrence of str are copied to the result. If a string of expr does not match str, it will not be copied to the result.

LEFTOF (expr, str)

All strings of expr are searched for str. If str matches a string of expr, all characters left of the occurrence of str are copied to the result. If a string of expr does not match str, it will not be copied to the result.

BETWEEN(expr,str,str)

Same as RIGHTOF and LEFTOF, but all characters between the two search strings are copied to the result.

3.2 Searching Strings and Variables (Regular Expressions)

"+"

One or more repetitions of the preceding regular expression.

"?"

The preceding regular expression is optional.

"*"

Any number (0..n) of repetitions of the preceding regular expression.

"."

Matches any character.

"(", ")"

Groups a subexpression.

"|"

Alternation. Either the regular expression to the left or the regular expression to the right may match.

"[", "]"

Range. Matches one character out of the specified range or character class. Ranges are written as "[a-z]". Defined character classes are: alpha, upper, lower, alnum, xdigit, space, print, punct, graph, cntrl, blank. Character classes are written as [:name:].

3.3 Notes on IDLE Semantics

Backphases (BP) and Frontphases (FP)

An IDLE program must follow the simple rule, that FP and BP alternate (Fig. 2)! It is therefore forbidden to use a BACK statement in a BP or to use a FRONT statement in a FP. Execution of an IDLE program always begins with the start-phase (i.e. the one named START). An IDLE program terminates when it reaches the end of a phase before reaching a FRONT or BACK statement.

Note that the FRONT and BACK statements are in fact jumps or mode switches to the respective phases rather than subroutine calls. When executing a FRONT or BACK statement, the current phase is terminated and the next phase is started. Jumps can also go backwards to previous phases.

Fig. 2 Basic mechanism of a translation server.

The Errophase (EP)

ERROR READ/OPEN

You can specify pairs of (errorsymptom, id) to catch typical error messages generated by the server. Every OPEN and READ statement will watch for the symptoms while they execute. If a symptom is found, the variable IDLE_ERROR receives the appropriate id and the statements after the BEGIN in the EP are executed. Note that specifying too many errorsymptoms will slow down the processing of READ statements.

TIMEOUT FRONT

sets the number of seconds a PAGE statement will wait for an answer (i.e. a follow-up request from the user), before it gives up.

TIMEOUT BACK

sets the number of seconds an OPEN or READ statement will wait to succeed before giving up.

Note: The EP is not yet implemented in the current version of the translation server.

OPEN / CLOSE / READ / WRITE

IF condition THEN stmseq [ELSE stmseq] END

"expr1 = expr2"

Both string lists must be the same. This means that all the strings must be the same and the order of the strings must be the same.

"expr1 # expr2"

Contrary of "=": the relation yields TRUE when one or more strings are different.

"expr CONTAINS str"

Yields TRUE if str matches any string of expr.

FOREACH var IN expr DO

WHILE condition DO stmseq END

PRINT

RESUME

Comments

3.5 The PAGE Statement

INPUT incontrol argpairs

STRING

The user is prompted for a string. The user may enter any string (including linefeeds, etc.).

PASSWORD

Same as STRING, but the characters are not shown in the entry-field.

MENU

The prompt strings are displayed in a menu among which the user is supposed to select one. The identifier of the selected menu item is returned.

RADIO

Radio buttons are used to create fields of mutually exclusive items of which the user can select one. The identifier of the selected button is returned.

CHECK

Similar to radio buttons, but allows multiple items to be selected. The identifiers of all the selected boxes are returned.

REF

The strings in the argpairs are displayed as hypertext-links to further information. If the user selects a link the identifier of that link is returned by INPUT. In contrast to the other input control elements, when the user selects a hyper-link, the selection is immediately returned. For all the other control elements, the user has to press a "submit"-button first.

Note: If several INPUT statements put their results into the same variable, the returned strings will be combined in a list. The order of the INPUT statements in the PAGE description defines the order of the returned strings.

OUTPUT outcontrol expr

TITLE

If this control occurs in the first OUTPUT-statement of a PAGE-statement, the expression will become the title of the page presented to the user.

HEADER

The HEADER flag allows placing headings on the PAGE. Currently, six different levels of headings are defined in IDLE (refer to the [HTML] documentation for further details).

PREDEFINED

The expression will not be formatted by the browser. A fixed width font will be used to display the text.

4. Implementation Aspects

4.1 Staying alive to keep connections open

Our solution is a small "starter-program". This program merely establishes a communication channel between the W3-Server and the translation server. When a request is served, the "starter-program" terminates and the translation server stays alive. This concept allows the translation server to keep communication channels open as long as needed. An alternative would be to modify an existing W3-Server to pass "special requests" directly to the translation server. This would restrict the translation server to run with a so-patched W3-Server only. This is not acceptable.

4.2 State Information

• Reading from streams can be done by using the simple synchronous (wait until data arrives) read-calls. With the database solution we would have been forced to use asynchronous (no wait) read-calls to allow a certain amount of parallelism in the translation server. The scheduling mechanism of Unix ensures, that waiting processes do not block the system.
• There is no extra programming for accessing a database nor is there any dependency on third-party databases because everything is included in the translation server.
• Creating new processes is far easier than interfering with a database system.
• The processing of multiple parallel requests is controlled by the operating system.

4.3 Parsing IDLE

5. An Example: SVI-Kurs

Fig. 3 Screen shot of existing Interface of SVI-Kurs.

FRONTPHASE START
BEGIN
    PAGE
        OUTPUT HEADER 1 "SVI-Kurs";
        OUTPUT "Select an access method:";
        INPUT RADIO 
         ("By Society", "1", "By Subject", "2",
         "By Date", "3", "All events", "4") INTO sel;
    END;
    IF sel = "1" THEN BACK soc END;
    IF sel = "2" THEN BACK subj END;
    IF sel = "3" THEN BACK date END;
    IF sel = "4" THEN BACK all END;
END

<HTML>
<HEAD> <TITLE>SVI-Kurs</TITLE></HEAD>
<BODY>
<FORM method="POST" action="/cgi-bin/svi_kurs">
<H1>SVI-Kurs</H1>
Select an access method:<p>
By Society<input name=sel type=radio value=1>
By Subject<input name=sel type=radio value=2> 
By Date<input name=sel type=radio value=3>
All events<input name=sel type=radio value=4>
<p>
<input type="submit" value="Submit">
</form>

The interface description of SVI-Kurs starts with a frontphase, requesting more information from the user (Fig. 4). In the PAGE statement, the user is offered four different access methods. If the user selects one of the choices, this selection is stored in the variable sel. Depending on the result a backphase is then selected. The whole interaction with the user, i.e. waiting for the initial user contact, preparing of HTML-pages, sending this page to the user, waiting for an answer, checking all follow-up requests to the same site for the answer from the same user and finally parsing this follow-up request for the necessary data, everything is done by the translation server.

Fig. 4 Result of frontphase START on a W3-browser.

Let us assume now that, the user wants to see all the events and courses stored in the system. Then the translation server will start backphase all.

BACKPHASE all
BEGIN
    OPEN PORT "ezinfo.ethz.ch" 23; (* login to SVI-Kurs *)
    WRITE "\r\n";
    READ UPTO "Username:";
    WRITE "GAST\r\n";
    READ UPTO "ezinfo>";
    WRITE "EXT SVI\n";
    READ UPTO "22;3H>";    (* Prompt of SVI-Kurs *)
    WRITE sel;    (* Forward the user selection *) 
    
    n := "";
    info := "";
    id := "";
    READ UPTO "(";
    READ COUNT 1 INTO c;
    WHILE c # "E" DO (* Now start to fetch data *)
        IF c = "N" THEN
            READ UPTO ">";
            WRITE "N";
            n := CONCAT (n, "N");
        ELSE 
            IF c # "M" THEN
                READ COUNT 3;
                READ UPTO "\n" INTO line;
                info := ADD (info, CONCAT (CONCAT (line, "!"), CONCAT (n, FIRST(c))));
                id := ADD (id, CONCAT (n, FIRST (c)));
            END;
        END;
        READ UPTO "(";
        READ COUNT 1 INTO c;
    END;
    WRITE "M";
    READ UPTO ">";

    FRONT all_out;    
END

Only now the translation server opens a connection to the information server, goes through the login procedure and gets the necessary data. The data is then temporarily stored in a variable info. Finally, the next frontphase all_out is called.

FRONTPHASE all_out
BEGIN
    PAGE
        OUTPUT HEADER 1 "List of all Events";
        FOREACH x IN info DO
            text := LEFTOF (x, "!");
            id   := RIGHTOF (x, "!");
            INPUT REF (text, id) INTO sel;
        END;
    END;
    BACK all_getitem;
END

<HTML>
<HEAD> <TITLE>SVI-Kurs</TITLE> </HEAD>
<BODY>
<H1>List of all Events</H1>
<ul>
...
<li>
<a href="/cgi-bin/svikurs2/SEQ=2/sel=N3">19.01.95 :
Workgroup und Workflow</a>
...
</ul>
</BODY>

This frontphase extracts the data from the variable info and sends a page to the user (Fig. 5). In this case, the page consists of a list of hypertext-links of which the user can select one. All this is invisible for the IDLE programmer. If the user selects one of the links, the translation server will eventually resume its operation with the next backphase, the variable sel then contains the identifier of the selected link. Note that in this case the user will have to select between hypertext-links. He or she does not have to push a "submit"-button after the selection.

Fig. 5 Result of frontphase all_out on a W3-browser.

BACKPHASE all_getitem
BEGIN
    WHILE sel # "" DO
        IF sel = "[0..9]" THEN (*First navigate to the right place *)
            WRITE sel;
            sel := "";
        ELSE
            WRITE "N";
            READ UPTO ">";
            sel := RIGHTOF (sel, "N");
        END;
    END;
    READ COUNT 23;
    READ UPTO "23;33]" INTO kurs; (* get the data *)
    kurs := LEFTOF (kurs, "23;33]");

    WRITE "M";
    READ UPTO ">";

    FRONT all_showitem;
END

In this phase, additional data for the selected course are fetched from the information server. Note that in the meantime, the translation server holds the connection open, so additional information can be read from the stream.

FRONTPHASE all_showitem
BEGIN
  PAGE
   OUTPUT HEADER 1 "Selected SVI-Kurs";
   OUTPUT PREDEFINED kurs;
  END;
  CLOSE;
END

<HTML>
<HEAD><TITLE>SVI-Kurs</TITLE></HEAD><BODY>
<H1>Selected SVI-Kurs</H1><PRE>
Tagung Workgroup und Workflow
19.01.95, Zuerich
H.Gonzenbach, P.H. Albrecht
SVD-Mitglieder Fr. 300.- Uebrige Teilnehmer Fr. 540.-
SVD, Postfach 373, 8037 Zuerich, Tel. 057 33 37 05
</PRE></BODY>

The final frontphase puts the detailed information concerning the selected course into another page and sends it back to the user (Fig. 6).

Fig. 6 Result of frontphase all_showitem on a W3-browser.

Other interactive information servers described with IDLE include information servers with the research-projects of ETH Zürich, the books of the library of the computer science department of ETH Zürich and the Swiss Constitution. The description of each interface took an experienced IDLE-programmer a few hours.

6. Summary

In the near future, translation servers will be able to bridge the gap between stateful and stateless information systems. Translation Servers are well suited to access existing stateful information servers [Perrochon 94a]. We believe in the power of stateless protocols for information services. However, also in the long range, for certain tasks, e.g. complex systems and well trained users, stateful protocols are better suited. Thus some systems will remain stateful. In this situation, translation servers or similar concepts allow untrained users to access at least a subset of the functionality of the whole system.

Appendix - IDLE Definition

ID         =  [ errorphase ] {(BACKPHASE | FRONTPHASE) 
              name BEGIN stmseq END }.
stmseq     =  stm { ";" stm }.
stm        =  ( var ":=" expr )                                 |
              ( OPEN [ fileid ](FILE string | PORT string num)) |
              ( CLOSE [ fileid ] )                              |
              ( WRITE [ fileid ] (expr | NULLBYTE) )            |
              ( READ [ fileid ]
                ( COUNT num | UPTO expr )[ INTO var ] )         |
              ( IF condition THEN stmseq [ ELSE stmseq ] END )  |
              ( FOREACH var IN expr DO stmseq END )             |
              ( WHILE condition DO stmseq END )                 |
              ( PAGE stmseq END )                               |
              ( OUTPUT [outcontrol] expr )                      |
              ( INPUT incontrol argpairs INTO var )             |
              ( PRINT expr )                                    |
              ( BACK name )                                     |
              ( FRONT name )                                    |
              RESUME.
outcontrol =  TITLE | ( HEADER [1-6] ) | PREDEFINED.
incontrol  =  STRING | PASSWORD | MENU | CHECK | RADIO | REF.
errorphase =  ERRORPHASE errorseq BEGIN stmseq END.
errorseq   =  errstm { ";" errstm }.
errorstm   =  ( TIMEOUT (FRONT | BACK) "(" num "," string ")" ) |
              ( ERROR (READ | OPEN)  argpairs ).
condition  =  ( expr relation expr ) | ( expr CONTAINS string ).
relation   =  "=" | "#".
expr       =  var | string | function.
function   =  ( ( ADD | DEL ) arg2e )                           |
              ( (CONCAT | RIGHTOF | LEFTOF) arg2s )             |
              ( BETWEEN arg3) .
string     =  text | ( ( FIRST | LAST ) arg1 ).
arg1       =  "(" expr ")".
arg2e      =  "(" expr "," expr ")".
arg2s      =  "(" expr "," string ")".
arg3       =  "(" expr "," string "," string ")".
argpairs   =  "(" string "," string 
              { "," string "," string } ")".
fileid     =  num.
var        =  name.
name       =  [a-zA-Z_] { [a-zA-Z0-9_] }.
num        =  { [0-9] }.
text       =  '"' { character } '"'.
comment    =  "(*" { character } "*)".

7. References

[Berners-Lee et al. 94]

Berners-Lee, T. Cailliau, R., Luotonen, A., Nielsen, H. F., Secret, A. The World-Wide Web. Communication of the ACM. Volume 37, Number 8, August 1994.

[CGI]

The Common Gateway Interface. See http://hoohoo.ncsa.uiuc.educgi/overview.html

[Comer et al. 94]

Comer, Douglas E., Stevens, David L. Internetworking with TCP/IP Volume III. Client-server programming and applications: Prentice-Hall. 1994. ISBN 0-13-474230-3

[CUBAI]

Balestra, A., Ferruci, M. Accessing Libraries in a Uniform Way: The CUBAI Project. An abstract is published in Computer Networks and ISDN Systems, Vol. 27 (1994), Number 2, November 1994. The service itself can be reached on http://www.oat.ts.astro.it/iblio/biblio.html

[Eichmann et al. 94]

Eichmann, D., McGregor, T., Danley, D. Integrating structured databases into the Web: The MORE system. The First International Conference on the World-Wide Web (WWW'95), May 25-27, 1994, CERN, Geneva. Published in Computer Networks and ISDN Systems Vol. 27 (1994), Number 2, November 1994, pp 281-288.

[HTML]

The Hypertext Markup Language. See http://info.cern.ch/ypertext/WWW/MarkUp/MarkUp.html

[HTTP]

The Hypertext Transfer Protocol. See http://info.cern.ch/hypertext/WWW/Protocols/HTTP/HTTP2.html

[Perrochon 94a]

Perrochon, L. Multiple Service Integration Confronted With Legacy Systems. Workshop "Offering the Same Information via Multiple Services" of the WWW'94, May 1994, CERN, Geneva. (PostScript or HTML at ftp://ftp.inf.ethz.ch/pub/publications/papers/s/ea/www94.(ps|html))).

[Perrochon 94b]

Perrochon, L. Translation Servers: Gateways between Stateless and Stateful Information Systems. Network Service Conference (NSC'94), November 28-30, 1994, London. (PostScript or HTML at ftp://ftp.inf.ethz.ch/pub/publications/papers/s/ea/nsc94.(ps|html))).

[Putz 94]

Putz, S. Interactive information services using World-Wide Web hypertext. The First International Conference on the World-Wide Web (WWW'95), May 25-27, 1994, CERN, Geneva. Published in Computer Networks and ISDN Systems Vol. 27 (1994), Number 2, November 1994, pp 273-280.

[WAIS]

Wide Area Information System. See http://info.cern.ch/hypertext/Products/WAIS/Overview.html

[WWW'94]

The First International Conference on the World-Wide Web, May 1994, CERN, Geneva. Selected papers and abstracts of the workshops are published in Computer Networks and ISDN Systems, Vol. 27 (1994), Number 2, November 1994. See also http://www1.cern.ch/WWW94/Welcome.tml