6. NGDF Communications Protocol

6.1 Introduction

The NGDF Discovery Metadata communications protocol sets out a definition for how applications servicing the needs of an end user browser may interrogate many different NGDF Metadata Service Providers, either directly or indirectly through the Gateway and other Service Providers, and, in turn, how the Gateway and Service Providers will interrogate other Service Providers. The first sections of this document provide standardised data formats, what cannot be standardised is the infrastructures (hardware, software, databases) being used by the service providers. Thus a communications protocol is required to enable the exchange of data across this diverse environment.

At the time of writing, the only means of facilitating this vision in a Web–based environment is Z39.50 (Z39.50 Maintenance Agency, 1995). Other protocols and approaches continue to evolve, but only Z39.50 can demonstrate an established base of interoperating geospatial services such as those which NGDF envisions.

ANSI standard Z39.50–1995, captured internationally as ISO 23950, is a powerful yet complex protocol which has evolved to enable the near–transparent query of multiple, remote, heterogeneous databases via one simplified user interface.

Z39.50 is principally used in the library community, with Z39.50 add–ons available for virtually all of the commercially available library systems. Within this community, Z39.50 is used to provide near–transparent access to collections data held in multiple databases, whether on a local level or more widely through services such as COPAC.

In the geospatial community, adoption of Z39.50 has been most apparent in government–related initiatives such as the United States’ Geospatial Clearinghouse network fgdclearhs.er.usgs.gov/ and the Australian Spatial Data Directory <www.environment.gov.au/database/metadata/asdd/>. In both these government examples, numerous agencies offer Z39.50 access to their databases, either individually for visitors to agency– and subject–specific interfaces, or collectively through the clearinghouse URLs given above.

A major benefit of Z39.50 is its neutrality. Z39.50 servers will effectively run on any hardware platform and on top of any standards–compliant DBMS. Importantly, collaborating partners in a Z39.50 project do not need to be using the same hardware, software or underlying data structure, so long as both can agree a common mapping into one or more Z39.50 ‘profiles’.

Although the underlying complexities of Z39.50 are important (Z39.50 Maintenance Agency, 1998a; Evans 1998a; Evans 1998b), it is sufficient at this stage to state that selecting the appropriate Z39.50 ‘profile’ is a crucial step in designing a Z39.50–based system. Fortunately, a profile has already been developed for geospatial data and this is suitable for NGDF requirements of both Discovery and Detailed metadata.

Version 2.2 of the Z39.50 Application Profile for Geospatial Metadata (GEO), <www.blueangeltech.com/Standards/GeoProfile/geo22.htm>, has been developed by the United States’ Federal Geographic Data Committee (FGDC) and others as a means of enabling cross–searching across distributed collections of geospatial metadata complying with FGDC’s Content Standards for Digital Geospatial Metadata (FGDC, 1998). This specifically geospatial profile is closely based upon earlier work which resulted in the Government Information Locator Service’s (GILS) application profile (GILS 1997).

Given its tailoring towards geospatial requirements, and the close relationship between the Content Standards for Digital Geospatial Metadata to which it was designed to provide access and the Metadata work of ISO’s Technical Committee 211 (1998), GEO is well placed to meet the requirements of NGDF.

The inner workings of both Z39.50 and the GEO Profile are, as with all applications of such potential, complex. It is not intended to discuss these details here, as they are only truly relevant to those building Z39.50 applications from scratch. Rather, sufficient information is given for geospatially cognisant individuals and organisations to make informed decisions about Z39.50 and the Profile. It is anticipated that implementers will either make use of existing Z39.50–capable applications or else sub–contract such work to Z39.50–experienced programmers, such that overloading of this report with technical detail would serve only to confuse most readers, whilst offering nothing new to the Z39.50–literate. Numerous references are offered for those in search of further detail.

The GEO Profile is a fully conformant application of ANSI/NISO Z39.50 version 2, and additionally provides requirements for the construction of a GEO–compliant service capable of serving Content Standards for Digital Geospatial Metadata–compliant data across the Internet in an interoperable fashion.

6.5.1 Z39.50 Services

Services are the basic structural blocks into which the Z39.50 standard is divided. In each case, the service defines a two–way exchange of information between client and target. Simplifying hugely, the Initialisation (or Init) service may be seen as a greeting from the client ("Hello. Do you speak English?") and a response from the target ("Hello. Yes I do, let’s talk"). Without this positive two–way dialogue, the Z39.50 session cannot proceed.

The Init, Search, and Present services are required for conformance with GEO. It is normally assumed that TCP/IP will be used as the transport service.

Following the simplistic language–based analogy, above, the remaining two core services for GEO may be explained, thus;

A Search request is transmitted from the client ("OK – can I have everything you’ve got about a place called ‘York’?"), and is responded to by the target ("I’ve got 23 records matching your request").

And finally, the client uses the Present service to ask for the data ("23, eh? Can I have the first ten, please?"), resulting in transmission of the records themselves from the target.

6.5.2 Attribute Sets

GEO requires that applications recognise the Bib–1 (Z39.50 Maintenance Agency 1998 b ) and GILS (1997) attribute sets and support the GEO attribute set. The GEO attribute set is outlined as part of the Profile, and the other two are often supported in existing Z39.50 applications.

6.5.3 Diagnostic Set

The GEO Profile utilises the commonly understood Bib–1 diagnostic set (Z39.50 Maintenance Agency 1998c) for transfer of diagnostic details between applications.

6.5.4 Record Syntax

Record syntaxes are used in Z39.50 to define the manner in which data is passed from target to client in response to a Present request.

The GEO Profile requires support of the HTML and SUTRS (Simple Unstructured Text Record Syntax) record syntaxes, and optionally allows use of SGML (ISO 1986), GRS–1 and USMARC (Library of Congress 1998) record syntaxes within conformant applications.

6.5.5 Schemas

Schemas may be used within Z39.50 to reflect dependencies and relationships between individual data elements. A schema might, for example, declare that where an ‘Easting’ is used, a ‘Northing’ must also be present.

The GEO Profile defines a GEO Schema which all conformant applications must support. This Schema is expressed as a Document Type Definition (DTD), and is similar in structure to the NGDF Transfer Format

Although only a subset of full Z39.50 functionality, the GEO Profile is over–large for NGDF Discovery Metadata requirements. As such, this section is intended to identify the set of elements and attributes required for an NGDF Discovery Metadata Service, mapping these to NGDF’s Discovery Metadata elements.

In the short term, NGDF–conforming applications should be designed to support the GEO profile in its unaltered state. There is scope, however, for a degree of streamlining in the future, resulting in a more tightly defined Profile specifically tailored to NGDF. This work might also usefully tailor GEO’s Summary Record Format to map exactly with NGDF’s Discovery Metadata elements. This is not currently the case, meaning that NGDF records can only reliably be presented using GEO’s Full Record Format.

Until such time as an NGDF Profile is deemed sufficiently desirable for the work to be done, all NGDF applications will utilise a subset of GEO. Any reference to an ‘NGDF Profile’ is therefore to a virtual subset of GEO; applications, clients and targets have no knowledge of any NGDF modifications to the Profile, nor any understanding of why a large number of GEO elements are always left empty. The one limitation in this subset approach is that a few NGDF metadata attributes (i.e. Status of Start Date of Capture and Status of End Date of Capture, Alternative Title and level of Spatial Detail) cannot be supported as there is no appropriate mapping to the GEO attribute sets. Until an NGDF Profile is defined these optional attributes will be excluded from the protocol.

Users of the NGDF Guidelines will also need to ensure that the bounding rectangle details are provided in latitude and longitude if they are to be suvccessfully searched by the NGDF Gateway. To successfully utilise the spatial searching capacity of the NGDF Gateway, using the Z39.50 protocol, there is a requirements that all coordinate descriptions be provided in a common base (e.g. latitude and longitude).