You are currently viewing a snapshot of www.mozilla.org taken on April 21, 2008. Most of this content is highly out of date (some pages haven't been updated since the project began in 1998) and exists for historical purposes only. If there are any pages on this archive site that you think should be added back to www.mozilla.org, please file a bug.

Skip to main content

Mozilla RDF / Z39.50 Integration Project

Contacts:
Dan Brickley (daniel.brickley@bristol.ac.uk)
Ray Denenberg Z39.50 Maintainance Agency, Library of Congress (rden@loc.gov)
Eliot Christian GILS initiative, (echristi@usgs.gov)
Sebastian Hammer Index Data (quinn@indexdata.dk)
Chris Waterson (waterson@netscape.com)

This is a project to investigate the integration of Z39.50 search capabilities with Mozilla's RDF-based information management environment.

Aims

  • Make Z39.50 data sources accessible for searching from within Mozilla
  • Find an RDF representation of Z39.50 attribute sets

There are thousands of networked Z39.50 servers in existence already. It should be possible to identify some mechanism whereby the Mozilla user interface allows people to send queries to these servers and have the resulting records appear within the standard Mozilla bookmarks/sitemaps interface.

Z39.50 Background

ANSI/NISO Z39.50 is a communications protocol for information retrieval in a client/server environment. Widely used in bibliographic and digital library applications, Z39.50 represents a significant body of experience relating to bibliographic data retrieval and the specification of interoperable semantics for community-specific metadata attribute sets. The Z39.50 Integration Project outlined here is intended to identify appropriate mechanisms that will allow us to exploit Z39.50 data sources and metadata attribute sets from within Mozilla's RDF-based 'Aurora' environment.

Getting Started

This page is intended to provide a useful set of resources from which contributors from the Z39.50, RDF and Mozilla communities can make a start on addressing common problems. Z39.50 can be rather complex for those unfamiliar with the protocol, but there are many developers out there who have considerable experience in this area. What we really need is a good introductory document aimed at outside contributors wanting to hack new functionality into Mozilla in the most sensible way possible. Some of the docs linked from the Mozilla RDF pages make a start in this direction. A good way to get started is simply to download one of the early-release copies of Mozilla and take a look at the way it works.

For Z39.50 developers unfamiliar with Mozilla...

If you understand Z39.50, you'll probably find this easy... The entire Mozilla UI is built from "XUL" files, Javascript snippets and calls to XP-COM interfaces for services such as RDF query and aggregation. Read up on the Cross Platform front end, which describes how the user interface is specified in XUL. XUL is an XML application that allows interfaces to be built using XML, HTML, Javascript and RDF. The XUL Template Reference describes in some detail how XUL widgets (trees, menus) are populated by querying RDF data graphs. You should read up on the formal RDF specs at the W3C RDF home page and explore the various RDF-in-Mozilla links on this site. In Mozilla, most interesting data sources manifest themselves to the user interface through the RDF graph abstraction. There are a number of options to explore regarding how we might get data from a Z39.50 server into such a graph and from there into XUL-based user interface elements. See the Datasource HOWTO for a step by step guide for implementors of new RDF datasources.

The basic recipe is as follows: we compile our component (on as many platforms as possible), implementing the RDF DataSource APIs and any additional XP-COM APIs we care to define (perhaps digging out some of the IDL for Z39.50 references floating around). We drop the files into the 'components/' directory of Mozilla and when the browser starts up, it'll inspect the new component, which will dutifully describe itself so that it can show up in the Javascript, RDF and XUL world from which Mozilla is built (see also the Aug'99 article in WebReview.com for a high level overview of the Mozilla architecture).

Here's the basic approach to calling using components from Javascript. They each have a name (eg. 'component://z3950service') and satisfy one or more scriptable interfaces. The fictitious snippet below shows the initialization of a Z39.50 object that can be treated as if it were an RDF graph, while also offering a more Z39.50 native interface.

 var z3950 = Components.classes["component://z3950service"].createInstance();
 z3950 = z3950.QueryInterface(Components.interfaces.nsIRDFDataSource);
 zapi = z3950.QueryInterface(Components.interfaces.IZ3950Service);

What happens next? Firstly, it is important that people don't go away and expend loads of effort working on something that may not fit in with the bigger picture sketched in the Mozilla development roadmap and on the Mozilla RDF pages. The best way to ensure this doesn't happen is to float ideas on the mozilla-rdf forum (see the Community section for details on accessing this as newsgroup or mailing list).

Prototypes

This should probably be what happens next! It should be relatively simple to put something together (eg. CGI script Z39.50 client outputting an XML/RDF file) which shows what a Mozilla Z39.50 client might look like. Once this is done, it'll be easier to appreciate the issues involved and figure out what we'd want from a full Z39.50 datasource (see below). For example: do we want to be able to store query results as bookmarks? Can we simply use the Dublin Core RDF vocabulary to represent these data structures?

Notealso that the M9 Mozilla milestone includes some non-Z39.50 internet search functionality (based on 'screenscraping' results from HTML pages returned by search engine queries).

Roadmap for a Z39.50 RDF datasource

If native client-side support for Z39.50 is to be integrated into Mozilla, we'll need to find code that implements a Z39.50 client and can be wrappped up as an RDF data source in Mozilla. It looks as is we can use the Yaz toolkit from IndexData, which will give us a pretty solid base for the Z39.50 client code (Yaz ships under a liberal license whose main constraints concern attribution statements and 'no warranty implied'). The main substantive piece of work is likely to be gluing Yaz to the RDF APIs and thinking through the different ways in which the RDF and Z39.50 data models might be related.

Open Issues

This should evolve into a more structured issues list on a separate page; for now, a freeform brainstorm seems more appropriate.

What is Mozilla RDF's notion of a 'searchable thing'? What other networked search protocols will also be supported in Mozilla? Is it feasible to consider adding full Z39.50 support into the client (eg. through contrib'd library?) or will tunnelling over HTTP (via XML/RDF or otherwise) prove to be a more realistic option? What constraints are there on code contributed by 3rd parties? How exactly do we set about hooking up externally authored libraries (eg. Yaz) as XP-COM plugin datasources?

IssueSummaryStatus and notes
Yaz library Can we get confirmation that Yaz license can be used with Mozilla? Need advise from Mozilla org; This appears to be a Frequently asked question. It looks like we don't have a problem here...
Service description We'll need an XML/RDF representation of the properties of various Z39.50 services (connection details, subject coverage etc) The GILS Locator Records (perhaps munged into RDF and Dublin Core?) seem a good basis for this. Important to coordinate with descriptions of SmartBrowsing and other RDF data services in Mozilla. Need to find out how this works in rest of Mozilla (eg. localised configuration of cache services, use of CSS/XUL for icon sets, chrome etc...). Also relates to RDF Sitemaps: a sitemap for some Web service should probably be allowed to contain RDF/XML describing Z39.50 search targets (and other machine interfaces) where available (need docs on RDF Sitemaps). Where does Necko fit in? Necko is the new Mozilla networking library. Connections to networked servers should go via Necko. It's not immediately clear how to 'do the right thing' here. The Necko overview helps a little but we really could do with some example code to copy... Ideally we set about writing a protocol handler associated with Z39.50 URLs. We also have the notion of Z39.50 tunnelled over HTTP to bear in mind, which might crosscut such developments...
Non-Z39.50 searching There are likely to be non-Z39.50 search services accessible via RDF/XUL, eg. portal searches, 'find' in bookmarks. Important to avoid needless duplication, and to get a roadmap for such work from Mozilla organisation to plan/coordinate etc. Bug 11347 against M9 has been filed as a reminder that the roadmaps need updating. Best way to avoid duplication is to keep discussion on the public forums, and read back through past threads on RDF search in the newsgroup.
BER utilities Z39.50 uses ASN.1 and BER encodings; Yaz provides code for dealing with BER. However, Mozilla's LDAP library will also have BER tools. What best to do? Find out whether the LDAP BER utilities be explosed via XP-COM or otherwise usable. Does the duplication matter that much - versus added complexity of liaising with LDAP work. Aside: is an LDAP RDF datasource still planned?
From Sebastian: If we can build it all as a run-time loadable module, then the overhead of carrying two BER stacks around is less significant. I believe the size of the BER code itself is smaller than the higher-level layers which encode the actual, structured Z39.50 (or LDAP) PDUs.
[Status: CLOSED]
General consensus that is better to use the Yaz BER libraries

Discussion

See the main Mozilla RDF pages for links to relevant lists/newsgroups.

Resources

Z39.50 reading

Mozilla reading

Disclaimer

This is an exploratory project, and an experiment to see whether we can make it easy for the Z39.50 community to contribute their expertise to the search environment under construction for Mozilla. It does not constitute any commitment to ship Z code with the '5.0' release, nor a promise that a great deal of engineering resources can be poured into this problem. This probably doesn't need stating, since Mozilla isn't NSCP/AOL. Basically, it's likely that some XML/RDF-based client/server metadata search functionality will appear in the Mozilla browser. Hopefully, this will be informed by and compatible with existing Z39.50 implementations. But we need your help...

Author: Dan Brickley daniel.brickley@bristol.ac.uk
Last Updated: $Id: z3950.html,v 1.7 2003/02/21 19:38:59 timeless%mozdev.org Exp $