You are currently viewing a snapshot of www.mozilla.org taken on April 21, 2008. Most of this content is highly out of date (some pages haven't been updated since the project began in 1998) and exists for historical purposes only. If there are any pages on this archive site that you think should be added back to www.mozilla.org, please file a bug.

Skip to main content

Mozilla Address Book “HotSync”

Rajiv Dayal

Reviewed by: MScott, David Bienvenu and Mail team

Background

 

AB HotSync is the terminology used for synchronizing the desktop Address Book (in our case Mozilla FAddress Book) with the Address Book on the Palm handheld device. The Palm OS comes with a built in application for the Address Book.

 

Depending on the state of both the Palm Address Book and the Mozilla Address Book database a decision would be made to either do a slow sync or a fast sync. A slow sync is the process of synchronizing the entire database whereas fast sync is the process of synchronizing only the changes.

 

Further either a one-way synchronization, i.e. only a download from Palm AB to Mozilla AB or only an upload from Mozilla AB to Palm AB, or a two-way synchronization, resulting into a mirror image of data on both the Palm AB and the Mozilla AB, can be done.

 

The HotSync Process

 

The HotSync Process can be described as the following steps:

  • The user connects the Palm device to the desktop by placing the device into its cradle and connecting the cradle either to the desktop’s serial port or the USB port. The cradle can be connected even via LAN and also synchronization can be done using an IR port if the desktop has one. Palm HotSync Manager takes care of communication with the ports.
  • The Palm HotSync Manager needs to be installed, configured to use the appropriate port and running on the desktop.
  • When the user initiates the HotSync process by pressing the HotSync Button on the cradle, the HotSync Manager on the desktop loads synchronization Conduit programs registered with it. An AB Conduit can be developed by the Address Book vendors to work with their AB using Palm CDK.
  • The AB Conduit when it gets loaded will do the verification to decide whether it needs to do an upload, download or a mirror-image synchronization as well as decide if it needs to do a slow sync or a fast sync. In order to do a fast sync the AB database needs to maintain a modification/status flag for the modification done to the record and the last modified time stamp.
  • The AB Conduit will interact with the AB database on the handheld and the desktop and do the data conversion from Palm AB record to the desktop AB schema and vice-versa and update (modify / delete / add) the AB database on the handheld and the desktop depending on the synchronization that needs to done.

The result of the above steps is identical Address Books on both the desktop and the handheld.

 

AB HotSync in Communicator / Netscape 4x

 

Netscape Communicator provides support for its Address Book synchronization with Palm. Besides customizing the Address Book Conduit for use with Communicator AB, a separate Dll (nab32.dll) for making the calls from its Conduit loaded by the Palm HotSync Manager to the Communicator process is provided.

 

The main emphasis in N4x has been the NAB Conduit, NAB API for Windows provided as a separate library nab32.dll and use of IPC mechanism similar to one used for MAPI implementation in N4x.

 

The N4x Conduit keeps a backup / archived local table of the AB from the last sync and during a new sync it loads it and synchronizes with the current AB data and in the process finds the updated data. The N4x conduit uses the IPC NAB API to interact with the Communicator to retrieve the current AB data record by record. The N4x then uses the Palm Conduit Framework to obtain the modified data from the handheld for the corresponding category. For each handheld record retrieved it compares the corresponding entry in the local table and based on the changes it then updates the handheld category or the NAB.

 

When the NAB Conduit makes the NAB API calls, Nab32.dll does the data conversion as per the Communicator data structure and puts the data into shared memory. Nab32.dll then does a SendMessage with a request id for the NAB API call and the name for the shared memory to a hidden window part of the Communicator IPC hook. The hook ‘s hidden window procedure on receiving the message gets the data from the shared memory and processes the NAB API function based on the request id. Since the hook is part of the Winfe module of communicator, it then directly uses the functions and objects of Communicator Address Book module.

 

The major issue with the N4x conduit implementation is the IPC NAB APIs, these APIs retrieve data from the NAB module in the remote Communicator process record by record. If a large database exists this could mean several IPC calls to get the data and populate the local table. Also the IPC mechanism needs to do memory management for the shared memory.

 

 

Requirements for Implementing the MozAB Hotsync

 

The basic requirement for implementing the Moz AB HotSync is to develop a conduit for interaction between the Palm Address Book and the Mozilla Address Book. The detail requirements for implementing this conduit are as follows:

 

  • The Conduit should get loaded when the user presses the HotSync button, this requires that it should be registered with the HotSync Manager. The user should have the flexibility to register and un-register it so that the desirable AB application can be used. This would require an install program to be developed.
  • The Conduit should either use the Palm Conduit Framework or the Sync Manager APIs to interact with the Handheld and retrieve the modified Address Book records.
  • The Conduit should be able to interact with the Mozilla Address Book. Since the Conduit gets loaded in the HotSync process space it should either be able to access the database used for storing Mozilla AB or should be able to interact with the Mozilla AB module using an IPC mechanism. An IPC framework needs to developed/reused for this.
  • In order for Conduit to access the Mozilla Address Book it would require to load/execute Mozilla if not already running.
  • The conduit should decide what needs to be done for a slow sync or a fast sync
  • Data Conversion needs to be done from Palm Address Book type to Mozilla Address Book type and vice versa to update one with the changes in the other.
  • Optimized method for data comparison between the Palm AB Category and Mozilla AB and its records needs to be developed.
  • Palm user and Mozilla Profile should be appropriately related and a basic authentication should be provided.
  • Synchronization logic needs to be developed to take care of all the following cases:
    • A Handheld is synchronized with one desktop and then with another
    • Category added on handheld
    • Address Book added on desktop
    • Record counts differ for Category and related Address Book
    • Category Ids differ but category desc and Address Book desc match
    • Record modified on handheld only
    • Record modified on desktop only
    • Record modified on both, different fields
    • Record modified on both, same fields
    • Record modified on either handheld or desktop and doesn’t exist on other
    • Record deleted on handheld only
    • Record deleted on desktop only
    • Record deleted on either desktop or handheld and updated on the other
    • Record deleted on both handheld and desktop
    • Record added on handheld only
    • Record added on desktop only
    • Record added on both desktop and handheld with different data
    • Record added on both desktop and handheld with same data
    • Record marked as secret on handheld
  • Optimization should be done in order to reduce the data passing between the handheld and the desktop as well the IPC calls between the conduit and Mozilla Address Book
  • Optimization should be done in order to reduce the usage of heap / memory, since Mork is in-memory database having an Mozilla AB open and also its data in either shared memory and the other process as well, will lead to excessive need of system memory.
  • Existing AB should be backed up for recovery in case of failure.

 

The above can be used to develop the test plan (test cases) for testing this feature.

 

 

Approach for Mozilla AB synchronization with Palm AB

 

Synchronization of Mozilla Address Books on desktop and the handheld can be approached in the either of the following ways:

 

  • Implement a Mozilla Address Book Conduit that uses the Palm Conduit Framework to communicate with the handheld and directly uses Mork to interact with the Mozilla Address Books. This would require a standalone build of Mork (requires xpcom and strings only) and a read only access to the prefs.js file in order to find the list of Mozilla Address Books. This would provide the most efficient way to handle the synchronization in terms of speed however would require more work since it can not take advantage of the Mozilla Address Book interfaces to access Mork. Also would require the code for parsing prefs.js. And the biggest issue is: what if Mozilla Address Books start using other database instead of Mork.

 

  • Implement the NAB APIs same as for Communicator and use most of the work done in the N4x conduit. However this would mean more work for implementing the IPC framework like the one in Communicator. We can though implement our own version of Nab32.dll using the IPC framework using MS COM like the one we used for MAPI and implementing NAB API support classes in Mozilla. However NAB APIs provide a record by record access interface which could effect performance if the Address Book is a large address book like the ones resulting from downloads of LDAP Address Books. Also the data structure for Mozilla Address Book needs to match the one for Communicator Address Book in order for the data conversion and synchronization logic to Palm in the N4x Conduit to work.

 

  • Implement a MozAB Conduit that will use the Palm Conduit Framework to communicate with the handheld and use MAPI to interact with the Mozilla Address Book. This would require extending MAPI Support in Mozilla to implement IABProvider MAPI interface for being an Address Book provider and the IAddrBook and IABContainer MAPI interfaces to access the Address Book data. Not all the methods of these interfaces need to be implemented though. The advantage is that MAPI is more widely used and would also support any MAPI client applications willing to access Mozilla AB. The disadvantage is the complexity of using the MAPI interfaces since it would require implementing / converting to the other data structures used by these MAPI interfaces.

 

  • Implement a MozAB Conduit that will use the Palm Conduit Framework to communicate with the handheld and use custom APIs optimized for our use. The IPC mechanism used will be MAPI framework we have. Since Mork is in-memory database, reading the data and storing it in memory (local tables) in the conduit will duplicate the data in the two processes, Mozilla and HotSync Manager (conduit). Rather if the synchronization logic is implemented in the Mozilla side it would be more efficient. Also the Mozilla string classes and other utility classes can be used for data conversion. Only the data from the Mozilla AB that needs to be updated in the handheld would be sent to the conduit.

 

Efficiency and complexity of implementation should be main criteria to decide which alternative is the best approach.  Based on these criteria the last alternative, MozABConduit using optimized MozAB IPC APIs, looks the most promising.

 

MozABConduit using optimized MozAB IPC APIs

 

The MozABConduit will use the Palm Conduit Framework to interact with Palm and retrieve the Palm data. Code for either 4x conduit or conduit samples can be reused for this part interacting with the Palm. Data is converted to Mozilla AB data structure for passing data between the Conduit and Mozilla. Data conversion is done in the Conduit so that any other type of AB Conduit can reuse the MozAB IPC APIs as it is.

 

For interaction with Mozilla, MozAB IPC APIs needs to be defined. Either the MAPI support IPC framework can be extended or a similar interface using MS COM can be defined. Defining new interface however would mean code required for its registration with Windows Registry done during Mozilla installation as well as the XPCOM wrapper around it so that module registration is automatic during startup.

 

If we reuse the MAPI support IPC framework, we can extend the nsIMapi interface in Mozilla and add three new methods:

GetAddressBookList(out catList),

SynchronizeAB(in catId, in abName, in hhRecList, out pcRecList), and

GetAllDataFromAB(in catId, in abName, out pcRecList)

 

The diagram below depicts the overall architecture:

 


 


When MozAB conduit gets loaded it will Login using a user profile, done using the MAPI framework Login. It will then call GetAddressBookList (above) to get Mozilla AB list and ObtainRemoteCategories (Palm Framework) to get list of Palm AB categories and will displays a dialog box to the user to select the AB to synchronize.

 

For the selected AB that exists on the desktop only GetAllDataFromAB  is called. For selected category on handheld only or for one with corresponding AB on desktop, SynchronizeAB is called with modified data from handheld for the category selected. PalmHotSync support module in Mozilla will then do the synchronization and return the modified data from Mozilla AB to the MozConduit which will then use the Palm Conduit Framework to update the Palm AB category.

 

Synchronization Logic

 

Implementation of the Synchronization Logic would require some changes to our Address Book code, however these changes would be minor. Since it is required to find which records are to be updated on the handheld or the desktop we need to store the corresponding Palm record id and category id as well as the last updated time stamp for each record in Mozilla AB. This can either be stored as part of the AB or in a separate sync table for each AB. The last HotSync time for each Mozilla AB should also be stored in the preferences file.

 

The Synchronization logic can be developed similar to the one in N4x that requires maintaining a status flag for each record and the data comparison with the handheld record is done in the conduit. All records from the last sync is stored into a local table. Also since the synchronization logic is part of the Conduit all the records from the NAB are read into the conduit record by record via an IPC call to sync the local table before beginning the Palm sync.

 

The AbSync synchronization logic can also be used. This logic keeps a history file of the last sync operation that contains the record Ids, status flag and the CRC. It then generates the CRC for each record in the current local AB and compares the CRC of the matching record in the history file. Similarly it compares each record from the remote server AB. The missing or changed (un-matching CRC) local records are used to update the remote server AB and the changed remote server records are used to update the local AB.

 

An alternative logic for synchronization using last updated time stamp is as follows:

  • On receiving the SynchronizeAB call, the HotSync module will get the last records with LastUpdatedTimeStamp > LastSyncTimeStamp or  = 0 from the MozillaAB into a pcList
  • Records Deleted after LastSyncTimeStamp are retrieved from an archive file, only record id, category id and deleted time stamp is stored in the archived file, into the pcList
  • Each record in the hhList is compared with the matching record in the pcList if any. If it does exist the fields are compared, if different fields are modified the changes are merged into the two list, if same fields are modified oops, log / ask user / pick one?
  • All the cases mentioned in the Requirements topic in this doc needs to be taken care when this comparison is done.
  • The result of the above comparison will be two lists, hhList to be used for updating the Mozilla AB and pcList returned back to the MozABConduit for updating handheld AB.

 

This alternative algorithm is more efficient than N4x algorithm since it does not duplicate the Mork AB data in both Mozilla and the Conduit. It is more efficient than the ABSync algorithm since it does not need to calculate the CRC for each AB record.

 

Mozilla interfaces used

 

Mozilla provides the following interfaces that can be used for interacting with the Address Books and directories in Mozilla:

 

  1. nsIAbDirectoryFactoryService to enumerate the directories and address books
  2. nsIAbDirectory / nsIMDBDirectory to create or access a directory, mailing list or address book object in the DB
  3. nsIAbDirectorySearch / nsIAbDirectoryQuery to search cards based on a criteria / search expression.
  4. nsIAddressBook to create and access an address book
  5. nsIAddrDatabase to the update and read each Mork based address book data
  6. nsIAbCard / nsIMDBCard to update and read each address book record field
  7. nsIFile / nsILocalFile for backups

 

Authentication

 

User will be asked to choose the right profile for the Mozilla AB to be updated, if Mozilla is not already running user will asked to confirm for the current logged in profile to be used for AB synchronization. Password based authentication is not provided as Mozilla AB doesn’t support it.