You are currently viewing a snapshot of www.mozilla.org taken on April 21, 2008. Most of this content is highly out of date (some pages haven't been updated since the project began in 1998) and exists for historical purposes only. If there are any pages on this archive site that you think should be added back to www.mozilla.org, please file a bug.

Skip to main content

Seamonkey Mail/News API Testing

Last Updated: June 5, 2000
Author: Par Pandit

Table of Contents

Introduction

This document will describe the current test plan for testing the Mail/News API for Netscape 6. There have been two previous test plans which are now obsolete.

The API calls use a scripting language to access the type library which we will access the C++ mail/news code. The goal is to solidify the mail/news protocol (NNTP, SMTP, IMAP, POP, MIME) and storage mechanism (database, address book).

XPConnect allows access into the functionality of the Mozilla and Netscape 6 mail client by API calls written in javascript and html. The methods and variables that are accessible and therefore testable reside in idl files. Third party developers can access mail contents and functionality using the interface calls available in these idl files.

Test Plan Strategy

The basic strategy is to test the API by stages. Stage one will consist of creating simple testcases which simply access the most basic functionality of an interface. This may be just a simple test to verify that we can access the attribute or call the method and return safely. Or it may be something more substantial such as being able to read the header of a message. The goal is to know if an external developer wanted to access that interface and its contents he should be able to. The Test Case Status link below contain all the tests needed for stage one.

The second stage will the development of more detailed interface tests. Instead of one simple call, there will be several testcases for one method or attribute. These tests will including trying different parameters, forcing error conditions, and verifying that third party developers can use the interface calls with full confidence. The second stage will be a major undertaking that will requires more resources and time which are not addressed in this testplan. These tests will need to be developed during further releases of Mozilla and Netscape 6.

Each testcase will be written with a similar design which allow it to be loaded and run independently from all other testcases. This design will incorporate code that is necessary to work with QA department's reporting format. It will also include a specific header that is approved for posting on mozilla.org

Currently there are 120 interfaces under the mozilla/mailnews tree that contain 1212 scriptable attributes or methods that can be tested. We are estimating one day per test - which includes development, test, and incorporation into NGDriver as well as checkin into the mozilla.org tree. This is highly variable as some test cases will take 1/2 day and others could take an entire week or more to understand and write a testcase for.

Test Environment and Tools

Software
  • Visual C++ 6.0 will be used for Windows NT platform
  • Linux - Red Hat version 6.0 and higher
  • JavaScript - testcases will be written in JavaScript and HTML.
  • NGDriver - Browser team software and setup needed to execute tests in automation mode. (not on mozilla.org at this time)
  • A JavaScript debugger is being developed for Mozilla that will be useful when ready.
Hardware
  • Each tester will have access to a Windows NT system and a Linux machine within their cubical. Each machine will maintain a debug build of Seamonkey. These machines will be used for development of the testcases
  • Testing by QA API Team will be done in two stages. Testcases will initially be run on Windows NT system after development. Second, the same testcase will executed on other supported platforms using lab and desktop machines.
Data
  • There will be prepared messages for some tests which will be located in accounts within Netscape servers. These servers are internal to Netscape only but we will copy the messages into external mail accounts eventually.
Platform
  • Since API testing is primarily backend testing, a testcase developed and tested on one platform should in theory be the same on all platforms.
  • Development of testcases will be primarily on Windows NT
  • Execution of the testcases will be on Windows NT and Linux
  • If time is available, then also run on Macs

Control Procedures

QA will use bugzilla for reporting bugs. This is the accepted convention used by Netscape and Mozilla and we will follow it.

Each testcase filename will be based on the following fields concatenated together. They are directory + interface name + testcase number. For example, baseMsgWindow01.html.

The testcases will be available in two locations. Within Netscape, you can execute a single testcase or a suite of testcases using NGDriver on bubblegum. Bubblegum is an Windows based personal computer internal to Netscape only. NGDriver is not available on mozilla.org yet but the same testcases will be available at http://www.mozilla.org/quality/ngdriver/suites/mail. These external testcases must be loaded individually rather than in a suite of testcases.

At the present time, the project is set up by Netscape mailnews client qa group. All design and implementation is being conducted by the group's API engineers. All approvals and reviews have been conducted thus far within mailnews QA. We would like to solicit mozilla.org contributors to help us with creating and executing these tasks.

Risks and Assumptions

There are several potential risks that need to be taken into account:
  • Changing code - Mozilla is becoming more mature but developers can and do change the IDL files and what interfaces will allow access to. They can also change the behavior of an interface which may require rewriting the testcase.
  • Security issues - There is now code available to allow javascript to access Messenger functionality. Not all issues have been resolved regarding security so there could be a need to rewrite testcases
  • XUL - Currently the testcases are written in HTML. There may be need to convert testcases to use XUL
  • Learning Curve - We will need time to learn any specific knowledge and/or code in order to write testcases. Examples could include security, signed scripts, mime,
  • Untestability - Some methods that take DOM, RDF as parameters cannot be tested from JS.
Some of the assumptions we are making are
  • The code base is stable - if the development tree for Mozilla remain unstable (i.e. red color on the tree) for any extended period of time, it will affect our development and testing
  • The schedule assumes an ideal environment with no turnovers and QA engineers contributing 1.5 days of work per day.

Schedule

Task Task Number Purpose Expect to Start Status
Stage 1 
All Methods and Attributes Are Accessible and have been tested once		 1 These tests should access and verify a function call and return value for the most basic test possible for method and attribute within an interface   In Progress. 0.0% done
  1a Base directory. 
456 testcases. Estimated 60.8 weeks		 3/1/2000 In Progress. 5.0% done
  1b Compose directory. 
109 testcases. Estimated 14.5 weeks
In Progress. 0.0% done
  1c Addrbook directory. 
154 testcases. Estimated 20.5 weeks
In Progress. 0.0% done
  1d Imap directory. 
65 testcases. Estimated 8.7 weeks
In Progress. 0.0% done
  1e Local directory. 
39 testcases. Estimated 5.2 weeks
In Progress. 0.0% done
  1f News directory. 
151 testcases. Estimated 20.1 weeks
Not started. 0.0% done
  1g Import directory. 
75 testcases. Estimated 10 weeks
Not started. 0.0% done
  1h Db directory. 
93 testcases. Estimated 12.4 weeks
Not started. 0.0% done
  1i AbSync directory. 
17 testcases. Estimated 2.3 weeks
Not started. 0.0% done
  1j Mime directory. 
47 testcases. 6.3 weeks
Not started 0.0% done
Stage 2 
Comprehensive Testing		 2 Thoroughly test all methods and attributes within major interfaces Unknown Not Started
Stage 2 
Complete Comprehensive Testing 2		 5 Thoroughly test all methods and attributes within all interfaces Unknown Not Started
Test New Features 7 Thoroughly test all methods and attributes within all interfaces Unknown Not Started

To see who is assigned to which testcases see the following link
http://www.bitlocker.com/site?c=30&r=2209193738109948&ts=959625135904

Other Projects

In addition to the tests described in this document, the QA team also writes testcases that are related to API that are used by other teams. The release team uses POP, IMAP, and NEWS testcases developed by this team. The international QA team has expressed interest in helping with some of the testing. This is mentioned as factors that can and will impact the testing schedule. 

People Involved in the Project

The following people are involved in API Testing project
  • Lisa Chiang - Manager of Netscape Mail/News QA
  • Par Pandit - QA API Lead (90% of time allocated)
  • Any mozilla.org members who wish to participate (none at the moment)
  • Any intern available to the mailnews QA group

    • Exit Criteria

    There is no designated exit criteria for the initial release of Netscape 6 or Mozilla 1.0. It will ship regardless of the condition of mailnews API code. It is safe to assume that most of the functionality does work because mail could not work properly if the XPCOM and XPConnect interfaces were not working properly.

    For Netscape 6.1 and the next version of Mozilla, an exit criteria should be set. This should include a goal of 50 to 100% of stage one testing. This should include an approval list to be signed off for.

    Useful Links and Documentation

    Mozilla.org Links O'Reilley Network Bitlocker Database Recommended Books Usenet Newsgroups
    • netscape.public.mozilla.builds
    • netscape.public.mozilla.jseng
    • netscape.public.mozilla.mail-news
    • netscape.public.mozilla.rdf
    • netscape.public.mozilla.xpcom
    Get the Current Release Build