You are currently viewing a snapshot of www.mozilla.org taken on April 21, 2008. Most of this content is highly out of date (some pages haven't been updated since the project began in 1998) and exists for historical purposes only. If there are any pages on this archive site that you think should be added back to www.mozilla.org, please file a bug.

Skip to main content

You are here: Browser/Composer front-end QA > front-end test plans > Plugins Test Plan

Client QA Component Test Plan: Seamonkey Project

Plugins

Written by Shrirang Khanzode
Email shrir@netscape.com

DOCUMENT CHANGE HISTORY

1. COMPONENT

  • Description

2. AREAS TO BE TESTED

  • Component Features
  • Related Areas
  • New Features For Mach V Release

3. AREAS NOT TO BE TESTED

  • Undocumented Features

4. APPROACH

  • Strategy
  • Automation
  • Acceptance Testing
  • Functional Testing
  • Usability Testing
  • Stress Testing
  • Regression Testing

5. RESOURCES

  • Software
  • Hardware
  • Data
  • Lab Time
  • Other Tools
    • Network
    • Remote Connection
    • Off-line

6. CONTINGENCIES

7. SIGNOFF APPROVAL

8. REFERENCE MATERIALS

  • Schedule link
  • Specifications link
  • Supported Platforms link
  • Automation

Document Change History:

History
Date Change Description Revision Updated By
08/10/1999 Initial Draft 1.0 Erik Krock
08/31/1999 Updated for LiveConnect, handed off to QA for further development 1.1 Erik Krock
09/07/1999 Added doc link, new test case 1.1 Erik Krock
07/18/2000 Updated for mozilla.org 1.2 Shrirang Khanzode
12/19/2000 Added testcase links and some updations 1.3 Shrirang Khanzode

* Note - This Test Plan is a "living document"; it can be continually modified to conform to changes during the project.

1.Component

Plugins functionality must be tested across all supported platforms(subject to plugin availability). It is not the responsibility of Netscape QA to verify that third-party plug-ins are inherently bug-free; this is the responsibility of the plug-in vendors. However, the plug-in API has been rewritten in Nav6 with the following goals (testable propositions):

(A) Existing Nav4 generation plug-in binaries which functioned in Nav4 should continue to function in Nav6.

(B) Corollary to A: plug-in content which could be viewed using a Nav4 generation plug-in should also be viewable in Nav6 using the same plug-in binary.

Exception to A and B: If plug-in content makes calls to the plug-in's Java/native interface from JavaScript via LiveConnect, these calls will fail silently when the Nav4-generation plug-in binary is used within Nav6.

(C) Once a plug-in has been upgraded to support the Nav6 plug-in API the upgraded plug-in binary should be 100% functional (for both ordinary and LiveConnect content) in Nav6.

(D) Once a plug-in has been upgraded to support the Nav6 plug-in API including support for the backward adapter, the upgraded plug-in binary should be 100% functional (for both ordinary and LiveConnect content) in both Nav4 and Nav6.

Here are the combinations represented as a table:

plugin binary designed for this browser version: Browser Version (Nav4) Browser Version (Nav6)
Nav4 n/a Case (A): Nav4 plug-in running in Nav6 works except for LiveConnect calls.
Nav6 Case (D): If upgraded plug-in includes backward adapter, plug-in binary should work perfectly including LiveConnect calls in Nav4. Case (C): Upgraded plug-in running in Nav6 works including LiveConnect calls.

Netscape must verify that the Nav6 plug-in architecture has achieved goals A-D. We will do this using a representative sample of popular plug-ins and content. Once we have proven that the architecture indeed enables goals A-D as intended, plug-in vendors will be able to upgrade their plug-ins on their own.

2.Areas To Be Tested

Component Features

  • plug-in launching in external window as a helper app (e.g. RealPlayer in its own window) vs. plug-in embedded within the page (e.g. RealAudio content embedded in a web page and playing without launching an external window)
  • when embedded within a page
    • EMBED tag for backward compatibility and OBJECT tag for HTML 4.0 compliance
    • standard content, standard content with embedded URLs to display in browser (RealPlayer case), and LiveConnect-enabled content
  • plug-in (RealPlayer, Flash, QuickTime ...)
  • plug-in version (designed for Nav4, designed for Nav6)
  • browser version (Nav4 plug-in running in Nav6, upgraded Nav6 plug-in running in Nav6, Nav6 plug-in running in Nav4 via backward adapter)
  • platform (Win32, MacOS, Linux -- in theory all tests should be performed on all 3 platforms, subject to plug-in availability)

Related Areas :

With the recent addition of different 'themes' to mozilla, all tests for Plugins should be run on all themes. The Plugins QA will be responsible to test plugins under all available themes.

New Features For Mach V Release:

  • Plug-in Finder changes

We are moving toward all Gecko based clients being xpi enabled.

All Netscape based clients up to this point (N 4.x, 6.0 - 6.2.1, 0.9.4 branch) have used a hard coded cgi for the plug-in finder service, which is:
http://cgi.netscape.com/cgi-bin/plug-in_finder.cgi?mimetype=[something ]

Based on this cgi, we have a back-end system that handles mimetype requests and sends users to a variety of places to download their plug-in. It is the goal going forward to make this a much better user experience, so the user doesn't have to surf all around the web to find the plug-in, then download it, then install it, then go back to the original webpage to view the content.

Instead the ideal experience is as seamless as possible to the user:

user clicks on the puzzle piece plug-in is found, downloaded, and installed
original page is refreshed, and embedded content is displayed

Going forward for Mach V, and the two main embedding projects (CS and AOL) we will be using a new cgi program to change the plug-ins program from being focused on sending users to the right web pages, to sending them to the right .xpi binary.

To recap some earlier meetings, some of the things that need to happen for us to get there:

  • Dan Veditz is working on enabling xpi in the embedding trunk. This should be landing soon.
  • Peter Lubczynski is/was going to land some code in the trunk that enables the client to pass additional information besides the legacy info like mimetype, file extension, etc. to the server side script. Most notably:
    • xpinstall parameter
    • plugins url
    • plugins page
    • domain name
    • others?
  • By passing plugins url, plugins page, and domain name, we will have the *option* of not only honoring the content developer's EMBED tags in certain situations, but we would be able to gain extremely valuable tracking data to effectively evangelize the most important plug-in vendors.
    • The best example of when it would be good to honor the content developer's EMBED tag (that I can think of), is if there is a PLUGINSURL that points to an xpi binary with a vendor's domain name that differs from the xpi that we (Netscape/AOL/CS) have in our database file due to a mime-type conflict.
    • The key thing to note is not necessarily the merit or how common an example like the above would be, but rather it gives us lots of options on how we can handle plug-ins requests.
  • New cgi's for all clients after the 0.9.8 milestone, developed by Ray Gerst in Web Development, could be something like:
    • if xpi available, call xpi-plug-in_finder.cgi
      • searches the database file for location of xpi binary and sends user there.
    • else, call no-xpi-plug-in_finder.cgi
      • no-xpi-plug-in_finder.cgi could do a number of things like:
      • send the user to a non-xpi binary (.exe)
      • honor the embed tag
      • send the user to an Netscape/AOL/CS branded webpage with mime-type or file extension search results (similar to existing system)
  • Regular Metrics reporting will be essential to this program's success. Most common plug-ins vary tremendously between Netscape and AOL/CS users. Being able differentiate between the brands and develop the right relationships with vendors will be key. Dave Barrowman put together a good start for data mining opportunities for the plug-in finder service.
  • Creating a tool that wraps a regular binary with an xpi script.
    • For vendors that are slow to adopt xpi or who don't want to create a full blown xpi installer, it would be a very good thing to create a tool that could do this either automatically for them or for us.
    • It would reduce the strain on both the evangelism team, and the engineering team if we need to do this in-house.
    • Who could develop this? Peter L? Dan V? XPInstall team? other?

3.What Will Not Be Tested

Basically, it is the responsibility of plugin vendors to provide upgraded plugins compatible to mozilla 6.0 plugin api. Any plugin other than the major ones like shockwave, flash, realplayer, acrobat, quicktime will not be thoroughly tested and QA'ed. Once a vendor comes up with a mozilla compatible plugin, we can use it to test with mozilla.

4.Approach

Strategy

Plugins testing should basically cover all major plugins identified by Netscape as crucial ones. This includes shockwave player, flash, realplayer, adobe acrobat and quicktime. Since there are no plugins released by the plugin vendors for the upgraded mozilla api, latest versions of all plugins will be used and tested for backward compatibility. Existing binaries will work as-is within Nav6, and existing content which uses the plug-in will work fine. Exception: if the plug-in offers an API callable from JavaScript and content calls that API, the JavaScript call will fail unless the vendor has upgraded their plug-in to support the Nav6 plug-in API and the user has installed this Nav6-ready version. The Nav6 plug-in API enables existing content (including JavaScript calls) to work unchanged in Nav6 as well as previous browser versions.

Plugins will be tested as embedded within the browser window and also as helper applications (Realplayer). Testcases should be written to cover all major plugins identified. These tests should basically cover all the essential functionality of the plugins. Bugs shall be written up in bugzilla for the problem that are observed by QA during testing. Plugins will be tested as per platform availability. We should achieve the goals as mentioned in section 1 of the test plan.

Automation

At present, no automation is being used for testing purposes.

Acceptance Testing

For Shockwave:

For Realplayer:

For Quicktime:

  • Go to www.quicktime.com and play any movie clip

Functional and Usability Testing

RealPlayer:

(A) Existing Nav4 generation plug-in binaries which functioned in Nav4 should continue to function in Nav6.

  • plug-in launching in external window as helper app
  • RealPlayer
  • plug-in embedded within web page
  • via EMBED tag
  • via OBJECT tag
  • standard content
  • standard content with embedded URLs for display within the browser

0) In Nav6, make sure you have RealPlayer version ___ (pre Nav6-upgrade) installed.
1) Open http://blues/users/ekrock/publish/bugs/M9/ecxpert.html
2) Verify that clip begins playing, embedded within browser window.
Note: This test is a simplified test case derived from the next one

1) Open http://media.netscape.com/launch.html#ECXperti
2) Click "Launch ECXpert" button.
3) Verify that clip begins playing, embedded within browser window. (Don't touch the button controls; some of them use LiveConnect, and we test that in a later step.)

1) Open http://media.netscape.com/launch.html#css1
2) Click "Launch CSS1" button.
3) Verify that clip begins playing, embedded within browser window, and that every few minutes the HTML page displayed within the right frame changes. (Don't touch the button controls; some of them use LiveConnect, and we test that in a later step.)

Note: This test can only be performed after RealNetworks releases a RealPlayer upgraded for Nav6.
(C) Once a plug-in has been upgraded to support the Nav6 plug-in API, including support for the backward adapter, the upgraded plug-in binary should be 100% functional (for both ordinary and LiveConnect content) in Nav6.

0) In Nav6, make sure you have RealPlayer G2 version ___ (post Nav6-upgrade) installed. Test the following:
1) Open http://media.netscape.com/launch.html#ECXperti
2) Click "Launch ECXpert" button.
3) Wait for audio to begin playing.
4) Once audio has begun playing, select "Customer Stories" from pulldown menu on left. Verify that presentation continues playing from the "Customer Stories" section. This confirms that the JavaScript LiveConnect call document.audioclip.SetPosition(msec) succeeded.

Note: This test can only be performed after RealNetworks releases a RealPlayer upgraded for Nav6.
(C) Once a plug-in has been upgraded to support the Nav6 plug-in API, including support for the backward adapter, the upgraded plug-in binary should be 100% functional (for both ordinary and LiveConnect content) in Nav6.

0) In Nav6, make sure you have RealPlayer G2 version ___ (post Nav6-upgrade) installed. Test the following:
1) Open http://media.netscape.com/launch.html#css1
2) Click "Launch CSS1" button.
3) Wait for audio to begin playing.
4) Once audio has begun playing, select "Customer Stories" from pulldown menu on left. Verify that presentation continues playing from the "Customer Stories" section. This confirms that the JavaScript LiveConnect call document.audioclip.SetPosition(msec) succeeded.
5) Click "Next Topic" button at left. Verify that presentation continues playing from the "What do style sheets do?" page. This confirms that the JavaScript LiveConnect call document.audioclip.DoNextItem(); succeeded.
6) Click "Last Topic" button at left. Verify that presentation continues playing from the "Introduction to Cascading Style Sheets, level 1" page. This confirms that the JavaScript LiveConnect call document.audioclip.DoPrevItem(); succeeded.

Note: This test can only be performed after RealNetworks releases a RealPlayer upgraded for Nav6 and including support for the backward adapter
(D) Once a plug-in has been upgraded to support the Nav6 plug-in API including support for the backward adapter, the upgraded plug-in binary should be 100% functional (for both ordinary and LiveConnect content) in both Nav4 and Nav6.

0) In Nav4, make sure you have RealPlayer G2 version ___ (post Nav6-upgrade) installed. Test the following:
1) Open http://media.netscape.com/launch.html#ECXperti
2) Click "Launch ECXpert" button.
3) Wait for audio to begin playing. Once it has, select "Customer Stories" from pulldown menu on left.
4) Verify that presentation continues playing from the "Customer Stories" section. This confirms that the JavaScript LiveConnect call document.audioclip.SetPosition(msec) succeeded.

Flash:

As far as Eric Krock knows, Flash content never plays in its own standalone window as a helper app, so it's not necessary to test that as it is for RealPlayer.

(A) Existing Nav4 generation plug-in binaries which functioned in Nav4 should continue to function in Nav6.

  • plug-in embedded within web page
  • via EMBED tag
  • via OBJECT tag
  • standard content
  • Flash

0) In Nav6, make sure you have Flash version ___ (pre Nav6-upgrade) installed.
1) In Nav6, open http://www.fridgefashions.com/f4/movie.html
2) Verify that the Flash animation plays and the picture flies up on to the refrigerator.

JavaScript LiveConnect Clock Test - on Nav6
Note: This test can only be performed after Macromedia releases a Flash plug-in upgraded for Nav6.
(C) Once a plug-in has been upgraded to support the Nav6 plug-in API, including support for the backward adapter, the upgraded plug-in binary should be 100% functional (for both ordinary and LiveConnect content) in Nav6.

0) In Nav6, make sure you have Flash version ___ (post Nav6-upgrade) installed. Test the following:
1) Get the demo JavaScript Flash clock; it can be found at these two places:

2) Unzip the clock and put it into a directory called "clock".
3) Open clock/Clock1.html
4) Click "Initialize Music Button" button. On the "Initialize Music Button" alert, click "OK." Verify that "Music On" button appears above.
5) Click "Music On" button. Verify that music begins playing.
6) Click "Music Off" button. Verify that music stops playing.
7) Mouseover the word "Move" in "Move mouse horizontally in here to adjust clock speed". Verify that clock hand ticks at a relatively rapid pace. (compare to speed in next step)
8) Mouseover the word "speed" in "Move mouse horizontally in here to adjust clock speed". Verify that clock hand ticks at a relatively slower pace. (compare to speed in previous step)

JavaScript LiveConnect Clock Test - on Nav4
Note: This test can only be performed after Macromedia releases a Flash plug-in upgraded for Nav6 and including support for the backward adapter.
(D) Once a plug-in has been upgraded to support the Nav6 plug-in API including support for the backward adapter, the upgraded plug-in binary should be 100% functional (for both ordinary and LiveConnect content) in both Nav4 and Nav6.

0) In Nav4, make sure you have Flash plug-in version ___ (post Nav6-upgrade) installed. Test the following:
1) Repeat the steps of the "JavaScript LiveConnect Clock Test" above in Nav4, making sure the upgraded plug-in functions as well in Nav4 as it does in Nav6.

JavaScript LiveConnect Hawaii Test - on Nav6
Hint: To View Frame Source of a frame in this demo, place the mouse cursor's tip on the 1 pixel wide white margin around the frame content and then right-click the mouse button to display the frame's popup menu. (If you place the mouse cursor on the content itself, you see the Flash popup menu which doesn't include View Frame Source.)

0) In Nav6, make sure you have Flash version ___ (post Nav6-upgrade) installed. Test the following:
1) Go to http://www.FlashCentral.com/Tech/HawaiiMap/
2) Click "Stop Bird". Verify that bird stops flying.
3) Click "Start Bird". Verify that bird starts flying.
4) Click "Bird 1". Verify that the bird's wings are in the down position.
5) Click "Bird 3". Verify that the bird's wings are in the up position.
6) Click "Forward One Frame" twice. Verify that the wings go forward two steps.
7) Click "Backward One Frame" twice. Verify that the wings go back two steps.
8) Click "Load Loop".
9) Click "Start Loop". Verify that plane starts flying.
10) As plane flies, click "Display Current Frame" several times. Verify that numbers between 1-38 are displayed in the text box.
11) Click "Stop Loop". Verify that plane stops in place.

JavaScript LiveConnect Hawaii Test - on Nav4
Note: This test can only be performed after Macromedia releases a Flash plug-in upgraded for Nav6 and including support for the backward adapter.

0) In Nav4, make sure you have Flash plug-in version ___ (post Nav6-upgrade) installed. Test the following:
1) Repeat the steps of the "JavaScript LiveConnect Hawaii Test" above in Nav4, making sure the upgraded plug-in functions as well in Nav4 as it does in Nav6

Functional testcase suite for plugins can be found on www.mozilla.org here. Also, an exhaustive testcase suite for shockwave player can be found here.

5.RESOURCES

Software

OS/ System Software

Please see the Seamonkey Platforms link at http://client/seamonkey/prd/seamonkey_platforms.html for a description of the hardware/software to be used in testing.

Hardware

Computers

Please see the Seamonkey Platforms link at http://client/seamonkey/prd/seamonkey_platforms.html for a description of the hardware/software to be used in testing

Data

n/a

Lab Time

n/a

Other Tools n/a

  • Network
  • Remote Connection
  • Off-line

6.CONTINGENCIES

Loss of Test Machine Resources

While highly unlikely there is the possibility that Test Machines could break down. Planning should be done, by each component area to ensure there are reserves of equipment to be drawn upon.

Bugs

If outstanding bug counts do not drop and cannot be mitigated between QA, Engineering, and Marketing there is the potential that the project schedule may have to slip. The time increment will be agreed to by the project leads.

Builds

There is the potential for build crisises to arise (tree spammage, etc.) which can impact QA and Engineering getting code to use for their work, and therefore adversely affect the schedule. Some planning should be done in this area in anticipation of the most probable things that could go wrong.

7.SIGNOFF APPROVAL

Name                           Date

QA Lead ________________________ / /

Eng Lead ________________________ / /

Automation Lead ________________________ / /

Project Manager ________________________ / /

Product Manager ________________________ / /

Dir. of Engineering ________________________ / /

Dir. of Quality Assurance ________________________ / /

8.REFERENCE MATERIALS

Contacts

Newsgroups