libimg2 (aka libpr0n) Overview
last update 2/15/2001saari@netscape.com
Motivation
Over the course of several discussions, bug reports, it became apparent that the current libimg implementation was insufficient in several critical ways:
- libimg doesn't cache animations, but rather redecodes each frame of the animation each time it displays. (~10% page load time slow down)
- Redundant retrieval and verification of images from the server is also slowing down page load times. This can be seen in particular on slashdot.org. (XXX get number from jgrm mail)
- The current image cache will become corrupt, returning the incorrect image.
- The current image cache and related structures account for a large percentage (20% XXX verify number) of dynamic memory footprint after loading (100?) pages.
- The number of image structures seems to grow steadily as we load pages, with no apparent bounding.
- We cannot properly support offline printing and full resolution printing of images.
These issues and more are reflected in a list of bugs we intend to address: http://bugzilla.mozilla.org/showdependencytree.cgi?id=66967
Goals For libpr0n
- Be correct; fix the current correctness errors.
- Be flexible enough to support embedding client's needs. This means machines with potentially low memory available, perhaps no disk, perhaps not full 24 bit color displays.
- Remove performance bottlenecks to increase performance (page load, back, printing, rendering, decoding).
- Reduce dynamic memory footprint whenever possible.
- Make the code configurable enough to allow clients to make time/space performance tradeoffs so they can tailor the imaging code to their needs.
- Fix as many of the performance, memory, and correctness critical bugs in a short time frame in order to provide the largest possible benefit to the next major releases from Mozilla and Netscape.
Why Not Patch The Current libimg Implementation?
Incrementally patching the current implementation would produce small improvements in the areas we target, but any of the large changes necessary for improved time/space performance cause cascading changes that affect the libimg design as a whole.
For example, to fix the performance bugs of not caching animations the cache logic would have to be heavily modified along with the cache entries. This would cause a cascade change to the image rendering code and the image decoders. Suddenly to fix this one seemingly simple bug you've touched every major part of libimg and the layout interface and use of libimg.
Suddenly, you've changed the semantics of the system as a whole, potentially exposing several dependencies and you've only addressed one problem.
In short, we believe that patching the current implementation does not provide the best bang for the buck.
Strategy
The strategy is to build an image lib that corrects the current design flaws in the most straight forward way possible and can provide significant performance and footprint improvements in a short time frame.
The new approach's major design level deviations from the current code are that it attempts to do the least amount of work possible, and do it only once. The goal is to download, verify, and decode the image data only once, including animations. For offline printing, we will use the cache (either mem or disk) to store full resolution images (compressed on disk in the http cache if available or in mem) as long as there is space on the cache device.
The work to date has been to get a rough but broad swath of the new design up and running to prove that the new approach is both viable and produces the expected positive results. This has been a success. We currently can render PNG and GIF images in both HTML content and XUL. In our performance tests we show improvements upon Netscape 6.0 performance and even performance challenging IE5 in some areas.
Progress
Currently we have GIFs and PNGs rendering in both XUL and HTML content. They fetch once off of the net, and use a makeshift image cache until the nsCacheManagerService is ready for use.
Things needing to be done:
- The JPEG decoder integration.
- Hooking up CSS to the new code.
- Printing needs to be implemented.
- Using the new nsCacheManagerService.
- Variable colorspace support (we always decode to 24bit just like we do in the current code).
- Memory usage vs. performance confirmability (caching/eviction policy, use of mem and disk cache)
- Polish.
Technical bits
nsImageLoader
- The image loader manages the image loads occurring for a page load. Hands out image request
proxies when needed. Manages shutdown of image loads in the case of an aborted
page load.
nsImageRequest
- An image request. Checks the image cache for a match and manages the decoder callbacks
along the necko stream feeding the decoder. One per nsImageRequest per URL, this
is the object added to the cache.
nsImageRequestProxy
- A proxy for the real image request that is handed out to multiple observers of the same image.
nsImageContainer - Container for an image or animation. Owns the frames of the image, color space info., the animation timer, etc.
nsImageFrame
- Frame of an a image. Most image containers will have only one frame, but animations
will have several. The implementations of nsImageFrame hold platform specific image information.
nsIImageDecoder
- Interface that image decoders implement.
nsIImageDecoderObserver
- The interface that image decoders use to notifiy their clients of progress.