Debugging Mozilla on Linux FAQ
This is the debugging Mozilla on Linux FAQ. It details some tricks that you can use to make it easier to debug and to work around some of the problems that GDB has.
GDB has problems that Mozilla exacerbates. The most serious of these problems is GDB's memory usage when debugging Mozilla's components. GDB can grow up to several hundred meg in size while debugging Mozilla. To work around this problem, it's necessary to load each of Mozilla's component libraries only when needed. This process is described in this FAQ. You will see this problem if you have less than 256 meg of ram in your computer.
Many distributions use a version of GDB that is somewhat old. The most recent version of GDB that is available at the time that this document is being written is version 5.0. Version 5.0 is much more usable than previous version for debugging Mozilla. It includes fixes for slow symbol loading and includes better thread functionality than previous versions. However, issues with regards to gdb's high memory usage with Mozilla are not solved with the 5.0 release of GDB.
You can get a more recent copy of GDB from the GDB site at sourceware.
- How do I debug Mozilla on Linux?
- The debugger uses a lot of memory. How do I fix it?
- How do I load shared libraries?
- How do I see what libraries I already have loaded?
- GDB is taking forever to load a shared library. What the hell is it doing?
- How do I set a breakpoint in a library that hasn't been loaded?
- How do I set a breakpoint when a component is loaded?
- This is hard. Give me a .gdbinit that works.
- I can't set a breakpoint. Why not?
- Debian's GDB doesn't work. What do I do?
- How do I display PRUnichar's?
- How do I display an nsString?
- How do I determine the concrete type of an object pointed to by an interface pointer?
- "run" or "prun" in GDB fails with "error in loading shared libraries."
- How do I pass arguments over to the app using GDB?
- How do I pass arguments in prun?
- Mozilla is aborting. Where do I set a breakpoint to find out where it is exiting?
- I keep getting a SIG32 in the debugger. How do I fix it?
- How can I debug race conditions and/or how can I make something different happen at NS_ASSERTION time?
- What performance tools do I have on Linux?
- How do I run the debugger to debug Mozilla in emacs/xemacs?
- I'm feeling brave (or my old gdb isn't working at all). How can I try out gdb 5.0?
- gdb 5 used to work for me, but now Mozilla won't start. What can I do?
- Mozilla crashes at startup when running under Valgrind. What can I do?
- I have questions. Who do I ask?
- I want to add to this FAQ. Who do I send my additions to?
- 1. How do I debug Mozilla on Linux?
-
The
mozilla
script that is launches the browser can also be used to launch the debugger as well. You can use it like this:[blizzard@gunhead bin]$ cd mozilla/dist/bin [blizzard@gunhead bin]$ ./mozilla -g
This will actually launch ddd, a graphical front-end on gdb. Redhat 6.2 doesn't install ddd by default, so you may get an error message. In this case,
[blizzard@gunhead bin]$ setenv LD_LIBRARY_PATH .
(csh), or
[blizzard@gunhead bin]$ LD_LIBRARY_PATH=.; export LD_LIBRARY_PATH
(bash), then run
[blizzard@gunhead bin]$ gdb mozilla-bin
Using GDB is well beyond the scope of this document. There is documentation available on your system if you have GDB installed in the form of info pages. You might want to use the gnome help browser to read that docuementation as many people find the info reader on Linux hard to use.
Additionally, you can use a graphical front end to GDB like ddd or insight.
Here are some other sites that have more information:
- 2. The debugger uses a lot of memory. How do I fix it?
-
As mentioned in the introduction of this FAQ GDB uses a lot of memory when loading Mozilla's shared libraries. The solution to this is to delay loading shared libraries until they are actually needed.
However, you need to make sure that the base libraries like libc and pthreads are loaded before you tell GDB to stop loading shared libraries. If you don't allow those libraries to be loaded then GDB will not be able to properly debug threads on Linux. Mozilla uses pthreads for its networking library so you need to be able to work in a threaded environment.
The best way to do this is to set a breakpoint in main, let the program run until main and then turn off automatic library loading. From there you can allow the program to continue running. Here's an example:
[blizzard@gunhead mozilla]$ cd dist/bin/ [blizzard@gunhead bin]$ ./mozilla -g .//run-mozilla.sh -g ./mozilla-bin MOZILLA_FIVE_HOME=/home/blizzard/src/mozilla/mozilla/dist/bin LD_LIBRARY_PATH=/home/blizzard/src/mozilla/mozilla/dist/bin SHLIB_PATH=/home/blizzard/src/mozilla/mozilla/dist/bin LIBPATH=/home/blizzard/src/mozilla/mozilla/dist/bin MOZ_PROGRAM=./mozilla-bin MOZ_TOOLKIT= moz_debug=1 moz_debugger= /usr/bin/gdb ./mozilla-bin -x /tmp/mozargs22288 GNU gdb 19991004 Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-redhat-linux"... (gdb) b main Breakpoint 1 at 0x804ec45: file nsAppRunner.cpp, line 811. (gdb) r Starting program: /home/blizzard/src/mozilla/mozilla/dist/bin/./mozilla-bin Breakpoint 1, main (argc=1, argv=0xbffff894) at nsAppRunner.cpp:811 811 InstallUnixSignalHandlers(argv[0]); (gdb) set auto-solib-add 0 (gdb) c Continuing. nsNativeComponentLoader: autoregistering begins. [...]
It's easy to define a function for gdb to do this for you automatically. You can add this function to the .gdbinit file in your home directory:
def prun tbreak main run set auto-solib-add 0 end
- 3. How do I load shared libraries?
-
If you have turned off automatic shared library loading, you will have to load them when you need them. GDB has a command to load libraries while the program is running. This is the
sharedlibrary
command. It can be shortened toshar
when used in GDB. The argument to the command is a regular expression for the libraries to be loaded.Here's what it looks like in the 4.18 version of the debugger:
^C Program received signal SIGINT, Interrupt. 0x404ccdeb in __sigsuspend (set=0xbf5ffbc0) at ../sysdeps/unix/sysv/linux/sigsuspend.c:48 48 ../sysdeps/unix/sysv/linux/sigsuspend.c: No such file or directory. Current language: auto; currently c (gdb) shar glib Reading symbols from /usr/lib/libglib-1.2.so.0...done. (gdb)
Here's what it looks like in a version 5.x of the debugger ( yet unreleased ):
^C Program received signal SIGINT, Interrupt. [Switching to Thread 2051 (runnable)] 0x404ccdeb in __sigsuspend (set=0xbf5ffbac) at ../sysdeps/unix/sysv/linux/sigsuspend.c:48 48 ../sysdeps/unix/sysv/linux/sigsuspend.c: No such file or directory. Current language: auto; currently c (gdb) shar gtk Reading symbols from /usr/lib/libgtk-1.2.so.0...done. Loaded symbols for /usr/lib/libgtk-1.2.so.0 Reading symbols from /usr/lib/gtk/themes/engines/libthinice.so...done. Loaded symbols for /usr/lib/gtk/themes/engines/libthinice.so (gdb)
As you see above, it's possible to load more than one library with the same load command in GDB.
- 4. How do I see what libraries I already have loaded?
-
You can see what libraries you already have loaded with the
info sharedlibrary
command:(gdb) info shar From To Syms Read Shared Object Library 0x4044a000 0x4044d08c Yes /lib/libdl.so.2 0x4044e000 0x4048ff90 Yes /usr/lib/libstdc++-libc6.1-1.so.2 0x40491000 0x404ad9d8 Yes /lib/libm.so.6 0x404ae000 0x405a285c Yes /lib/libc.so.6 0x40000000 0x40013ed0 Yes /lib/ld-linux.so.2 0x40607000 0x4061554c No /usr/lib/libz.so.1 0x40763000 0x4088af74 No /usr/lib/libgtk-1.2.so.0 0x4088b000 0x408c04d4 No /usr/lib/libgdk-1.2.so.0 [...]
- 5. GDB is taking forever to load a shared library. What the hell is it doing?
-
Versions of GDB earlier than 5 ( yet unreleased ) use a very slow algoritm for walking the list of already loaded symbols as it adds symbols to its symbol table. This has been fixed in version 5 of GDB. There isn't a fix for older versions.
- 6. How do I set a breakpoint in a library that hasn't been loaded?
-
Unfortunately, there isn't a way to set a breakpoint in GDB in a library that hasn't been loaded. If you are actually intersted in setting a breakpoint in a real component when it's loaded please see the section on setting a breakpoint when a component is loaded.
If you are completely desperate and have to set a breakpoint in a library when it's loaded you can set a breakpoint on the symbol
_dl_open
. This function is called when a new library is loaded. You can set your breakpoint after you see that your library has been loaded. - 7. How do I set a breakpoint when a component is loaded?
-
There's a faculity in XPCOM that allows you to set an environment variable that triggers the program to drop into the debugger when it loads a certain component.
You have to set the
XPCOM_BREAK_ON_LOAD
environment variable before you run Mozilla. You set the variable to a string that contains the names of libraries you might want to load. For example, if you wanted to stop when a library namedraptor
ornecko
is loaded, you can set the variable toraptor:necko
. Here's an example:(gdb) set env XPCOM_BREAK_ON_LOAD raptor:necko (gdb) prun
- 8. This is hard. Give me a .gdbinit that works.
-
Ok. Here's one. It's got four function def's in it.
- "prun" to start the browser and disable library loading.
- "pmail" which will launch mail.
- "pu" which will display a (PRUnichar *) string.
- "ps" which will display a nsString.
- 9. I can't set a breakpoint. Why not?
-
You probably can't set a breakpoint because the library in which that breakpoint is located hasn't been loaded. If you know the library has actually been loaded into Mozilla but you can't set the breakpoint see the section on loading shared libraries. If the library hasn't been loaded yet you will have to wait until it is. If you want to set a breakpoint as soon as the library is loaded, see the section on breaking when a component is loaded and breaking on a library load.
- 10. Debian's GDB doesn't work. What do I do?
-
[submitted by Bruce Mitchener]
Debian's unstable distribution currently uses glibc 2.1 and GDB 4.18. However, there is no package of GDB for Debian with the appropriate threads patches that will work with glibc 2.1. I was able to get this to work by getting the GDB 4.18 RPM from Red Hat's rawhide server and installing that. It has all of the patches necessary for debugging threaded software. These fixes are expected to be merged into GDB, which will fix the problem for Debian Linux. - 11. How do I display PRUnichar's?
-
Several people have suggested ways to do this:
-
(gdb) print ((PRUnichar*)uri.mBuffer)[0]@16 $47 = {114, 100, 102, 58, 110, 117, 108, 108, 0, 0, 8, 0, 0, 0, 37432, 16514}
-
(gdb) print aURI $1 = (const PRUnichar *) 0x855e6e0 (gdb) x/32ch aURI 0x855e6e0: 104 'h' 116 't' 116 't' 112 'p' 58 ':' 47 '/' 47 '/' 119 'w' 0x855e6f0: 119 'w' 119 'w' 46 '.' 109 'm' 111 'o' 122 'z' 105 'i' 108 'l' 0x855e700: 108 'l' 97 'a' 46 '.' 111 'o' 114 'r' 103 'g' 47 '/' 115 's' 0x855e710: 116 't' 97 'a' 114 'r' 116 't' 47 '/' 0 '\0' 25 '\031' 0 '\0' (gdb)
-
Define helper functions in your .gdbinit
# define "pu" command to display PRUnichar * strings (100 chars max) def pu set $uni = $arg0 set $i = 0 while (*$uni && $i++<100) if (*$uni < 0x80) print *(char*)$uni++ else print /x *(short*)$uni++ end end end # define "ps" command to display nsString/nsAutoString/nsCString/nsCAutoString def ps set $ns = $arg0 if ($ns->mCharSize) pu $ns->mUStr else print $ns->mStr end end
This is hard. Give me a .gdbinit that already has the functions.
-
Define a small helper function "punichar" in #ifdef NS_DEBUG code somewhere.
-
- 12. How do I display an nsString?
-
You can call the ToNewCString() method on the nsString in question. It leaks a little memory but it shouldn't hurt anything if you only do it a few times in one gdb session. ( from akkana@netscape.com )
(gdb) p string.ToNewCString()
- 13. How do I determine the concrete type of an object pointed to by an interface pointer?
-
You can determine the concrete type of any object pointed to by an XPCOM interface pointer by looking at the mangled name of the symbol for the object's vtable:
(gdb) p aKidFrame $1 = (nsIFrame *) 0x85058d4 (gdb) x/wa *(void**)aKidFrame 0x4210d380 <__vt_14nsRootBoxFrame>: 0x0 (gdb) p *(nsRootBoxFrame*)aKidFrame [ all the member variables of aKidFrame ]
(If you're using gcc 3.x, the output is slightly different from the above (which is for gcc 2.9x), but just pay attention to the vtable symbol (which in this case would be
_ZTV14nsRootBoxFrame
) rather than the mangled name of the first virtual function, since some classes may not override the first virtual function (although they usually do, since it'sQueryInterface
).)Note that you won't get anything useful if the shared library containing the implementation of the object you're looking at is not loaded. See "How do I load shared libraries?" and "How do I see what libraries I already have loaded?".
- 14. "run" or "prun" in GDB fails with "error in loading shared libraries."
-
Attempting to run mozilla-bin inside GDB fails with an error message similar to the following:
Starting program: /u/dmose/s/mozilla/mozilla-all/mozilla/dist/bin/./mozilla-bin /u/dmose/s/mozilla/mozilla-all/mozilla/dist/bin/./mozilla-bin: error in loading shared libraries: libraptorgfx.so: cannot open shared object file: No such file or directory
Your LD_LIBRARY_PATH is probably being reset by your .cshrc or .profile. From the GDB manual:
*Warning:* GDB runs your program using the shell indicated by your 'SHELL' environment variable if it exists (or '/bin/sh' if not). If your 'SHELL' variable names a shell that runs an initialization file -- such as '.cshrc' for C-shell, or '.bashrc' for BASH--any variables you set in that file affect your program. You may wish to move setting of environment variables to files that are only run when you sign on, such as '.login' or '.profile'.
- 15. How do I pass arguments over to the app using GDB/ddd?
-
You should now be able to convey arguments using ./mozilla -g your-list-of-arguments For example you can call the mozilla script like this if you want to launch mail at startup: ./mozilla -g -mail
- 16. How do I pass arguments in prun?
-
Set the arguments in GDB before calling prun. Here's an example on how to do that:
(gdb) set args http://www.mozilla.org (gdb) prun
- 17. Mozilla is aborting. Where do I set a breakpoint to find out where it is exiting?
-
On Linux there are two possible symbols that are causing this. They are
PR_ASSERT()
andNS_ASSERTION()
. To catch the program before it exits to see where it's asserting you can stop at two places:(gdb) b abort (gdb) b exit
- 18. I keep getting a SIG32 in the debugger. How do I fix it?
-
If you are getting a SIG32 while trying to debug Mozilla you might have turned off shared library loading before the pthreads library was loaded, e.g. by having
set auto-solib-add 0
in your.gdbinit
file.In this case you can either:
- Remove it and use the method explained in the section about GDB's memory usage instead.
- Use
handle SIG32 noprint
either in gdb or in your.gdbinit
file.
Alternatively the problem might lie in your pthread library. If this library has its symbols stripped, then GDB can't hook into thread events, and you end up with SIG32 signals. You can check if your libpthread is stripped by checking the output of
file /lib/libpthread*
for'stripped'.
To fix this problem for Gentoo Linux, you can re-emerge glibc after adding"nostrip"
to yourFEATURES
in/etc/make.conf
. - 19. How can I debug race conditions and/or how can I make something different happen at NS_ASSERTION time?
-
[submitted by Dan Mosedale]
Since Linux is unable to generate useful core files for multi-threaded applications, tracking down race-conditions that don't show up under the debugger can be a bit tricky.Unless you've given the
--enable-crash-on-assert
switch toconfigure
, you can now change the behavior of NS_ASSERTION (actually nsDebug::Break) using the XPCOM_DEBUG_BREAK environment variable. The following values are accepted:warn
- This is equivalent to the default of simply spitting an error message to stderr and ringing the bell.
stack
- In addition to ringing the bell and printing a warning, a stack trace is written to stderr.
suspend
- The process-group is sent a
SIGSTOP
signal. This suspends all threads so that you can attach to the process with the debugger before the conditions that caused the assertion disappear. abort
- This makes
NS_ASSERTION
act likeassert(3)
and is equivalent to using--enable-crash-on-assert
.
- 20. What performance tools do I have on Linux?
-
The amazing Jim Nance has written a tool called jprof that uses sampling to do performance analysis. Other tools are listed on the performance tools page.
Unfortunately, there aren't tools like Quantify available for Linux. You will have to go run Quantify on Solaris for that.
- 21. How do I run the debugger in emacs/xemacs?
-
Emacs and XEmacs contain modes for doing visual debugging that many programmers use. However, you might want to set up environment variables to make sure that when running the debugger and Mozilla they know where to load symbols from and where to find components.
The easiest way to set up those environment variables is to use the
run-mozilla.sh
script that's in the dist/bin directory of your build. This script will set up the environment that you need to run the editor, shell or debugger. Another trick is to use the script to run /bin/bash ( or your favorite shell ) to set up a shell that has the right setup and then you can run any commands you want inside it.[blizzard@gunhead bin]$ ./run-mozilla.sh /bin/bash MOZILLA_FIVE_HOME=/home/blizzard/src/mozilla/build/dist/bin LD_LIBRARY_PATH=/home/blizzard/src/mozilla/build/dist/bin LIBRARY_PATH=/home/blizzard/src/mozilla/build/dist/bin SHLIB_PATH=/home/blizzard/src/mozilla/build/dist/bin LIBPATH=/home/blizzard/src/mozilla/build/dist/bin ADDON_PATH=/home/blizzard/src/mozilla/build/dist/bin MOZ_PROGRAM=/bin/bash MOZ_TOOLKIT= moz_debug=0 moz_debugger= [blizzard@gunhead bin]$
- 22. I'm feeling brave (or my old gdb isn't working at all). How can I try out gdb 5.0?
-
You can try http://people.redhat.com/blizzard/software/. But no guarantees!
- 23. gdb 5 used to work for me, but now Mozilla won't start. What can I do?
-
A recent threading change (see bug 57051 for defails) caused a problem on some systems: Mozilla would get partway through its initialization, then die before showing a window.
A recent change to gdb has fixed this problem. Download and build the latest version of Insight, or if you don't want a GUI, the latest version of gdb.
- 24. Mozilla crashes at startup when I run it under Valgrind. What can I do?
-
GTK 2 builds of Mozilla bring in ORBit, which is sensitive to the memory alignment used by malloc(). The system implementation aligns to 8 bytes, while Valgrind aligns to 4 bytes by default. This causes ORBit to crash.
To run Mozilla under Valgrind, it is recommended that you use the
--alignment=8
option to Valgrind. - 25. I have questions. Who do I ask?
-
Post your questions to this newsgroup:
news://news.mozilla.org/netscape.public.mozilla.unix.
If you are trying to debug Mozilla on UNIX, we'll gladly help.
- 26. I want to add to this FAQ. Who do I send my additions to?
-
Post your additions to this newsgroup:
news://news.mozilla.org/netscape.public.mozilla.unix
Or send mail to blizzard@mozilla.org.