addr2line - The forgotten ingredient for debugging

Whether you have just started programming for linux or are an old nix programmer, you must have encountered the mysterious crashes with a stack dump saying "maybe a null pointer" and a long list of hex values. Well, you know it I know it, program crashing is rarely a mystery and is almost always caused by human error. Same stands true for this scenario. And linux is being generous by giving us that stack dump and saying "you messed up, and here is where you made a mistake".

The question is, how to use this information provided in the dump? To make sense of the numbers in the dump you need some tools. Here we will only talk about addr2line which helps to find out where was your program counter when the crash occurred.

addr2line is a short for "address to line" and this tool  does exactly what its name suggests. It takes an address in hex format and tries to find the line of code in the given executable at that address. But before you use addr2line, you will need to compile your code with debug symbol information. Without debug information you cannot debug your application nor use addr2line.

How to generate debug symbols information ? Here is how:

#gcc -g myfile.c -o myexec

Now that you have debug information available, run your application and let it crash. Mine crashed with the following dump.

Defered Exception context

CURRENT PROCESS:

COMM=qsig2 PID=263

TEXT = 0x06b50000-0x06b5a2c8        DATA = 0x06b602c8-0x06b70ce0

 BSS = 0x06b70ce0-0x06a80000  USER-STACK = 0x06a9fe70

return address: [0x06b56d38]; contents of:

0x06b56d10:  b0f8  b139  b17a  bb4e  3046  bbc0  6000  bbf0 

0x06b56d20:  b999  e408  004d  3210  9152  e410  0221  bbe0 

0x06b56d30:  2006  b9e0  bbf0  b9ea [9110] bbe0  b9e0  0c00 

0x06b56d40:  13f9  a138  3008  64b1  3001  4f10  5008  4f10 

SEQUENCER STATUS: Not tainted

 SEQSTAT: 00000027  IPEND: 0030  SYSCFG: 0006

  HWERRCAUSE: 0x0

  EXCAUSE   : 0x27

 RETE: <0x00000000> /* Maybe null pointer? */

 RETN: <0x06ad8000> [ qsig2 + 0x0 ]

 RETX: <0x06b56d38> [ /qsig2 + 0x6d38 ]

 RETS: <0x06b52d80> [ /qsig2 + 0x2d80 ]

 PC  : <0x06b56d38> [ /qsig2 + 0x6d38 ]

DCPLB_FAULT_ADDR: <0x0000000c> /* Maybe null pointer? */

ICPLB_FAULT_ADDR: <0x06b56d38> [ /qsig2 + 0x6d38 ]

PROCESSOR STATE:

 R0 : 0000000d    R1 : 0000000e    R2 : 00000001    R3 : 00000000

 R4 : 06b407e4    R5 : 06ac5448    R6 : 00000030    R7 : 06ac5448

 P0 : 06ae3298    P1 : 06b61768    P2 : 0000000d    P3 : 06b61768

 P4 : 06ae3e24    P5 : 06ae3e24    FP : 06ae3248    SP : 06ad7f24

 LB0: 0031efcd    LT0: 0031efcc    LC0: 00000000

 LB1: 0031cf7b    LT1: 0031cf7a    LC1: 00000000

 B0 : 06ca0388    L0 : 00000000    M0 : 07f7d615    I0 : 003358ec

 B1 : 06ca0348    L1 : 00000000    M1 : 06ca0408    I1 : 00000000

 B2 : 06cffaf4    L2 : 00000000    M2 : 06ca0388    I2 : 00000000

 B3 : 02b1235c    L3 : 00000000    M3 : 06ca03c8    I3 : 00000000

A0.w: 00000000   A0.x: 00000000   A1.w: 00000000   A1.x: 00000000

USP : 06ae3208  ASTAT: 02003004

You have a harware stack, a call stack , information cache and data cache information. What you need to find out here is your program counter "PC".

PC  : <0x06b56d38> [ /qsig2 + 0x6d38 ]

The system says the application crashed when it was executing the instruction at the mentioned address (0x6d38 in this case).

Locate this address in code and you have found your culprit. But how to do this ? Here is how:

#addr2line -C -e myexec 0x6d38

-C is to get demangled C code.

-e is to specify executable file

This will return your culprit filename, function name and line number.

So here you have it, enjoy more productive coding

To get complete list of attributes for addr2line try

addr2line -h

man addr2line