Difference between revisions of "Development/GDB"
(→Disassembling) |
S Richling (talk | contribs) (→Branch record tracing) |
||
| (18 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
| + | {{Softwarepage|devel/gdb}} |
||
| + | |||
{| width=600px class="wikitable" |
{| width=600px class="wikitable" |
||
|- |
|- |
||
| Line 5: | Line 7: | ||
| module load |
| module load |
||
| devel/gdb |
| devel/gdb |
||
| − | |- |
||
| − | | Availability |
||
| − | | [[bwUniCluster]] | [[BwForCluster_Chemistry]] |
||
|- |
|- |
||
| License |
| License |
||
| Line 25: | Line 24: | ||
|} |
|} |
||
<br> |
<br> |
||
| − | = |
+ | = Description = |
| − | The '''GNU Debugger (GDB)''' is a standard debugger for serial programs although it can be used for parallel and even distributed programs with few processes too. |
+ | The '''GNU Debugger (GDB)''' is a standard debugger for serial programs although it can be used for parallel and even distributed programs with few processes too. In the past Intel supported their own '''idb''' debugger, however this has been deprecated in favor of their own port called <kbd>gdb-ia</kbd>. |
<br> |
<br> |
||
| − | <br> |
||
| − | = Versions and Availability = |
||
| − | A list of versions currently available on all bwHPC-C5-Clusters can be obtained from the |
||
| − | <br> |
||
| − | <big> |
||
| − | [https://cis-hpc.uni-konstanz.de/prod.cis/ Cluster Information System CIS] |
||
| − | |||
| − | </big> |
||
| − | {{#widget:Iframe |
||
| − | |url=https://cis-hpc.uni-konstanz.de/prod.cis/bwUniCluster/devel/gdb |
||
| − | |width=99% |
||
| − | |height=120 |
||
| − | }} |
||
| − | On the command line interface of any bwHPC cluster you'll get a list of available versions |
||
| − | by using the command ''''module avail devel/gdb''''. |
||
| − | <pre> |
||
| − | $ : bwUniCluster |
||
| − | $ module avail devel/gdb |
||
| − | ------------------------ /opt/bwhpc/common/modulefiles ------------------------- |
||
| − | devel/gdb/7.7</pre> |
||
| − | |||
| − | <font color=red>Attention!</font><br> |
||
| − | The default Linux GDB may be a different version. |
||
| − | Check the <font color=green>default</font> version by: ''''gdb --version'''' <font color=red>without loaded GDB-module</font>. |
||
| − | <pre> |
||
| − | $ module clear # clear all loaded modules |
||
| − | Are you sure you want to clear all loaded modules!? [n] y |
||
| − | $ gdb --version # this is the default version |
||
| − | GNU gdb (GDB) Red Hat Enterprise Linux (7.2-83.el6) |
||
| − | [...] |
||
| − | $ module avail devel/gdb # available GDB-module |
||
| − | ------------------------ /opt/bwhpc/common/modulefiles ------------------------- |
||
| − | devel/gdb/7.7 |
||
| − | </pre> |
||
| − | ''It is not recommended (or impossible) the debug MPI-based binaries with GDB. You can't attach to MPI'ed jobs, too.'' |
||
| − | <br> |
||
| − | <br> |
||
| − | |||
| − | = Loading = |
||
| − | It is possible to choose between several versions of GDB. By default both programs are installed. You can check the version which is currently used with: |
||
| − | <pre>$ gdb --version</pre> |
||
| − | <pre>$ gcc --version</pre> |
||
| − | <br> |
||
| − | To use a different version of GCC or GDB, you have to load it through the module system. For example you need GDB 7.7 or higher for GCC version 4.8 and above. In this case load the devel/gdb module. |
||
| − | <br> |
||
| − | How to load the desired version (e.g. GDB version 7.7): |
||
| − | <pre>$ module load devel/gdb/7.7</pre> |
||
| − | <br> |
||
| − | If you want to use IDB load the Intel compiler module: |
||
| − | <pre>$ module load compiler/intel</pre> |
||
| − | <br> |
||
| − | = Documentation = |
||
| − | For '''online documentation''' see the links section in the summary table at the top of this page. For '''local documentation''' consult the man page. |
||
| − | <pre>$ man gdb</pre> |
||
| − | or |
||
| − | <pre>$ man idb</pre> |
||
| − | <br> |
||
= Basic commands = |
= Basic commands = |
||
| − | The code you want to debug should be compiled with the <font color=green>-g</font> option. If the optimization flag is not set, |
+ | The code you want to debug should be compiled with the <kbd><font color=green>-g</font></kbd> option. If the optimization flag is not set, GCC will still do some basic optimization, like dead-code elimination or reorder instruction execution obfuscating the order when debugging. Therefore, it is recommended to turn off optimization explicitly with the <font color=green>-O0</font> parameter for debugging. To start a debug session for a program execute GDB with the program path as parameter: |
<pre>$ gdb ./example</pre> |
<pre>$ gdb ./example</pre> |
||
| Line 134: | Line 76: | ||
|} |
|} |
||
| − | '''Example:''' |
||
| − | We debug the following program called bug.c which crashes on execution. |
||
| − | <source lang="c"> |
||
| − | #include <stdio.h> |
||
| − | int global = 0; |
||
| + | = Branch record tracing = |
||
| − | void begin() { |
||
| + | Starting with GBD-10.1, the debugger has been installed with Intel Processor Trace [https://github.com/intel/libipt libipt], allowing recording and replaying of process state. |
||
| − | global = 1; |
||
| + | This allows disassembling previously executed instructions, checking for previously called functions and branch tracing. |
||
| − | } |
||
| + | Honestly, Segmentation Violations are better caught using [[Development/Valgrind|Valgrind]]. However in this case, |
||
| − | void loop() { |
||
| + | <kbd>valgrind</kbd> would ''not'' have helped: this loops overwrites <kbd>v</kbd> an |
||
| − | int v[2]; |
||
| + | array of 2 ints on the stack and the return address leading to the execution of IP <kbd>0x07</kbd>. |
||
| − | int i, k; |
||
| + | More information is available in [https://sourceware.org/gdb/current/onlinedocs/gdb/Process-Record-and-Replay.html#Process-Record-and-Replay gdb's feature documentation] |
||
| − | for(i = 0; i < 8; i++) { |
||
| − | k = i/2*2; /* should have been k = i/(2*2); */ |
||
| − | v[k] = i; |
||
| − | } |
||
| − | } |
||
| − | void end() { |
||
| − | global = 2; |
||
| − | } |
||
| − | |||
| − | int main() { |
||
| − | begin(); |
||
| − | loop(); |
||
| − | end(); |
||
| − | |||
| − | return 0; |
||
| − | } |
||
| − | </source> |
||
| − | |||
| − | '''Sample GDB session:''' |
||
| − | <pre> |
||
| − | $ gcc -g bug.c -o bug |
||
| − | $ gdb ./bug |
||
| − | GNU gdb (GDB) Red Hat Enterprise Linux (7.2-60.el6_4.1) |
||
| − | Copyright (C) 2010 Free Software Foundation, Inc. |
||
| − | License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> |
||
| − | This is free software: you are free to change and redistribute it. |
||
| − | There is NO WARRANTY, to the extent permitted by law. Type "show copying" |
||
| − | and "show warranty" for details. |
||
| − | This GDB was configured as "x86_64-redhat-linux-gnu". |
||
| − | For bug reporting instructions, please see: |
||
| − | <http://www.gnu.org/software/gdb/bugs/>... |
||
| − | Reading symbols from /pfs/data2/home/xx/xxx/xxxx/bug...done. |
||
| − | (gdb) break main |
||
| − | Breakpoint 1 at 0x4005b2: file bug.c, line 26. |
||
| − | (gdb) run |
||
| − | Starting program: /pfs/data2/home/xx/xxx/xxxx/bug |
||
| − | |||
| − | Breakpoint 1, main () at bug.c:26 |
||
| − | 26 begin(); |
||
| − | Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64 libgcc-4.4.7-4.el6.x86_64 |
||
| − | (gdb) next |
||
| − | 27 loop(); |
||
| − | (gdb) next |
||
| − | |||
| − | Program received signal SIGSEGV, Segmentation fault. |
||
| − | 0x0000000000000005 in ?? () |
||
| − | (gdb) # now we know that the bug is in loop(). start again. |
||
| − | (gdb) run |
||
| − | The program being debugged has been started already. |
||
| − | Start it from the beginning? (y or n) y |
||
| − | Starting program: /pfs/data2/home/xx/xxx/xxxx/bug |
||
| − | |||
| − | Breakpoint 1, main () at bug.c:26 |
||
| − | 26 begin(); |
||
| − | (gdb) next |
||
| − | 27 loop(); |
||
| − | (gdb) step |
||
| − | loop () at bug.c:13 |
||
| − | 13 for(i = 0; i < 8; i++) |
||
| − | (gdb) next |
||
| − | 15 k = i/2*2; |
||
| − | (gdb) next |
||
| − | 16 v[k] = i; |
||
| − | (gdb) # maybe k gets too big? |
||
| − | (gdb) watch (k >= 2) |
||
| − | Hardware watchpoint 2: (k >= 2) |
||
| − | (gdb) continue |
||
| − | Continuing. |
||
| − | Hardware watchpoint 2: (k >= 2) |
||
| − | |||
| − | Old value = 0 |
||
| − | New value = 1 |
||
| − | loop () at bug.c:16 |
||
| − | 16 v[k] = i; |
||
| − | (gdb) # k is too big |
||
| − | (gdb) print k |
||
| − | $1 = 2 |
||
| − | (gdb) print i |
||
| − | $2 = 2 |
||
| − | (gdb) quit |
||
| − | </pre> |
||
<br> |
<br> |
||
| Line 239: | Line 96: | ||
<pre>$ gdb ./ex core.xxx</pre> |
<pre>$ gdb ./ex core.xxx</pre> |
||
Now you can print a backtrace to check in which function the error happened and what values the parameters had. Additionally you can examine the values of your variables to reproduce the error. |
Now you can print a backtrace to check in which function the error happened and what values the parameters had. Additionally you can examine the values of your variables to reproduce the error. |
||
| − | <br> |
||
<br> |
<br> |
||
| + | |||
= Multithreaded debugging = |
= Multithreaded debugging = |
||
GDB can also be useful for multithreaded applications for example when OpenMP was used. By going through each thread separately you can better see what is really going on and you can check the computation step by step. |
GDB can also be useful for multithreaded applications for example when OpenMP was used. By going through each thread separately you can better see what is really going on and you can check the computation step by step. |
||
The following commands are useful for multithreaded debugging: |
The following commands are useful for multithreaded debugging: |
||
| − | |||
| − | {| width=600px class="wikitable" |
||
| − | |- |
||
| − | ! Command |
||
| − | ! Description |
||
| − | |- |
||
| − | | info threads |
||
| − | | Shows the status of all existing threads. |
||
| − | |- |
||
| − | | thread ''num'' |
||
| − | | Switches to the thread with the number ''num'' |
||
| − | |} |
||
| − | |||
| − | '''Example:''' |
||
| − | We debug the following program called thread_bug.c which crashes on execution. |
||
| − | <source lang="c"> |
||
| − | #include <stdio.h> |
||
| − | #include <pthread.h> |
||
| − | |||
| − | pthread_t thread; |
||
| − | |||
| − | void* thread3 (void* d) |
||
| − | { |
||
| − | int w[2]; |
||
| − | int c, l; |
||
| − | |||
| − | for(c = 0; c < 8; c++) { |
||
| − | l = c/2*2; /* should have been l = c/(2*2); */ |
||
| − | w[l] = c; |
||
| − | } |
||
| − | |||
| − | return NULL; |
||
| − | } |
||
| − | |||
| − | void* thread2 (void* d) |
||
| − | { |
||
| − | int v[2]; |
||
| − | int i, k; |
||
| − | |||
| − | for(i = 0; i < 8; i++) { |
||
| − | sleep(4); |
||
| − | k = i/(2*2); /* should have been k = i/(2*2); */ |
||
| − | v[k] = i; |
||
| − | } |
||
| − | |||
| − | return NULL; |
||
| − | } |
||
| − | |||
| − | int main (){ |
||
| − | |||
| − | pthread_create (&thread, NULL, thread2, NULL); |
||
| − | pthread_create (&thread, NULL, thread3, NULL); |
||
| − | |||
| − | //Thread 1 |
||
| − | int count1 = 0; |
||
| − | |||
| − | while(count1 < 4000){ |
||
| − | printf("Thread 1: %d\n", count1++); |
||
| − | } |
||
| − | |||
| − | pthread_join(thread, NULL); |
||
| − | return 0; |
||
| − | } |
||
| − | </source> |
||
| − | |||
| − | '''Sample GDB thread session:''' |
||
| − | <pre> |
||
| − | $ gcc -g thread_bug.c -o thread_bug -lpthread |
||
| − | $ gdb ./thread_bug |
||
| − | [...] |
||
| − | <http://www.gnu.org/software/gdb/bugs/>... |
||
| − | Reading symbols from /pfs/data2/home/xx/xxx/xxxx/bug...done. |
||
| − | (gdb) break thread3 |
||
| − | Breakpoint 1 at 0x40060c: file thread_bug.c, line 11. |
||
| − | (gdb) break thread2 |
||
| − | Breakpoint 2 at 0x400650: file thread_bug.c, line 24. |
||
| − | (gdb) break main |
||
| − | Breakpoint 3 at 0x40069e: file thread_bug.c, line 35. |
||
| − | (gdb) run |
||
| − | Starting program: /tank/home/doros/.t/thread_bug |
||
| − | [Thread debugging using libthread_db enabled] |
||
| − | |||
| − | Breakpoint 3, main () at thread_bug.c:35 |
||
| − | 35 pthread_create (&thread, NULL, thread2, NULL); |
||
| − | Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6.x86_64 |
||
| − | (gdb) info threads |
||
| − | * 1 Thread 0x7ffff7fe5700 (LWP 28260) main () at thread_bug.c:35 |
||
| − | (gdb) next |
||
| − | [New Thread 0x7ffff7fe3700 (LWP 28303)] |
||
| − | 36 pthread_create (&thread, NULL, thread3, NULL); |
||
| − | (gdb) info threads |
||
| − | 2 Thread 0x7ffff7fe3700 (LWP 28303) thread2 (d=0x0) at thread_bug.c:24 |
||
| − | * 1 Thread 0x7ffff7fe5700 (LWP 28260) main () at thread_bug.c:36 |
||
| − | (gdb) next |
||
| − | [Switching to Thread 0x7ffff7fe3700 (LWP 28303)] |
||
| − | |||
| − | Breakpoint 2, thread2 (d=0x0) at thread_bug.c:24 |
||
| − | 24 for(i = 0; i < 8; i++) { |
||
| − | (gdb) next |
||
| − | 25 sleep(4); |
||
| − | (gdb) next |
||
| − | [New Thread 0x7ffff77e2700 (LWP 28344)] |
||
| − | [Switching to Thread 0x7ffff77e2700 (LWP 28344)] |
||
| − | |||
| − | Breakpoint 1, thread3 (d=0x0) at thread_bug.c:11 |
||
| − | 11 for(c = 0; c < 8; c++) { |
||
| − | (gdb) info threads |
||
| − | * 3 Thread 0x7ffff77e2700 (LWP 28344) thread3 (d=0x0) at thread_bug.c:11 |
||
| − | 2 Thread 0x7ffff7fe3700 (LWP 28303) 0x000000362f8accdd in nanosleep () from /lib64/libc.so.6 |
||
| − | 1 Thread 0x7ffff7fe5700 (LWP 28260) 0x000000362f8725db in _IO_new_file_overflow () from /lib64/libc.so.6 |
||
| − | (gdb) thread 2 |
||
| − | [Switching to thread 2 (Thread 0x7ffff7fe3700 (LWP 28303))]#0 0x000000362f8accdd in nanosleep () from /lib64/libc.so.6 |
||
| − | (gdb) next |
||
| − | Single stepping until exit from function nanosleep, |
||
| − | which has no line number information. |
||
| − | [Switching to Thread 0x7ffff77e2700 (LWP 28344)] |
||
| − | |||
| − | Breakpoint 1, thread3 (d=0x0) at thread_bug.c:11 |
||
| − | 11 for(c = 0; c < 8; c++) { |
||
| − | (gdb) thread 2 |
||
| − | [Switching to thread 2 (Thread 0x7ffff7fe3700 (LWP 28303))]#0 0x000000362f8acce9 in nanosleep () from /lib64/libc.so.6 |
||
| − | (gdb) next |
||
| − | Single stepping until exit from function nanosleep, |
||
| − | which has no line number information. |
||
| − | 0x000000362f8acb50 in sleep () from /lib64/libc.so.6 |
||
| − | (gdb) info threads |
||
| − | 3 Thread 0x7ffff77e2700 (LWP 28344) thread3 (d=0x0) at thread_bug.c:11 |
||
| − | * 2 Thread 0x7ffff7fe3700 (LWP 28303) 0x000000362f8acb50 in sleep () from /lib64/libc.so.6 |
||
| − | 1 Thread 0x7ffff7fe5700 (LWP 28260) 0x000000362f8476f0 in vfprintf () from /lib64/libc.so.6 |
||
| − | (gdb) thread 3 |
||
| − | [Switching to thread 3 (Thread 0x7ffff77e2700 (LWP 28344))]#0 thread3 (d=0x0) at thread_bug.c:11 |
||
| − | 11 for(c = 0; c < 8; c++) { |
||
| − | (gdb) next |
||
| − | 12 l = c/2*2; /* should have been l = c/(2*2); */ |
||
| − | (gdb) watch (k >= 2) |
||
| − | No symbol "k" in current context. |
||
| − | (gdb) watch (l >= 2) |
||
| − | Hardware watchpoint 4: (l >= 2) |
||
| − | (gdb) continue |
||
| − | Continuing. |
||
| − | Thread 1: 0 |
||
| − | Thread 1: 1 |
||
| − | Thread 1: 2 |
||
| − | Thread 1: 3 |
||
| − | Thread 1: 4 |
||
| − | [...] |
||
| − | Hardware watchpoint 4: (l >= 2) |
||
| − | |||
| − | Old value = 0 |
||
| − | New value = 1 |
||
| − | thread3 (d=0x0) at thread_bug.c:13 |
||
| − | 13 w[l] = c; |
||
| − | (gdb) print l |
||
| − | $1 = 2 |
||
| − | (gdb) print c |
||
| − | $2 = 2 |
||
| − | (gdb) quit |
||
| − | </pre> |
||
| − | = Disassembling = |
||
| − | {| width=600px class="wikitable" |
||
| − | |- |
||
| − | ! Command |
||
| − | ! Description |
||
| − | |- |
||
| − | | info functions |
||
| − | | Shows names and data types of all defined functions. |
||
| − | |- |
||
| − | | info line "function" |
||
| − | | Map source lines to memory adresses (and back). |
||
| − | |- |
||
| − | | disassemble ''function'' |
||
| − | | Disassembles "function" (or a function fragment). |
||
| − | |} |
||
| − | '''Sample GDB disassembling session:''' |
||
| − | <pre> |
||
| − | |||
| − | |||
| − | </pre> |
||
| − | [[Category:debugger software]][[Category:bwUniCluster]] |
||
Latest revision as of 00:49, 9 December 2022
|
The main documentation is available via |
| Description | Content |
|---|---|
| module load | devel/gdb |
| License | GPL |
| Citing | n/a |
| Links | Homepage | Documentation | Wiki | Mailinglists |
| Graphical Interface | No |
| Included modules | icc | icpc | ifort | idb |
Contents
1 Description
The GNU Debugger (GDB) is a standard debugger for serial programs although it can be used for parallel and even distributed programs with few processes too. In the past Intel supported their own idb debugger, however this has been deprecated in favor of their own port called gdb-ia.
2 Basic commands
The code you want to debug should be compiled with the -g option. If the optimization flag is not set, GCC will still do some basic optimization, like dead-code elimination or reorder instruction execution obfuscating the order when debugging. Therefore, it is recommended to turn off optimization explicitly with the -O0 parameter for debugging. To start a debug session for a program execute GDB with the program path as parameter:
$ gdb ./example
Inside GDB is a prompt where you can enter commands. Important commands are listed below.
| Command | Description |
|---|---|
| help cmd | Show help for command cmd. |
| break func | Set a breakpoint at function func. |
| run | Start program. |
| next | Go to next program line. Do not enter functions. |
| step | Go to next program line. Enter functions. |
| list | Show the surrounding source code of the currently processed line. |
| print expr | Print the value of the expression expr. |
| display expr | Display the value of the expression expr every time the program stops. |
| watch expr | Stop when value of the expression expr changes. |
| continue | Continue execution until a breakpoint or a watchpoint appears. |
| backtrace | Print a list of functions that are currently active. |
| quit | Exit GDB. |
3 Branch record tracing
Starting with GBD-10.1, the debugger has been installed with Intel Processor Trace libipt, allowing recording and replaying of process state. This allows disassembling previously executed instructions, checking for previously called functions and branch tracing.
Honestly, Segmentation Violations are better caught using Valgrind. However in this case, valgrind would not have helped: this loops overwrites v an array of 2 ints on the stack and the return address leading to the execution of IP 0x07.
More information is available in gdb's feature documentation
4 Core dumps
When the program crashes, a log file (called core dump) can be created which contains the state of the program when it crashed. This is turned off by default because these core dumps can get quite large. If you want to turn it on you have to change your ulimits, for example:
$ ulimit -c unlimited
Every time your program crashes a new file called core.xxx (where xxx is a number) will be created in the directory from which you started the executable. You can call gdb to examine your core dump using the following command (assuming your program is called ex):
$ gdb ./ex core.xxx
Now you can print a backtrace to check in which function the error happened and what values the parameters had. Additionally you can examine the values of your variables to reproduce the error.
5 Multithreaded debugging
GDB can also be useful for multithreaded applications for example when OpenMP was used. By going through each thread separately you can better see what is really going on and you can check the computation step by step. The following commands are useful for multithreaded debugging:
