Development/General compiler usage: Difference between revisions

From bwHPC Wiki
Jump to navigation Jump to search
 
(59 intermediate revisions by 7 users not shown)
Line 4: Line 4:
|-
|-
| module load
| module load
| compiler/gnu|intel|pgi|...
| compiler/gnu or compiler/intel or compiler/llvm and others...
|-
| Availability
| [[bwUniCluster]] | [[BwForCluster_Chemistry]] | bwGRiD_tu
|-
|-
| License
| License
| [[Intel_Compiler|Intel]]: Commercial | [[GCC|GNU]]: GPL | PGI: Commercial
| [[Development/Intel_Compiler|Intel]]: Commercial | [[Development/GCC|GNU]]: GPL | LLVM: Apache 2 | PGI/NVIDIA: Commercial
|}
|}
<br>
= Description =
The basic operations can be performed with the same commands for all available compilers. For advanced usage such as optimization and profiling you should consult the best practice guide of the compiler you intend to use ([[BwHPC_BPG_Compiler#GCC|GCC]], [[BwHPC_BPG_Compiler#Intel Suite|Intel Suite]]).
<br>
<br>
= Versions and Availability =
A list of versions currently available compilers on the bwHPC-C5-Clusters can be obtained from the
<br>
<big>
<br>
[https://cis-hpc.uni-konstanz.de/prod.cis/ Cluster Information System CIS]
<br>
</big>
== GNU ==
{{#widget:Iframe
|url=https://cis-hpc.uni-konstanz.de/prod.cis/bwUniCluster/compiler/gnu
|width=99%
|height=470
}}
== Intel ==
{{#widget:Iframe
|url=https://cis-hpc.uni-konstanz.de/prod.cis/bwUniCluster/compiler/intel
|width=99%
|height=430
}}
== PGI ==
{{#widget:Iframe
|url=https://cis-hpc.uni-konstanz.de/prod.cis/bwUniCluster/compiler/pgi
|width=99%
|height=200
}}
On the command line interface of any bwHPC cluster you'll get a list of available versions
by using the command <br>
''''module avail compiler''''.
<pre>
$ : bwUniCluster
$ module avail compiler
------------------------ /opt/bwhpc/common/modulefiles -------------------------
compiler/gnu/4.5 compiler/intel/12.1
compiler/gnu/4.7(default) compiler/intel/13.1
compiler/gnu/4.8 compiler/intel/14.0
compiler/gnu/4.9 compiler/intel/15.0(default)
compiler/gnu/5.2
$ : bwForCluster (Justus)
$ module avail compiler
------------------------ /opt/bwhpc/common/modulefiles -------------------------
compiler/gnu/4.5 compiler/intel/15.0(default)
compiler/gnu/4.7(default) compiler/pgi/12.10(default)
compiler/gnu/4.8 compiler/pgi/12.10_static
compiler/gnu/4.9 compiler/pgi/13.7
compiler/gnu/5.2 compiler/pgi/13.7_static
compiler/intel/12.1 compiler/pgi/14.10
compiler/intel/13.1 compiler/pgi/14.10_static
compiler/intel/14.0
</pre>
<br>



= Loading the module =
= Description =
== Default Version ==

You can load the default version of the a compiler with the command<br>
Basically, compilers translate human-readable source code (e.g. C++ interpreted as adhering to ISO/IEC 14882:2014, encoded in UTF-8 text) into binary byte code (e.g. x86-64 with Linux ABI in ELF-format).
'''module load compiler'''/'''name-of-the-compiler-suite'''.
Compilers are complex software and have become very powerful in the last decades, to '''guide''' you as a programmer writing better, more portable, more performant programs. Use the compiler as a tool -- and best use multiple compilers on the same source code for best results.
<br>
The basic operations and hints can be performed with the same or similar commands on all available compilers. For advanced usage such as optimization and profiling you should consult the best practice guide of the compiler you intend to use ([[Development/GCC|GCC]], [[Development/Intel_Compiler|Intel Suite]]).
<u>Example with Intel on bwUniCluster</u>

More information about the MPI versions of the GNU and Intel Compilers is available here:
* [[Development/Parallel_Programming|Best Practices Guide for Parallel Programming]].

= Loading compilers as modules =

Modules and loading of modules is described [[Software_Modules_Lmod|here for Lmod]] and [[Environment_Modules|here for traditional Environment Modules]].

However, modules need to be mentioned, since on any system there's a pre-installed set of compilers (for C, C++ and usually Fortran), which are provided by the Linux distribution -- the so-called system compilers. Which however may lack certain options for optimization, for warnings or other features. On RedHat Enterprise Linux this is GNU compiler v8.3.1.
Be advised to check out the newer compilers available as modules.

Since Fortran (and very old C++) requires compiling and linking libraries with the very same compiler, many libraries, first-and-foremost the MPI libraries need to be provided for specific versions of a compiler.
On [[BwUniCluster_2.0]], these provided libraries will only be visible to <kbd>module avail</kbd>, once a compiler is loaded.
Hence, check out loading
<pre>
<pre>
$ module avail compiler/intel
$ module avail compiler/intel
...
------------------------ /opt/bwhpc/common/modulefiles -------------------------
compiler/intel/12.1 compiler/intel/14.0
$ module load compiler/intel/2021.4.0
...
compiler/intel/13.1 compiler/intel/15.0(default)
$ module load compiler/intel
$ module avail
$ module list
Currently Loaded Modulefiles:
1) compiler/intel/15.0(default)
</pre>
</pre>
to see the available MPI modules.
Here, we got the "default" version 15.0 (example).

<br>
All Intel, GCC and PGI have compilers for different programming languages which will be available
The module will try to load modules it needs to function.
If loading the module fails, check if you have already loaded the module with ''''module list''''.
== Specific (newer or older) Version ==
If you wish to load a specific compiler version and release (if available), you can do so using<br>
'''module load compiler''/''name-of-the-compiler-suite''/''version-of-the-compiler-suite'''<br>
to load the version you desires.
<br>
<u>Example with Intel compiler, version 14.0 on bwUniCluster</u>
<pre>
$ module avail compiler/intel
------------------------ /opt/bwhpc/common/modulefiles -------------------------
compiler/intel/12.1 compiler/intel/14.0
compiler/intel/13.1 compiler/intel/15.0(default)
$ module load compiler/intel/14.0
$ module list
Currently Loaded Modulefiles:
1) compiler/intel/14.0
</pre>
Intel C-Compiler "version 14.0" is loaded now (example).
<br>
<br>
All Intel, GCC and PGI have compilers for different languages which will be available
after the module is loaded.
after the module is loaded.

== Linux Original Compiler ==

The original Compiler installed on all compute nodes is GNU.
== Linux Default Compiler ==

The default Compiler installed on all compute nodes is the GNU Compiler Collection (GCC) or in short GNU compiler.
* Don't get distracted with the available compiler modules.
* Don't get distracted with the available compiler modules.
* Only the modules are loading the complete environments needed.
* Only the modules are loading the complete environments needed.
<u>Example</u>
<u>Example</u>
<pre>
<pre>
$ module clear # unload all modules
$ module purge # unload all modules
Are you sure you want to clear all loaded modules!? [n] y
$ module list # control
$ module list # control
No Modulefiles Currently Loaded.
No Modulefiles Currently Loaded.
$ gcc --version # see version of default Linux GNU compiler
$ gcc --version # see version of default Linux GNU compiler
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-16)
gcc (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
[...]
[...]
$ module load compiler/gnu # load default GNU compiler module
$ module load compiler/gnu # load default GNU compiler module
$ module list # control
$ module list # control
Currently Loaded Modulefiles:
Currently Loaded Modulefiles:
1) compiler/gnu/4.7(default)
1) compiler/gnu/10.2(default)
$ gcc --version # now, check the current (loaded) module
$ gcc --version # now, check the current (loaded) module
gcc (GCC) 4.7.3
gcc (GCC) 10.2.0
[...]
[...]
</pre>
</pre>

<br>
= Synoptical Tables =
= Synoptical Tables =

== Compilers (no MPI) ==

{| width=600px class="wikitable"
{| width=600px class="wikitable"
|-
|-
Line 139: Line 75:
! Command
! Command
|-
|-
| style="vertical-align:top;" rowspan="3" | <font color=green><big>Intel Composer</big></font><br><br> [[Intel_Compiler|&bull;&nbsp;Best Practice Guides on Intel Compiler Software]]
| style="vertical-align:top;" rowspan="3" | <font color=green><big>Intel Composer (pre-OneAPI)</big></font><br><br> [[Development/Intel_Compiler|&bull;&nbsp;Best Practice Guides on Intel Compiler Software]]
| C
| C
| icc
| icc
Line 149: Line 85:
| ifort
| ifort
|-
|-
| style="vertical-align:top;" rowspan="3" | <font color=green><big>GCC</big></font><br><br> [[GCC|&bull;&nbsp;Best Practice Guides on GNU Compiler Software]]
| style="vertical-align:top;" rowspan="3" | <font color=green><big>Intel OneAPI (llvm-based)</big></font><br><br> [[Development/Intel_Compiler|&bull;&nbsp;Best Practice Guides on Intel Compiler Software]]
| C
| icx
|-
| C++
| icpx
|-
| Fortran
| ifx
|-
| style="vertical-align:top;" rowspan="3" | <font color=green><big>GCC</big></font><br><br> [[Development/GCC|&bull;&nbsp;Best Practice Guides on GNU Compiler Software]]
| C
| C
| gcc
| gcc
Line 159: Line 105:
| gfortran
| gfortran
|-
|-
| style="vertical-align:top;" rowspan="3" | <font color=green><big>PGI</big></font>
| style="vertical-align:top;" rowspan="3" | <font color=green><big>LLVM</big></font>
| C
| clang
|-
| C++
| clang++
|-
| Fortran 77/90
| flang
|-
| style="vertical-align:top;" rowspan="3" | <font color=green><big>PGI/NVIDIA</big></font>
| C
| C
| pgcc
| pgcc
Line 169: Line 125:
| pgf77 or pgf90
| pgf77 or pgf90
|}
|}

<br>
== MPI compiler and Underlying Compilers ==

MPI implementations such as MPIch, Intel-MPI (derived from MPIch) or Open MPI provide compiler wrappers, easing the usage of MPI by providing the Include-Directory <kbd>-I</kbd> and required libraries as well as the MPI implementations library directorie <kbd>-L</kbd> for linking.
The following table lists available MPI compiler commands and the underlying compilers, compiler families, languages, and application binary interfaces (ABIs) that they support.

{| width=600px class="wikitable"
|-
! MPI Compiler Command !! Default Compiler !! Supported Language(s) !! Supported ABI's
|-
| colspan=4 style="background-color:#DCDCDC;" | Generic Compilers
|-
| mpicc || gcc, cc || C || 32/64 bit
|-
| mpicxx || g++ || C/C++ || 32/64 bit
|-
| mpifc || gfortran || Fortran77/Fortran 95 || 32/64 bit
|-
| colspan=4 style="background-color:#DCDCDC;" | [[Development/GCC|GNU Compiler]] Versions 3 and higher
|-
| mpigcc || gcc || C || 32/64 bit
|-
| mpigxx || g++ || C/C++ || 32/64 bit
|-
| mpif77 || g77 || Fortran 77 || 32/64 bit
|-
| mpif90 || gfortran || Fortran 95 || 32/64 bit
|-
| colspan=4 style="background-color:#DCDCDC;" | [[Development/Intel_Compiler|Intel Fortran, C++ Compilers]] Versions 13.1 through 14.0 and Higher
|-
| mpiicc || icc || C || 32/64 bit
|-
| mpiicpc || icpc || C++ || 32/64 bit
|-
|impiifort || ifort || Fortran77/Fortran 95 || 32/64 bit
|-
|}

= How to use =
= How to use =

The following compiler commands work for all the compilers in the list above even though
The following compiler commands work for all the compilers in the list above even though
the examples will be for '''icc''' only.
the examples will be for '''icc''' only.

== Commands ==
== Commands ==

When ''hello.c'' is a C source code file such as
The typical introduction is a "Hello World" program. The following C source code shows best practices:
<source lang=C style="font: normal normal 2em monospace">
<source lang="c">
#include <stdio.h>
#include <stdio.h> // for printf
int main() {
#include <stdlib.h> // for EXIT_SUCCESS and EXIT_FAILURE
printf("Hello world\n");
int main (int argc, char * argv[]) { // std. definition of a program taking arguments
return 0;
printf("Hello World\n"); // Unix Output is line-buffered, end line with New-line.
}
return EXIT_SUCCESS; // End program by returning 0 (No Error)
</source>
}</source>
it can be compiled and linked with the single command
It may be compiled and linked with the single command
<pre>$ icc hello.c -o hello</pre>
<pre>$ icc hello.c -o hello</pre>
to produce an executable named ''hello''.
to produce an executable named ''hello''.

<br>

<br>
This process can be divided into two steps:
This process can be divided into two steps:
<pre>
<pre>
Line 193: Line 189:
</pre>
</pre>
When using libraries you must sometimes specify where the
When using libraries you must sometimes specify where the
* include files are (option '''-I''') and where the
* include files are (option <kbd>-I</kbd>) and where the
* library files are (option '''-L''').
* library files are (option <kbd>-L</kbd>).
In addition you have to tell the compiler which
In addition you have to tell the compiler which
* library you want to use (option '''-l''').
* library you want to use (option <kbd>-l</kbd>).
For example after loading the module numlib/fftw you can compile code for fftw using
For example after loading the module numlib/fftw you can compile code for fftw using
<pre>
<pre>
Line 203: Line 199:
</pre>
</pre>
When the program crashes or doesn't produce the expected output the compiler can
When the program crashes or doesn't produce the expected output the compiler can
help you by printing warning messages:
help you by printing all warning messages <kbd>-Wall</kbd> and adding flags for debugging <kbd>-g</kbd>:
<pre>$ icc -Wall hello.c -o hello</pre>
<pre>$ icc -Wall -g hello.c -o hello</pre>


== Debugger ==
== Debugger ==

If the problem can't be solved this way you can inspect what exactly your program
If the problem can't be solved this way you can inspect what exactly your program
does [[BwHPC_BPG_Debugger|using a debugger]].
does using a debugger, e.g. [[Development/GDB|GDB]].

<br>
<font color=green>To use the debugger properly with your program you have to compile it with debug information (option -g)</font>:
<font color=green>To use the debugger properly with your program you have to compile it with debug information (option <kbd>-g</kbd>)</font>:

<br>
<u>Example</u>
<u>Example</u>
<pre>$ icc -g hello.c -o hello</pre>
<pre>$ icc -g hello.c -o hello</pre>
Although -Wall should always be set, the -g option should only be stated when you want
Although the compiler option <kbd>-Wall</kbd> (and possibly others) should always be set, the <kbd>-g</kbd> option should only be passed for
debugging purposes to find bugs.
to find bugs, since it may slow down execution and enlarges the binary due
to debugging symbols.
It may slow down execution and enlarges the binary due to debugging symbols.

<br>
== Optimization ==
== Optimization ==

The usual and common way to compile your source is to apply compiler optimization.
The usual and common way to compile your source is to apply compiler optimization.

<br>
Since there are many optimization options, as a start for now the <font color=green>optimization level -O2</font> is recommended:
Since there are many optimization options, as a start for now the <font color=green>optimization level -O2</font> is recommended:
<pre>$ icc -O2 hello.c -o hello</pre>
<pre>$ icc -O2 hello.c -o hello</pre>
<font color=red>Beware:</font>&nbsp;The optimization-flag used is a capital-O (like Otto) and not a 0 (Zero)!
<font color=red>Beware:</font>&nbsp;The optimization-flag used is a capital-O (like Otto) and not a 0 (Zero)!
<br>
<br>
Both compilers offer a multitude of options (with regard to the above and others),
one may check the complete list of options with short explanation on [[GCC|GCC]] and
[[Intel_Compiler|Intel Suite]] using option '''-v''' '''--help''':
<pre>$ icc -v --help hello.c -o hello</pre>
<br>
= Makefile =
If you're working in a project that already uses make, there should be a file called Makefile in the top-level directory.
Running:
<pre>$ make</pre>
should build the project from source.
== What is make? ==
Make is a tool designed to manage dependencies in a build process.
=== Simple Makefile for hello.c ===
For instance, if you have a source file called ''hello.c'' and you need to build
the binary/executable ''hello'', then you might have a Makefile in the same
directory that looks like this:
<source lang=make style="font: normal normal 2em monospace">
# define the C compiler to use
# you do not need this if you load the compiler module first
# # CC = icc


# define any compile-time flags
# -g # adds debugging information to the executable file
# -O2 # optimization level 2
# -Wall # turns on most, but not all, compiler warnings
CFLAGS = -g -O2 -Wall


All compilers offer a multitude of optimization options,
# unix/linux removal command used for 'make clean'
one may check the complete list of options with short explanation on [[Development/GCC|GCC]], [[LLVM|LLVM]] and
RM = rm -f
[[Development/Intel_Compiler|Intel Suite]] using option '''-v''' '''--help''':
<pre>
$ icc -v --help | less
$ gcc -v --help | less
$ clang -v --help | less
</pre>


Please note, that the optimization level <kbd>-O2</kbd> produces code for a general instruction set.
# the build target executable
If you want to set the instruction set available, and take advantage of AVX2 or AVX512f, you have to
# use a variable if you'd like to rename the binary later
either add the machine-dependent <kbd>-mavx512f</kbd> or set the specific architecture of your
TARGET = hello
target processor.

For [[BwUniCluster_2.0]] this depends on whether you run your application on any node, then you would select
# default action when make was invoked without options
the older Broadwell CPU, or whether You target the newer HPC nodes (which feature Xeon Gold 6230, aka "Cascade Lake"
default: all
architecture).

# starts section beginning with :hello
all: $(TARGET)

# main build section of this Makefile
$(TARGET): $(TARGET).c
$(CC) $(CFLAGS) -o $(TARGET) $(TARGET).c

# clean all garbage with 'make clean' oder 'make veryclean'
clean veryclean:
$(RM) $(TARGET) $(TARGET).o

# run the programm with 'make run'
run:
./$(TARGET)
</source>
When ordered to build a file, make will ensure that all dependencies are up to date, and it will not rebuild any those which need not be rebuilt.

=== Advaneced Makefile Example ===
Another makefile (using makedepend and more advanced make syntax). Here we use the GNU-C-Compiler.<br>
<pre>
<pre>
$ gcc -O2 -o hello hello.c # General optimization for any architecture
$module load compiler/gnu
$ gcc -O2 -march=broadwell -o hello hello.c # Will work on any compute node on bwUniCluster 2.0
$ module list
$ gcc -O2 -march=cascadelake -o hello hello.c # This may not run on Broadwell nodes
Currently Loaded Modulefiles:
1) compiler/gnu/4.7(default)
</pre>
A list of all defined environments set by the 'module load'-command can be
displayed by: 'module show compiler/gnu' (e.g. GNU compiler).
<pre>
$ module show compiler/gnu
-------------------------------------------------------------------
/opt/bwhpc/common/modulefiles/compiler/gnu/4.7:
[...]
setenv GNU_VERSION 4.7.3
setenv GNU_HOME /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64
setenv GNU_BIN_DIR /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/bin
setenv GNU_MAN_DIR /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/share/man
setenv GNU_LIB_DIR /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib64
prepend-path PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/bin
prepend-path MANPATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/share/man
prepend-path LD_RUN_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib
prepend-path LD_LIBRARY_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib
prepend-path LD_RUN_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib64
prepend-path LD_LIBRARY_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib64
setenv CC gcc
setenv CXX g++
setenv F77 gfortran
setenv FC gfortran
setenv F90 gfortran
[...]
</pre>
</pre>
This envs may be used in your lokal Makefile.
<source lang=make style="font: normal normal 2em monospace">
#
# 'make depend' uses makedepend to automatically generate dependencies
# (dependencies are added to end of Makefile)
# 'make' build executable file 'mycc'
# 'make clean' removes all .o and executable files
#


While adding <kbd>-march=broadwell</kbd> adds the compiler options such as <kbd>-mavx -mavx2 -msse3 -msse4 -msse4.1 -msse4.2 -mssse3</kbd>,
# define the C compiler to use.
adding <kbd>-march=cascadelake</kbd> will further this by <kbd>-mavx512bw -mavx512cd -mavx512dq -mavx512f -mavx512vl -mavx512vnni -mfma</kbd>,
# CC is set by the 'module load'-command so it's unnecessary here.
where <kbd>-mfma</kbd> is the setting for allowing fused-multiply-add.
# # CC = gcc
These options may provide considerable speed-up to your code as is.
'''Please note''' however, that Cascade Lake may throttle the processor's clock speed, when executing AVX-512 instructions, possibly running slower than
(older) AVX2 code paths would have.


You should then pay attention to vectorization attained by the compiler -- and concentrate on the time-consuming loops,
# define any compile-time flags
where the compiler was not able to vectorize.
CFLAGS = -Wall -g -O2
Further vectorization as described in the Best Practice Guides may help.
This information is available with the Intel compiler using <kbd>-qopt-report=5</kbd> producing a lot of output in <kbd>hello.optrpt</kbd>,
while GCC offers this information using <kbd>-fopt-info-all</kbd>


For GCC the options in use are best visible by calling <kbd>gcc -O2 -fverbose-asm -S -o hello.S hello.c</kbd>.
# define any directories containing header files other than /usr/include
The option <kbd>-fverbose-asm</kbd> stores all the options in the assembler file <kbd>hello.S</kbd>.
#
INCLUDES = -I/home/newhall/include
# -I../include -I$(GNU_INC_DIR) (example)


== Warnings and Error detection ==
# define library paths in addition to /usr/lib
All compilers have improved tremendously with regards to analyzing and detecting suspicious code: do make '''use''' of such warnings and hints.
# if I wanted to include libraries not in /usr/lib I'd specify
The amount of false positives has reduced and it will make your code more accessible, less error-prone and more portable.
# their path using -Lpath, something like:
LFLAGS = -L/home/newhall/lib -L../lib


The typical warning flags are <kbd>-Wall</kbd> to turn on ''all'' warnings.
# define any libraries to link into executable:
However, there's multiple other worthwhile warnings, which are not covered (since they might increase false positives, or since they are not yet considered so prominent).
# if I want to link in libraries (libx.so or libx.a) I use the -llibname
E.g. <kbd>-Wextra</kbd> turns on several other warnings, which will in the above example show that neither <kbd>argc</kbd> nor <kbd>argv</kbd> have been used inside of <kbd>main</kbd>.
# option, something like (this will link in libmylib.so and libm.so:
LIBS = -l$(GNU_LIB_DIR) -lmylib -lm


For LLVM's <kbd>clang</kbd> the flag <kbd>-Weverything</kbd> turns on all available warnings, albeit leading to many warnings on larger projects.
# define the C source files (examples only)
However, the fix-it hints are very helpful as well.
SRCS = emitter.c error.c init.c lexer.c main.c symbol.c parser.c


All the compilers offer the flag <kbd>-Werror</kbd> which turns any warning (allowing completion of compilation) into hard errors.
# define the C object files
<br>
#
<br>
# This uses Suffix Replacement within a macro:
# $(name:string1=string2)
# For each word in 'name' replace 'string1' with 'string2'
# Below we are replacing the suffix .c of all words in the macro SRCS
# with the .o suffix
#
OBJS = $(SRCS:.c=.o)


[[File:static_code_analysis.png|right|border|513px|Copyright: HS Esslingen)]]
# define the executable file
Another powerful feature available in GNU- and LLVM-compilers is ''static code analysis''', otherwise only available in Commercial tools, like [https://www.synopsys.com/software-integrity/security-testing/static-analysis-sast.html Coverity].
MAIN = mycc
Static code analysis evaluates '''each''' and '''every''' code path, making assumptions on input values and branches taken, detecting corner cases which might lead to real errors -- without having to actually execute this code path.


For GCC this is turned on using <kbd>-fanalyzer</kbd> which will detect e.g. cases of memory usage after a <kbd>free()</kbd> of said memory and many others. [https://gcc.gnu.org/onlinedocs/gcc/Static-Analyzer-Options.html#Static-Analyzer-Options GCC's documentation] on Static Analysis provides further details.
#
# The following part of the makefile is generic; it can be used to
# build any executable just by changing the definitions above and by
# deleting dependencies appended to the file from 'make depend'
#


For LLVM recompile your project using <kbd>scan-build</kbd>, e.g.:
.PHONY: depend clean
<pre>
$ scan-build make
</pre>


This produces warnings on <kbd>stdout</kbd>, but more importantly scan reports in directory <kbd>/scratch/scan-build-XXX</kbd>, where XXX is date and time of the build.
all: $(MAIN)
For example the output of Open MPI includes real issues of missed memory releases in error code paths:
@echo Simple compiler named mycc has been compiled

$(MAIN): $(OBJS)
$(CC) $(CFLAGS) $(INCLUDES) -o $(MAIN) $(OBJS) $(LFLAGS) $(LIBS)

# this is a suffix replacement rule for building .o's from .c's
# it uses automatic variables $<: the name of the prerequisite of
# the rule(a .c file) and $@: the name of the target of the rule (a .o file)
# (see the gnu make manual section about automatic variables)
.c.o:
$(CC) $(CFLAGS) $(INCLUDES) -c $< -o $@

clean:
$(RM) *.o *~ $(MAIN)

depend: $(SRCS)
makedepend $(INCLUDES) $^

# DO NOT DELETE THIS LINE -- make depend needs it
</source>

== Makefile structure ==
Makefiles contain definitions and rules.
* A definition has the form:<br>
VAR=value
* A rule has the form: <br>
output files: input files<br>
<TAB><TAB>commands to turn inputs to outputs
* All commands must be <font color=green>tab-indented</font>.
* # are marking the beginning of a comment. Rest of the line will be ignored.
* To reference the variable VAR, surround it with <font color=green>$(VAR)</font>.
* Try running ''''man make'''' for more details.
<br>
<TAB> = Tabulator
<br>
<br>
[[Category:Compiler_software]][[Category:bwUniCluster]]

Latest revision as of 01:14, 9 December 2022

Description Content
module load compiler/gnu or compiler/intel or compiler/llvm and others...
License Intel: Commercial | GNU: GPL | LLVM: Apache 2 | PGI/NVIDIA: Commercial


Description

Basically, compilers translate human-readable source code (e.g. C++ interpreted as adhering to ISO/IEC 14882:2014, encoded in UTF-8 text) into binary byte code (e.g. x86-64 with Linux ABI in ELF-format). Compilers are complex software and have become very powerful in the last decades, to guide you as a programmer writing better, more portable, more performant programs. Use the compiler as a tool -- and best use multiple compilers on the same source code for best results. The basic operations and hints can be performed with the same or similar commands on all available compilers. For advanced usage such as optimization and profiling you should consult the best practice guide of the compiler you intend to use (GCC, Intel Suite).

More information about the MPI versions of the GNU and Intel Compilers is available here:

Loading compilers as modules

Modules and loading of modules is described here for Lmod and here for traditional Environment Modules.

However, modules need to be mentioned, since on any system there's a pre-installed set of compilers (for C, C++ and usually Fortran), which are provided by the Linux distribution -- the so-called system compilers. Which however may lack certain options for optimization, for warnings or other features. On RedHat Enterprise Linux this is GNU compiler v8.3.1. Be advised to check out the newer compilers available as modules.

Since Fortran (and very old C++) requires compiling and linking libraries with the very same compiler, many libraries, first-and-foremost the MPI libraries need to be provided for specific versions of a compiler. On BwUniCluster_2.0, these provided libraries will only be visible to module avail, once a compiler is loaded. Hence, check out loading

$ module avail compiler/intel
...
$ module load compiler/intel/2021.4.0
...
$ module avail

to see the available MPI modules.

All Intel, GCC and PGI have compilers for different programming languages which will be available after the module is loaded.


Linux Default Compiler

The default Compiler installed on all compute nodes is the GNU Compiler Collection (GCC) or in short GNU compiler.

  • Don't get distracted with the available compiler modules.
  • Only the modules are loading the complete environments needed.

Example

$ module purge                     # unload all modules
$ module list                      # control
No Modulefiles Currently Loaded.
$ gcc --version                    # see version of default Linux GNU compiler
gcc (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
[...]
$ module load compiler/gnu         # load default GNU compiler module
$ module list                      # control
Currently Loaded Modulefiles:
  1) compiler/gnu/10.2(default)
$ gcc --version                    # now, check the current (loaded) module
gcc (GCC) 10.2.0
[...]

Synoptical Tables

Compilers (no MPI)

Compiler Suite Language Command
Intel Composer (pre-OneAPI)

• Best Practice Guides on Intel Compiler Software
C icc
C++ icpc
Fortran ifort
Intel OneAPI (llvm-based)

• Best Practice Guides on Intel Compiler Software
C icx
C++ icpx
Fortran ifx
GCC

• Best Practice Guides on GNU Compiler Software
C gcc
C++ g++
Fortran gfortran
LLVM C clang
C++ clang++
Fortran 77/90 flang
PGI/NVIDIA C pgcc
C++ pgCC
Fortran 77/90 pgf77 or pgf90

MPI compiler and Underlying Compilers

MPI implementations such as MPIch, Intel-MPI (derived from MPIch) or Open MPI provide compiler wrappers, easing the usage of MPI by providing the Include-Directory -I and required libraries as well as the MPI implementations library directorie -L for linking. The following table lists available MPI compiler commands and the underlying compilers, compiler families, languages, and application binary interfaces (ABIs) that they support.

MPI Compiler Command Default Compiler Supported Language(s) Supported ABI's
Generic Compilers
mpicc gcc, cc C 32/64 bit
mpicxx g++ C/C++ 32/64 bit
mpifc gfortran Fortran77/Fortran 95 32/64 bit
GNU Compiler Versions 3 and higher
mpigcc gcc C 32/64 bit
mpigxx g++ C/C++ 32/64 bit
mpif77 g77 Fortran 77 32/64 bit
mpif90 gfortran Fortran 95 32/64 bit
Intel Fortran, C++ Compilers Versions 13.1 through 14.0 and Higher
mpiicc icc C 32/64 bit
mpiicpc icpc C++ 32/64 bit
impiifort ifort Fortran77/Fortran 95 32/64 bit

How to use

The following compiler commands work for all the compilers in the list above even though the examples will be for icc only.

Commands

The typical introduction is a "Hello World" program. The following C source code shows best practices:

#include <stdio.h>                   // for printf
#include <stdlib.h>                  // for EXIT_SUCCESS and EXIT_FAILURE
int main (int argc, char * argv[]) { // std. definition of a program taking arguments
    printf("Hello World\n");         // Unix Output is line-buffered, end line with New-line.
    return EXIT_SUCCESS;             // End program by returning 0 (No Error)
}

It may be compiled and linked with the single command

$ icc hello.c -o hello

to produce an executable named hello.


This process can be divided into two steps:

$ icc -c hello.c
$ icc hello.o -o hello

When using libraries you must sometimes specify where the

  • include files are (option -I) and where the
  • library files are (option -L).

In addition you have to tell the compiler which

  • library you want to use (option -l).

For example after loading the module numlib/fftw you can compile code for fftw using

$ icc -c hello.c -I$FFTW_INC_DIR
$ icc hello.o -o hello -L$FFTW_LIB_DIR -lfftw3

When the program crashes or doesn't produce the expected output the compiler can help you by printing all warning messages -Wall and adding flags for debugging -g:

$ icc -Wall -g hello.c -o hello


Debugger

If the problem can't be solved this way you can inspect what exactly your program does using a debugger, e.g. GDB.

To use the debugger properly with your program you have to compile it with debug information (option -g):

Example

$ icc -g hello.c -o hello

Although the compiler option -Wall (and possibly others) should always be set, the -g option should only be passed for debugging purposes to find bugs. It may slow down execution and enlarges the binary due to debugging symbols.

Optimization

The usual and common way to compile your source is to apply compiler optimization.

Since there are many optimization options, as a start for now the optimization level -O2 is recommended:

$ icc -O2 hello.c -o hello

Beware: The optimization-flag used is a capital-O (like Otto) and not a 0 (Zero)!


All compilers offer a multitude of optimization options, one may check the complete list of options with short explanation on GCC, LLVM and Intel Suite using option -v --help:

$ icc -v --help | less
$ gcc -v --help | less
$ clang -v --help | less

Please note, that the optimization level -O2 produces code for a general instruction set. If you want to set the instruction set available, and take advantage of AVX2 or AVX512f, you have to either add the machine-dependent -mavx512f or set the specific architecture of your target processor. For BwUniCluster_2.0 this depends on whether you run your application on any node, then you would select the older Broadwell CPU, or whether You target the newer HPC nodes (which feature Xeon Gold 6230, aka "Cascade Lake" architecture).

$ gcc -O2 -o hello hello.c                        # General optimization for any architecture
$ gcc -O2 -march=broadwell -o hello hello.c       # Will work on any compute node on bwUniCluster 2.0
$ gcc -O2 -march=cascadelake -o hello hello.c     # This may not run on Broadwell nodes

While adding -march=broadwell adds the compiler options such as -mavx -mavx2 -msse3 -msse4 -msse4.1 -msse4.2 -mssse3, adding -march=cascadelake will further this by -mavx512bw -mavx512cd -mavx512dq -mavx512f -mavx512vl -mavx512vnni -mfma, where -mfma is the setting for allowing fused-multiply-add. These options may provide considerable speed-up to your code as is. Please note however, that Cascade Lake may throttle the processor's clock speed, when executing AVX-512 instructions, possibly running slower than (older) AVX2 code paths would have.

You should then pay attention to vectorization attained by the compiler -- and concentrate on the time-consuming loops, where the compiler was not able to vectorize. Further vectorization as described in the Best Practice Guides may help. This information is available with the Intel compiler using -qopt-report=5 producing a lot of output in hello.optrpt, while GCC offers this information using -fopt-info-all

For GCC the options in use are best visible by calling gcc -O2 -fverbose-asm -S -o hello.S hello.c. The option -fverbose-asm stores all the options in the assembler file hello.S.

Warnings and Error detection

All compilers have improved tremendously with regards to analyzing and detecting suspicious code: do make use of such warnings and hints. The amount of false positives has reduced and it will make your code more accessible, less error-prone and more portable.

The typical warning flags are -Wall to turn on all warnings. However, there's multiple other worthwhile warnings, which are not covered (since they might increase false positives, or since they are not yet considered so prominent). E.g. -Wextra turns on several other warnings, which will in the above example show that neither argc nor argv have been used inside of main.

For LLVM's clang the flag -Weverything turns on all available warnings, albeit leading to many warnings on larger projects. However, the fix-it hints are very helpful as well.

All the compilers offer the flag -Werror which turns any warning (allowing completion of compilation) into hard errors.

Copyright: HS Esslingen)

Another powerful feature available in GNU- and LLVM-compilers is static code analysis', otherwise only available in Commercial tools, like Coverity. Static code analysis evaluates each and every code path, making assumptions on input values and branches taken, detecting corner cases which might lead to real errors -- without having to actually execute this code path.

For GCC this is turned on using -fanalyzer which will detect e.g. cases of memory usage after a free() of said memory and many others. GCC's documentation on Static Analysis provides further details.

For LLVM recompile your project using scan-build, e.g.:

$ scan-build make

This produces warnings on stdout, but more importantly scan reports in directory /scratch/scan-build-XXX, where XXX is date and time of the build. For example the output of Open MPI includes real issues of missed memory releases in error code paths: