Development/General compiler usage
| Description | Content |
|---|---|
| module load | compiler/gnu|intel|llvm|pgi|... |
| License | Intel: Commercial | GNU: GPL | LLVM: Apache 2 | PGI: Commercial |
Description
The basic operations can be performed with the same commands for all available compilers. For advanced usage such as optimization and profiling you should consult the best practice guide of the compiler you intend to use (GCC, Intel Suite).
More information about the MPI versions of the GNU and Intel Compilers is available here:
Versions and availability
A list of versions currently available compilers on the bwHPC-C5-Clusters can be obtained from the CIS system. All the available versions are listed at the end of this page.
On the command line interface of any bwHPC cluster you'll get a list of available versions by using the command 'module avail compiler'.
$ : bwUniCluster 2.0 $ module avail compiler ---------------------- /opt/bwhpc/common/modulefiles/Core ----------------------- compiler/clang/9.0 compiler/gnu/8.3.1 compiler/gnu/9.3 compiler/gnu/10.2 (D) compiler/intel/19.0 compiler/intel/19.1 (L,D) compiler/llvm/10.0 compiler/pgi/2020 Please note, that further libraries, like MPI Libraries are dependent on the compiler and are only visible, if a compiler has been loaded. Due to this reason, the default system compiler (here '''compiler/gnu/8.3.1''') has it's own module file. $ : bwForCluster (Justus) $ module avail compiler ---------------------- /opt/bwhpc/common/modulefiles ----------------------- compiler/gnu/4.5 compiler/intel/15.0(default) compiler/gnu/4.7(default) compiler/pgi/12.10(default) compiler/gnu/4.8 compiler/pgi/12.10_static compiler/gnu/4.9 compiler/pgi/13.7 compiler/gnu/5.2 compiler/pgi/13.7_static compiler/intel/12.1 compiler/pgi/14.10 compiler/intel/13.1 compiler/pgi/14.10_static compiler/intel/14.0
Loading the module
Default Version
You can load the default version of the a compiler with the command
module load compiler/name-of-the-compiler-suite.
Example with Intel on bwUniCluster
$ module avail compiler/intel ---------------------- /opt/bwhpc/common/modulefiles/Core ----------------------- compiler/intel/19.0 compiler/intel/19.1 (L,D) $ module load compiler/intel $ module list Currently Loaded Modulefiles: 1) compiler/intel/19.1(default)
Here, we got the "default" version 19.1 (example).
The module will try to load modules it needs to function. If loading the module fails, check if you have already loaded the module with 'module list'.
Specific (newer or older) Version
If you wish to load a specific compiler version and release (if available), you can do so using
module load compiler/name-of-the-compiler-suite/version-of-the-compiler-suite
to load the version you desires.
Example with Intel compiler, version 19.0 on bwUniCluster
$ module avail compiler/intel ---------------------- /opt/bwhpc/common/modulefiles ----------------------- compiler/intel/19.0 compiler/intel/19.1 (L,D) $ module load compiler/intel/19.0 $ module list Currently Loaded Modulefiles: 1) compiler/intel/19.0
Intel C-Compiler "version 14.0" is loaded now (example).
All Intel, GCC and PGI have compilers for different languages which will be available
after the module is loaded.
Linux Original Compiler
The original Compiler installed on all compute nodes is GNU.
- Don't get distracted with the available compiler modules.
- Only the modules are loading the complete environments needed.
Example
$ module purge # unload all modules $ module list # control No Modulefiles Currently Loaded. $ gcc --version # see version of default Linux GNU compiler gcc (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5) [...] $ module load compiler/gnu # load default GNU compiler module $ module list # control Currently Loaded Modulefiles: 1) compiler/gnu/10.2(default) $ gcc --version # now, check the current (loaded) module gcc (GCC) 10.2.0 [...]
Synoptical Tables
Compilers (no MPI)
| Compiler Suite | Language | Command |
|---|---|---|
| Intel Composer • Best Practice Guides on Intel Compiler Software |
C | icc |
| C++ | icpc | |
| Fortran | ifort | |
| GCC • Best Practice Guides on GNU Compiler Software |
C | gcc |
| C++ | g++ | |
| Fortran | gfortran | |
| PGI | C | pgcc |
| C++ | pgCC | |
| Fortran 77/90 | pgf77 or pgf90 |
MPI compiler and Underlying Compilers
The following table lists available MPI compiler commands and the underlying compilers, compiler families, languages, and application binary interfaces (ABIs) that they support.
| MPI Compiler Command | Default Compiler | Supported Language(s) | Supported ABI's |
|---|---|---|---|
| Generic Compilers | |||
| mpicc | gcc, cc | C | 32/64 bit |
| mpicxx | g++ | C/C++ | 32/64 bit |
| mpifc | gfortran | Fortran77/Fortran 95 | 32/64 bit |
| GNU Compiler Versions 3 and higher | |||
| mpigcc | gcc | C | 32/64 bit |
| mpigxx | g++ | C/C++ | 32/64 bit |
| mpif77 | g77 | Fortran 77 | 32/64 bit |
| mpif90 | gfortran | Fortran 95 | 32/64 bit |
| Intel Fortran, C++ Compilers Versions 13.1 through 14.0 and Higher | |||
| mpiicc | icc | C | 32/64 bit |
| mpiicpc | icpc | C++ | 32/64 bit |
| impiifort | ifort | Fortran77/Fortran 95 | 32/64 bit |
How to use
The following compiler commands work for all the compilers in the list above even though the examples will be for icc only.
Commands
When hello.c is a C source code file such as
#include <stdio.h>
int main() {
printf("Hello world\n");
return 0;
}
it can be compiled and linked with the single command
$ icc hello.c -o hello
to produce an executable named hello.
This process can be divided into two steps:
$ icc -c hello.c $ icc hello.o -o hello
When using libraries you must sometimes specify where the
- include files are (option -I) and where the
- library files are (option -L).
In addition you have to tell the compiler which
- library you want to use (option -l).
For example after loading the module numlib/fftw you can compile code for fftw using
$ icc -c hello.c -I$FFTW_INC_DIR $ icc hello.o -o hello -L$FFTW_LIB_DIR -lfftw3
When the program crashes or doesn't produce the expected output the compiler can help you by printing warning messages:
$ icc -Wall hello.c -o hello
Debugger
If the problem can't be solved this way you can inspect what exactly your program does using a debugger.
To use the debugger properly with your program you have to compile it with debug information (option -g):
Example
$ icc -g hello.c -o hello
Although -Wall should always be set, the -g option should only be stated when you want to find bugs, since it may slow down execution and enlarges the binary due to debugging symbols.
Optimization
The usual and common way to compile your source is to apply compiler optimization.
Since there are many optimization options, as a start for now the optimization level -O2 is recommended:
$ icc -O2 hello.c -o hello
Beware: The optimization-flag used is a capital-O (like Otto) and not a 0 (Zero)!
Both compilers offer a multitude of options (with regard to the above and others),
one may check the complete list of options with short explanation on GCC and
Intel Suite using option -v --help:
$ icc -v --help hello.c -o hello
Please note, that the optimization level -O2 produces code for a general instruction set. If You want to set the instruction set available, and take advantage of AVX2 or AVX512f, You have to either add the machine-dependent -mavx512f or set the specific architecture of your target processor. For BwUniCluster_2.0 this depends on whether You run your application on any node, then You would select the older Broadwell CPU, or whether You target the newer HPC nodes (which feature Xeon Gold 6230, aka "Cascade Lake" architecture.
$ gcc -O2 -o hello hello.c # General optimization for any architecture $ gcc -O2 -march=broadwell -o hello hello.c # Will work on any compute node on bwUniCluster 2.0 $ gcc -O2 -march=cascadelake -o hello hello.c # This may not run on Broadwell nodes
While adding -march=broadwell adds the compiler options such as -mavx -mavx2 -msse3 -msse4 -msse4.1 -msse4.2 -mssse3, adding -march=cascadelake will further this by -mavx512bw -mavx512cd -mavx512dq -mavx512f -mavx512vl -mavx512vnni -mfma, where -mfma is the setting for allowing fused-multiply-add. These options may provide considerable speed-up to your code as is.
For GCC the options in use are best visible by calling gcc -O2 -fverbose-asm -S -o hello.S hello.c. The option -fverbose-asm stores all the options in the assembler file hello.S.
You should then pay attention to vectorization attained by the compiler -- and concentrate on the time-consuming loops, where the compiler was not able to vectorize. This information is available with the Intel compiler using -qopt-report=5 producing a lot of output in hello.optrpt, while GCC offers this information using -fopt-info-all
Makefile
If you're working in a project that already uses make, there should be a file called Makefile in the top-level directory. Running:
$ make
should build the project from source.
What is make?
Make is a tool designed to manage dependencies in a build process.
Simple Makefile for hello.c
For instance, if you have a source file called hello.c and you need to build the binary/executable hello, then you might have a Makefile in the same directory that looks like this:
# define the C compiler to use
# you do not need this if you load the compiler module first
# # CC = icc
# define any compile-time flags
# -g # adds debugging information to the executable file
# -O2 # optimization level 2
# -Wall # turns on most, but not all, compiler warnings
CFLAGS = -g -O2 -Wall
# unix/linux removal command used for 'make clean'
RM = rm -f
# the build target executable
# use a variable if you'd like to rename the binary later
TARGET = hello
# default action when make was invoked without options
default: all
# starts section beginning with :hello -> :hello = :$(TARGET)
all: $(TARGET)
# main build section of this Makefile
# same as: hello: hello.c
# $(CC) $(CFLAGS) -o hello hello.c
$(TARGET): $(TARGET).c
$(CC) $(CFLAGS) -o $(TARGET) $(TARGET).c
# clean all garbage with 'make clean' oder 'make veryclean'
clean veryclean:
$(RM) $(TARGET) $(TARGET).o
# run the programm with 'make run'
run:
./$(TARGET)
When ordered to build a file, make will ensure that all dependencies are up to date, and it will not rebuild any those which need not be rebuilt.
Load Compiler Environments
Another makefile (using makedepend and more advanced make syntax).
Here we use the GNU-C-Compiler.
$module load compiler/gnu $ module list Currently Loaded Modulefiles: 1) compiler/gnu/4.7(default)
A list of all defined environments set by the 'module load'-command can be displayed by: 'module show compiler/gnu' (e.g. GNU compiler).
$ module show compiler/gnu ------------------------------------------------------------------- /opt/bwhpc/common/modulefiles/compiler/gnu/4.7: [...] setenv GNU_VERSION 4.7.3 setenv GNU_HOME /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64 setenv GNU_BIN_DIR /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/bin setenv GNU_MAN_DIR /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/share/man setenv GNU_LIB_DIR /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib64 prepend-path PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/bin prepend-path MANPATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/share/man prepend-path LD_RUN_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib prepend-path LD_LIBRARY_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib prepend-path LD_RUN_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib64 prepend-path LD_LIBRARY_PATH /opt/bwhpc/common/compiler/gnu/4.7.3/x86_64/lib64 setenv CC gcc setenv CXX g++ setenv F77 gfortran setenv FC gfortran setenv F90 gfortran [...]
Advaneced Makefile Examples
This envs may be used in your local Makefile.
#
# 'make depend' uses makedepend to automatically generate dependencies
# (dependencies are added to end of Makefile)
# 'make' build executable file 'mycc'
# 'make clean' removes all .o and executable files
#
# define the C compiler to use.
# CC is set by the 'module load'-command so it's unnecessary here.
# # CC = gcc
# define any compile-time flags
CFLAGS = -Wall -g -O2
# define any directories containing header files other than /usr/include
#
INCLUDES = -I/home/newhall/include
# -I../include -I$(GNU_INC_DIR) (example)
# define library paths in addition to /usr/lib
# if I wanted to include libraries not in /usr/lib I'd specify
# their path using -Lpath, something like:
LFLAGS = -L/home/newhall/lib -L../lib
# define any libraries to link into executable:
# if I want to link in libraries (libx.so or libx.a) I use the -llibname
# option, something like (this will link in libmylib.so and libm.so:
LIBS = -l$(GNU_LIB_DIR) -lmylib -lm
# define the C source files (examples only)
SRCS = emitter.c error.c init.c lexer.c main.c symbol.c parser.c
# define the C object files
#
# This uses Suffix Replacement within a macro:
# $(name:string1=string2)
# For each word in 'name' replace 'string1' with 'string2'
# Below we are replacing the suffix .c of all words in the macro SRCS
# with the .o suffix
#
OBJS = $(SRCS:.c=.o)
# define the executable file
MAIN = mycc
#
# The following part of the makefile is generic; it can be used to
# build any executable just by changing the definitions above and by
# deleting dependencies appended to the file from 'make depend'
#
.PHONY: depend clean
all: $(MAIN)
@echo Simple compiler named mycc has been compiled
$(MAIN): $(OBJS)
$(CC) $(CFLAGS) $(INCLUDES) -o $(MAIN) $(OBJS) $(LFLAGS) $(LIBS)
# this is a suffix replacement rule for building .o's from .c's
# it uses automatic variables $<: the name of the prerequisite of
# the rule(a .c file) and $@: the name of the target of the rule (a .o file)
# (see the gnu make manual section about automatic variables)
.c.o:
$(CC) $(CFLAGS) $(INCLUDES) -c $< -o $@
clean:
$(RM) *.o *~ $(MAIN)
depend: $(SRCS)
makedepend $(INCLUDES) $^
# DO NOT DELETE THIS LINE -- make depend needs it
This is an excerpt from the CIS makefile to show how you can do branchings in a makefile.
We use the shell-command $(shell uname -s) to determine the machine is a Linux.
[...] # # LINUX system # UNAME_S := $(shell uname -s) ifeq ($(UNAME_S),Linux) LDFLAGS=-g -m32 -lm -lcrypt CFLAGS=-g -Wimplicit -Wunused -Wformat -Werror -Wreturn-type \ -Wmissing-prototypes -m32 -funsigned-char -Wno-parentheses \ -D_XOPEN_SOURCE -D_GNU_SOURCE -Wno-pointer-sign -Wno-unused-but-set-variable endif # # MAC-OS # ifeq ($(UNAME_S),Darwin) ....... [...] OBJ=db/db.o db/app.o db/w3tool.o db/w3lib.o db/maildecode.o ALL=.dependent db.c w3dbs cis svnout x tools SRC=db.c app.c w3tool.c w3lib.c cis.c x.c maildecode.c db/%.o: %.c @echo "compile $< ..." @$(CC) $(CFLAGS) -c $< -o $@ [...] all: $(ALL) clean: @echo "cleaning ..." @rm -rf *.o $(ALL) core tags db.h db.c html/model.html a.out gmon.out core.* x tools svnout w3lib.tgz *.dSYM *.gcno *.gcda *.gcov g* .dependent $(OBJ) cis: db/cis.o $(OBJ) @echo "building cis ..."
Makefile structure
Makefiles contain definitions and rules.
- A definition has the form:
VAR=value
- A rule has the form:
output files: input files
<TAB><TAB>commands to turn inputs to outputs
- All commands must be tab-indented.
- # are marking the beginning of a comment. Rest of the line will be ignored.
- To reference the variable VAR, surround it with $(VAR).
- Try running 'man make' for more details.
<TAB> = Tabulator