BinAC/Memory Usage

BwForCluster BinAC: Memory Usage

Often compute jobs get aborted because there is not enough memory available on the compute node to hold all the data. This page gives guidelines and tips for some tools how much memory should be requested for these tools and how the required memory scales with data and number of processors.

Specify memory in the job script

Please remember to specify the memory your job needs in the jobscript. Otherwise it can happen that the job is cancelled by the scheduler when it trys to allocate memory.

#PBS -l mem=xxgb

Tools

Mothur

When analyzing reads with Mothur, at some point a distance matrix will be created. This pair wise distances are saved in a file with the .dist prefix. This matrix is held in memory when you have Mothur commands like cluster or cluster.split.

As stated in this Mothur forum post, the required memory increases with the number of used processes. If you experience memory problems with your Mothur script, you may have to use fewer processes for the clustering.

Note: The more processors used the more memory is required. Each process will load a distance matrix into memory (RAM).