BinAC/Memory Usage

From bwHPC Wiki
< BinAC
Revision as of 14:29, 1 December 2020 by F Bartusch (talk | contribs) (Created page with "= Memory Usage = Often compute jobs get aborted because not enough memory is available to hold all the data. This page should give guidelines and tips for some tools how much...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

1 Memory Usage

Often compute jobs get aborted because not enough memory is available to hold all the data. This page should give guidelines and tips for some tools how much memory should be requested for specific tools and how the required memory scales with data and number of processors.

1.1 Specify memory in the job script

Please remember to specify the memory your job needs in the jobscript. Otherwise it can happen that the job is cancelled by the scheduler when it trys to allocate memory on the compute node.

#PBS -l mem=xxgb

1.2 Tools

1.2.1 Mothur

When analysing reads with Mothur, at some point a distance matrix will be created. This pair wise distances are saved in a file with the .dist prefix. When using this distance matrix with Mothur commands like cluster or cluster.split, this distance matrix is held in memory.

As stated in this Mothur forum post, the required memory increases with the number of used processes. If you experience memory problems with your Mothur script, you may have to use fewer processes for the clustering.