BinAC2/Software/Alphafold: Difference between revisions

From bwHPC Wiki
Jump to navigation Jump to search
No edit summary
Tag: Reverted
(Undo revision 16554 by F Bartusch (talk))
Tag: Undo
 
Line 1: Line 1:
{{Softwarepage|bio/bismark/3.1.0}}
{{Softwarepage|bio/alphafold}}


{| width=600px class="wikitable"
{| width=600px class="wikitable"
Line 6: Line 6:
|-
|-
| module load
| module load
| bio/bismark/3.1.0
| bio/alphafold
|-
|-
| License
| License
| GPL-3.0 - see [https://github.com/FelixKrueger/Bismark/blob/master/license.txt]
| ACC BY-NC-SA 4.0 - see [https://github.com/google-deepmind/alphafold3/blob/main/LICENSE]
|-
|-
| Citing
| Citing
| See [https://github.com/google-deepmind/alphafold3?tab=readme-ov-file#citing-this-work]
| [https://academic.oup.com/bioinformatics/article/27/11/1571/216956 Krueger F, Andrews SR. Bismark: a flexible aligner and methylation caller for Bisulfite-Seq applications. Bioinformatics 27(11), 1571-1572 (2011)]
|-
|-
| Links
| Links
| Bismark Documentation: [https://felixkrueger.github.io/Bismark/] <br>
| DeepMind AlphaFold Website: [https://deepmind.google/technologies/alphafold/] <br>
Bismark Repository: [https://github.com/FelixKrueger/Bismark]
Alphafold 3 Repository: [https://github.com/google-deepmind/alphafold3]
|}
|}


= Description =
= Description =


AlphaFold 3 developed by DeepMind predicts the 3D structure and interactions of biological molecules: proteins, DNA, RNA, ligands, and other small molecules.
Bismark maps bisulfite-converted reads (WGBS, RRBS, EM-seq, ...) to a reference genome and calls cytosine methylation in CpG, CHG and CHH context.


BinAC 2 provides almost everything you need for working with Alphafold 3:
The module <code>bio/bismark/3.1.0</code> is the '''Rust rewrite''' of Bismark. Upstream validates its default output as byte-identical to Perl Bismark v0.25.1. We checked this on BinAC 2 with the upstream test data (all outputs identical) and with 10 million human EM-seq read pairs (identical alignments). It is much faster than the Perl version, mostly in the methylation extraction:
* Alphafold 3 installed in an Apptainer image
* Alphafold 3 database


'''BUT''' you have to bring your own parameter file! Due to license issues we are not allowed to share them publicly. You can find information on how to obtain them [https://github.com/google-deepmind/alphafold3?tab=readme-ov-file#obtaining-model-parameters here].
{| class="wikitable"
! Step (10M read pairs, GRCh38, 16 cores) !! Perl 0.25.1 (<code>--multicore 8</code>) !! Rust 3.1.0 (<code>-p 8</code>)
|-
| Alignment || 785 s || 699 s
|-
| Deduplication || 190 s || 169 s
|-
| Methylation extraction (with cytosine report) || 3067 s || 293 s
|-
| '''Total''' || '''4042 s''' || '''1162 s'''
|}


= Interpreting Predictions =
All classic tool names are available: <code>bismark</code>, <code>bismark_genome_preparation</code>, <code>deduplicate_bismark</code>, <code>bismark_methylation_extractor</code>, <code>bismark2bedGraph</code>, <code>coverage2cytosine</code>, <code>bam2nuc</code>, <code>bismark2report</code>, <code>bismark2summary</code> and more. Your existing commands keep working, but see [[#Moving from Perl Bismark 0.25.1|below]] for the thread options.


When using AlphaFold3, users should treat predicted structures as hypotheses rather than definitive representations of molecular reality.
The module loads <code>bio/bowtie2/2.5.5</code> automatically. HISAT2 (<code>--hisat2</code>) and minimap2 (<code>--minimap2</code>) are optional and must be loaded separately. samtools is not needed.
While the method often produces highly accurate models, confidence can vary substantially across regions, especially for flexible loops, disordered segments, or novel interactions not well represented in training data. Users should therefore critically assess confidence metrics provided by the model and, where possible, validate predictions against experimental data or independent computational approaches. Biological plausibility, consistency with known functional or biochemical evidence, and sensitivity to alternative inputs should also be considered. Overall, AlphaFold3 outputs are most powerful when used as guidance for downstream analysis and experimental design, not as final ground truth.


AlphaFold 3 supplies multiple confidence metrics to help you critically assess its predictions:
= Quick start =


* Predicted LDDT (pLDDT): predicted atomic coordinates are accompanied by pLDDT scores. These reflect AlphaFold 3’s local confidence in the prediction of the position of that particular atom.
<pre>
* Predicted Aligned Error (PAE) scores and a PAE plot: an indication of AlphaFold’s confidence in the packing and relative positions of domains, molecular chains such as proteins and DNA, and other entities like ligands and ions.
#!/bin/bash
* Predicted TM (pTM) score: a single-value metric reflecting the accuracy of the overall predicted structure.
#SBATCH --nodes=1
* Interface-predicted TM (ipTM) score: measures the accuracy of predictions of one component of the complex relative to the other components of the complex.
#SBATCH --ntasks=1
* Per chain pTM and per-chain pair ipTM: confidence in individual chains or pairs of chains.
#SBATCH --cpus-per-task=32
#SBATCH --mem=32G
#SBATCH --time=04:00:00


Further Reading:
module load bio/bismark/3.1.0


* [https://www.ebi.ac.uk/training/online/courses/alphafold/alphafold-3-and-alphafold-server/introducing-alphafold-3/how-have-alphafold-3s-predictions-been-validated/ AlphaFold3 validation overview]
GENOME=/path/to/genome_folder # prepared with bismark_genome_preparation
cd "$TMPDIR" # fast local scratch of the job


* [https://www.ebi.ac.uk/training/online/courses/alphafold/alphafold-3-and-alphafold-server/how-to-assess-the-quality-of-alphafold-3-predictions/ How to assess the quality of AlphaFold3 predictions]
bismark --genome "$GENOME" -p 15 \
-1 /path/to/sample_R1.fq.gz -2 /path/to/sample_R2.fq.gz
deduplicate_bismark -p --parallel 4 sample_R1_bismark_bt2_pe.bam
bismark_methylation_extractor -p --gzip --bedGraph \
sample_R1_bismark_bt2_pe.deduplicated.bam


= Usage =
mkdir -p /path/to/results
cp *.bam *.txt *.gz *.html /path/to/results/ 2>/dev/null
</pre>


AlphaFold's algorithm can be devided into two steps:
For 50 million read pairs against the human genome this takes about one hour.


# CPU-only part: Computation of several multiple sequence alignments (MSA)
= Preparing the genome =
# GPU-part: MSA used as input for neural network for infering structure


This results in two optimal resource profiles regarding the number of CPU cores and GPUs.
<pre>
Therefore we provide two template jobscripts for both steps at <code>/opt/bwhpc/common/bio/alphafold/3.0.1/bwhpc-examples</code>.
module load bio/bismark/3.1.0
bismark_genome_preparation --parallel 16 /path/to/genome_folder/
</pre>


== Input ==
* '''Large genomes:''' Bowtie 2 builds a large index (<code>.bt2l</code>) automatically for genomes above 2 GiB (human, mouse and most mammals); you do not need <code>--large-index</code>. Giving it anyway does no harm. Bismark finds the large index automatically when aligning.
* '''Use one file extension for all FASTA files''' in the genome folder. Bismark reads only the first non-empty group of <code>.fa</code>, <code>.fa.gz</code>, <code>.fasta</code>, <code>.fasta.gz</code> and silently ignores the others. For example, if your genome is <code>genome.fasta.gz</code> and you add a spike-in as <code>lambda.fa.gz</code>, only the spike-in is indexed. Rename the files so they all end the same way.
* Check the result: <code>grep -c '^>' /path/to/genome_folder/Bisulfite_Genome/CT_conversion/genome_mfa.CT_conversion.fa</code> must show the number of sequences of your genome.
* Bismark builds two indexes (C to T and G to A converted genome); <code>--parallel N</code> builds both at the same time with N threads each. '''Request <code>--cpus-per-task=32 --mem=64G</code>''' for <code>--parallel 16</code>: each index build needs about 25 GiB, and both run at the same time. For the human genome this takes less than one hour.
* Prepare the index once and reuse it for all samples.


You can provide inputs to <code>run_alphafold.py</code> in one of two ways:
= Choosing threads and resources =


* Single input file: Use the <code>--json_path</code> flag followed by the path to a single JSON file.
All recommendations below were measured on BinAC 2 standard nodes (AMD EPYC 7543) with human GRCh38 and EM-seq reads (2 x 151 bp). Other genomes and read lengths need different times and memory, but the thread recommendations apply in general.
* Multiple input files: Use the <code>--input_dir</code> flag followed by the path to a directory of JSON files.


Please see the official [https://github.com/google-deepmind/alphafold3/blob/main/docs/input.md#top-level-structure AlphaFold 3 documentation] for more information regarding JSON file structure.
== Recommended job size ==


The example jobscript uses a very simple input JSON:
Request '''<code>--cpus-per-task=32</code> and <code>--mem=32G</code>'''. On BinAC 2, Slurm counts hardware threads as CPUs, so this gives you 16 physical cores with 2 hardware threads each. In our tests this was the most efficient job size: with twice as many cores, the alignment of one sample was only 1.26x faster.


<syntaxhighlight>
If you have many samples, run one job per sample with these settings rather than fewer jobs with more cores.
{
"name": "2PV7",
"sequences": [
{
"protein": {
"id": ["A", "B"],
"sequence": "GMRESYANENQFGFKTINSDIHKIVIVGGYGKLGGLFARYLRASGYPISILDREDWAVAESILANADVVIVSVPINLTLETIERLKPYLTENMLLADLTSVKREPLAKMLEVHTGAVLGLHPMFGADIASMAKQVVVRCDGRFPERYEWLLEQIQIWGAKIYQTNATEHDHNMTYIQALRHFSTFANGLHLSKQPINLANLLALSSPIYRLELAMIGRLFAQDAELYADIIMDKSENLAVIETLKQTYDEALTFFENNDRQGFIDAFHKVRDWFGDYSEQFLKESRQLLQQANDLKQG"
}
}
],
"modelSeeds": [1],
"dialect": "alphafold3",
"version": 1
}
</syntaxhighlight>


== AlphaFold 3 database ==
== Alignment: use <code>-p</code>, not <code>--multicore</code> ==


Don't change <code>--db_dir=$ALPHAFOLD_DATABASES</code> without a good reason. Upon loading the AlphaFold 3 module, the environment variable <code>$ALPHAFOLD_DATABASES</code> is set to the centrally stored database at <code>/pfs/10/project/db/alphafold/3.0.1/databases/</code>.
Bismark runs 2 Bowtie 2 processes (4 with <code>--non_directional</code>), each with <code>-p N</code> threads. Choose


== AlphaFold 3 model parameters ==
: <code>-p</code> = cpus-per-task / 2 - 1


Set the path to your AlphaFold 3 model parameters you received from DeepMind after applying for it. If you place the file <code>af3.bin.zst</code> in the directory <code>$HOME/af3-models</code> the example scripts will work out of the box.
so <code>-p 15</code> for <code>--cpus-per-task=32</code>. The 2 remaining hardware threads are used by Bismark itself.


Please note: You can also store the parameters in your project directory or a workspace for slightly better performance. The template uses <code>$HOME/af3-models</code> because this environment variable works for everyone.
{| class="wikitable"
! Setting (10M read pairs, 16 cores) !! Time
|-
| <code>-p 8</code> || 720 s
|-
| <code>-p 15</code> || '''551 s'''
|-
| <code>-p 16</code> || 599 s
|}


== Output ==
* <code>--multicore</code> splits the input and starts a full Bismark with its own copy of the index per worker. It was clearly slower than <code>-p</code> with the same number of threads (559 s vs 374 s on a full node) and needs much more memory. Results are identical either way.
* Do not use more threads than you requested: oversubscribed jobs get slower, not faster.


You can set the output directory via the <code>--output_dir=$ALPHAFOLD_RESULTS_DIR</code> option. The template jobscript creates a workspace called <code>alphafold</code> as output directory.
== Deduplication ==


For every input job, AlphaFold 3 writes all its outputs in a directory called by the sanitized version of the job name. E.g. for job name "My first fold (TEST)", AlphaFold 3 will write its outputs in a directory called <code>My_first_fold_TEST</code> (the case is respected). If such directory already exists, AlphaFold 3 will append a timestamp to the directory name to avoid overwriting existing data unless <code>--force_output_dir</code> is passed.
Use <code>--parallel 4</code>. For 10M read pairs this reduces the time from 169 s to 36 s. The output is identical.


Please see the official [https://github.com/google-deepmind/alphafold3/blob/main/docs/output.md AlphaFold 3 documentation] for more information regarding output directory structure.
== Methylation extraction ==


= CPU-only: Multiple Sequence Alignment =
* <code>--multicore</code> does not make the extraction of BAM files faster. The extractor already uses about 4 cores on its own.
* '''Use <code>--cytosine_report</code> only if you need it.''' It writes every CpG of the genome, including uncovered ones, and takes most of the extraction time and memory:
{| class="wikitable"
! Extraction (10M read pairs) !! Time !! Memory
|-
| <code>--bedGraph</code> || 106 s || 0.8 GiB
|-
| <code>--bedGraph --cytosine_report</code> || 293 s || 3.4 GiB
|}
: Many downstream tools can read the coverage file (<code>*.bismark.cov.gz</code>) directly. If you need the cytosine report later, create it from the coverage file with <code>coverage2cytosine</code>.


In the beginning, AlphaFold 3 computes three multiple sequence alignments (MSA).
== Time, memory and scratch space ==
These MSAs are computed on the CPU sequentially and the number of threads are hard-coded:


* jackhmmer on UniRef90 using 8 threads</br>
Measured with <code>--cpus-per-task=32</code>, <code>-p 15</code>, <code>--parallel 4</code>, human GRCh38:
* jackhmmer on MGnify using 8 threads</br>
* HHblits on BFD + Uniclust30 using 4 threads</br>


We provide a template jobscript at: <code>/opt/bwhpc/common/bio/alphafold/3.0.1/bwhpc-examples/binac2-af3-alignment.slurm
{| class="wikitable"
</code>
! Read pairs !! Alignment !! Deduplication !! Extraction !! Extraction with cytosine report !! Scratch space during alignment
|-
| 10 million || 9 min || 0.6 min || 1.6 min || 4.8 min || 10 GB
|-
| 25 million || 23 min || 1.4 min || 3.9 min || 7.2 min || 26 GB
|-
| 50 million || 45 min || 2.6 min || 7.6 min || 11 min || 47 GB
|}


You can change <code>--time</code> and <code>--mem</code>, although these are sensible defaults.
Rules of thumb:
* '''Time:''' about 1 minute per million read pairs for alignment, deduplication and extraction together. In jobs that share a node with other jobs, plan about 15% more.
* '''Memory:''' about 12 GiB for aligning against the human genome, '''independent of the number of reads'''. Deduplication and extraction stay below 4 GiB up to 50M read pairs. A job with <code>--mem=16G</code> processed 50M read pairs successfully; <code>--mem=32G</code> leaves a safe margin.
* '''Scratch space:''' about 1 GB per million read pairs during the alignment (temporary converted read files), plus the input and output files. Work in the job's <code>$TMPDIR</code> (local disk) and copy the results back at the end. Alignment reads and writes a lot of data (50M pairs: about 180 GB read, 130 GB written); doing this on the parallel file system is slower and loads it for all users.


<syntaxhighlight>
= Moving from Perl Bismark 0.25.1 =
#!/bin/bash
#SBATCH --partition=compute
#SBATCH --nodes=1


# Alphafold creates alignments sequentially, using at max 8 cores.
* Results are the same as with Perl Bismark 0.25.1. The <code>@PG</code> line of the BAM header therefore still reports <code>VN:v0.25.1</code>.
#SBATCH --ntasks-per-node=8
* '''Replace <code>--multicore</code> by <code>-p</code>''' for the alignment, and add <code>--parallel 4</code> to <code>deduplicate_bismark</code>.
* If you ran Perl Bismark with <code>--multicore</code>, a tiny fraction of your methylation calls may differ from a new run (399 of 58.6 million CpG report lines in our test). Perl's <code>--multicore</code> writes the BAM in a different order, so deduplication keeps different but equivalent duplicates. The alignments themselves are identical.


#SBATCH --time=10:00:00
= Support =
#SBATCH --mem=100gb
</syntaxhighlight>


The MSAs are stored in the directory specified by <code>--output_dir</code>.
If you have questions or problems, contact hpcmaster [at] uni-tuebingen.de.

= GPU-part: Model Inference =

Aftere computing the MSAs, AlphaFold performs model inference on the GPU. Only one GPU is used.
This use case has this optimal resource profile:

<syntaxhighlight>
#!/bin/bash
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --cpus-per-task=8
#SBATCH --gres=gpu:a100:1
#SBATCH --time=06:00:00
#SBATCH --mem=100gb
</syntaxhighlight>

The template jobscript only differs to CPU-only part in the option <code>--norun_data_pipeline</code>. This means the MSAs aren't recomputed, but are taken from the previous CPU-only job.

= Chaining CPU- and GPU-Jobs =

To run the data pipeline first and then start the inference job as soon as the first one is finished, you can chain them like this:

<syntaxhighlight>
JOBID=$(sbatch --parsable binac2-af3-alignment.slurm)
sbatch --dependency=afterok:$JOBID binac2-af3-inference.slurm
</syntaxhighlight>






<!-- This is a comment
= Benchmark on BinAC 2 =

We ran some CASP14 targets with the <code>--benchmark=true</code> on BinAC. The following table gives you some guidance for choosing meaningful memory and walltime values.

{| class="wikitable" style="margin:auto"
|+ Benchmark results on BinAC (work in progress)
|-
! Target !! #Residues !! jackhmmer UniRef90 [s] !! jackhmmer MGnify [s] !! HHblits on BFD [s] !! Inference [s] !! Memory Usage [GB]
|-
| ... || ... || ... || ... || ... || ... || ...
|-
| ... || ... || ... || ... || ... || ... || ...
|-
| ... || ... || ... || ... || ... || ... || ...
|}
-->

Latest revision as of 15:03, 28 October 2026

The main documentation is available on the cluster via module help bio/alphafold. Most software modules for applications provide working example batch scripts.


Description Content
module load bio/alphafold
License ACC BY-NC-SA 4.0 - see [1]
Citing See [2]
Links DeepMind AlphaFold Website: [3]

Alphafold 3 Repository: [4]

Description

AlphaFold 3 developed by DeepMind predicts the 3D structure and interactions of biological molecules: proteins, DNA, RNA, ligands, and other small molecules.

BinAC 2 provides almost everything you need for working with Alphafold 3:

  • Alphafold 3 installed in an Apptainer image
  • Alphafold 3 database

BUT you have to bring your own parameter file! Due to license issues we are not allowed to share them publicly. You can find information on how to obtain them here.

Interpreting Predictions

When using AlphaFold3, users should treat predicted structures as hypotheses rather than definitive representations of molecular reality. While the method often produces highly accurate models, confidence can vary substantially across regions, especially for flexible loops, disordered segments, or novel interactions not well represented in training data. Users should therefore critically assess confidence metrics provided by the model and, where possible, validate predictions against experimental data or independent computational approaches. Biological plausibility, consistency with known functional or biochemical evidence, and sensitivity to alternative inputs should also be considered. Overall, AlphaFold3 outputs are most powerful when used as guidance for downstream analysis and experimental design, not as final ground truth.

AlphaFold 3 supplies multiple confidence metrics to help you critically assess its predictions:

  • Predicted LDDT (pLDDT): predicted atomic coordinates are accompanied by pLDDT scores. These reflect AlphaFold 3’s local confidence in the prediction of the position of that particular atom.
  • Predicted Aligned Error (PAE) scores and a PAE plot: an indication of AlphaFold’s confidence in the packing and relative positions of domains, molecular chains such as proteins and DNA, and other entities like ligands and ions.
  • Predicted TM (pTM) score: a single-value metric reflecting the accuracy of the overall predicted structure.
  • Interface-predicted TM (ipTM) score: measures the accuracy of predictions of one component of the complex relative to the other components of the complex.
  • Per chain pTM and per-chain pair ipTM: confidence in individual chains or pairs of chains.

Further Reading:

Usage

AlphaFold's algorithm can be devided into two steps:

  1. CPU-only part: Computation of several multiple sequence alignments (MSA)
  2. GPU-part: MSA used as input for neural network for infering structure

This results in two optimal resource profiles regarding the number of CPU cores and GPUs. Therefore we provide two template jobscripts for both steps at /opt/bwhpc/common/bio/alphafold/3.0.1/bwhpc-examples.

Input

You can provide inputs to run_alphafold.py in one of two ways:

  • Single input file: Use the --json_path flag followed by the path to a single JSON file.
  • Multiple input files: Use the --input_dir flag followed by the path to a directory of JSON files.

Please see the official AlphaFold 3 documentation for more information regarding JSON file structure.

The example jobscript uses a very simple input JSON:

{
  "name": "2PV7",
  "sequences": [
    {
      "protein": {
        "id": ["A", "B"],
        "sequence": "GMRESYANENQFGFKTINSDIHKIVIVGGYGKLGGLFARYLRASGYPISILDREDWAVAESILANADVVIVSVPINLTLETIERLKPYLTENMLLADLTSVKREPLAKMLEVHTGAVLGLHPMFGADIASMAKQVVVRCDGRFPERYEWLLEQIQIWGAKIYQTNATEHDHNMTYIQALRHFSTFANGLHLSKQPINLANLLALSSPIYRLELAMIGRLFAQDAELYADIIMDKSENLAVIETLKQTYDEALTFFENNDRQGFIDAFHKVRDWFGDYSEQFLKESRQLLQQANDLKQG"
      }
    }
  ],
  "modelSeeds": [1],
  "dialect": "alphafold3",
  "version": 1
}

AlphaFold 3 database

Don't change --db_dir=$ALPHAFOLD_DATABASES without a good reason. Upon loading the AlphaFold 3 module, the environment variable $ALPHAFOLD_DATABASES is set to the centrally stored database at /pfs/10/project/db/alphafold/3.0.1/databases/.

AlphaFold 3 model parameters

Set the path to your AlphaFold 3 model parameters you received from DeepMind after applying for it. If you place the file af3.bin.zst in the directory $HOME/af3-models the example scripts will work out of the box.

Please note: You can also store the parameters in your project directory or a workspace for slightly better performance. The template uses $HOME/af3-models because this environment variable works for everyone.

Output

You can set the output directory via the --output_dir=$ALPHAFOLD_RESULTS_DIR option. The template jobscript creates a workspace called alphafold as output directory.

For every input job, AlphaFold 3 writes all its outputs in a directory called by the sanitized version of the job name. E.g. for job name "My first fold (TEST)", AlphaFold 3 will write its outputs in a directory called My_first_fold_TEST (the case is respected). If such directory already exists, AlphaFold 3 will append a timestamp to the directory name to avoid overwriting existing data unless --force_output_dir is passed.

Please see the official AlphaFold 3 documentation for more information regarding output directory structure.

CPU-only: Multiple Sequence Alignment

In the beginning, AlphaFold 3 computes three multiple sequence alignments (MSA). These MSAs are computed on the CPU sequentially and the number of threads are hard-coded:

  • jackhmmer on UniRef90 using 8 threads
  • jackhmmer on MGnify using 8 threads
  • HHblits on BFD + Uniclust30 using 4 threads

We provide a template jobscript at: /opt/bwhpc/common/bio/alphafold/3.0.1/bwhpc-examples/binac2-af3-alignment.slurm

You can change --time and --mem, although these are sensible defaults.

#!/bin/bash
#SBATCH --partition=compute
#SBATCH --nodes=1

# Alphafold creates alignments sequentially, using at max 8 cores.
#SBATCH --ntasks-per-node=8

#SBATCH --time=10:00:00
#SBATCH --mem=100gb

The MSAs are stored in the directory specified by --output_dir.

GPU-part: Model Inference

Aftere computing the MSAs, AlphaFold performs model inference on the GPU. Only one GPU is used. This use case has this optimal resource profile:

#!/bin/bash
#SBATCH --partition=gpu
#SBATCH --nodes=1
#SBATCH --cpus-per-task=8
#SBATCH --gres=gpu:a100:1
#SBATCH --time=06:00:00
#SBATCH --mem=100gb

The template jobscript only differs to CPU-only part in the option --norun_data_pipeline. This means the MSAs aren't recomputed, but are taken from the previous CPU-only job.

Chaining CPU- and GPU-Jobs

To run the data pipeline first and then start the inference job as soon as the first one is finished, you can chain them like this:

JOBID=$(sbatch --parsable binac2-af3-alignment.slurm)
sbatch --dependency=afterok:$JOBID binac2-af3-inference.slurm