...

Computing resources on Athena are assigned based on PLGrid computing grants. To perform computations on Athena you need to obtain a computing grant through the PLGrid Portal (https://portal.plgrid.pl/) and apply for Athena access through the PLGrid portal (https://aplikacje.plgrid.pl/service/dostep-do-klastra-athena-w-osrodku-cyfronet/).

The work on Athena's storage is still underway. In the current setup, the long-term storage is sourced from Ares. Thus you need, access to Ares, to use the group directory storage on Athena. Performing high IO computations on group space is strictly forbidden!

If your grant is active, and you have applied for the service access, the request should be accepted in about half an hour. Please report any issues through the helpdesk.

...

Job submission

...

Accounts and computing grants

Athena uses a new naming scheme of naming Slurm accounts GPU computing grants. GPU computing grants using A100 GPU resources use the for GPU computing accounts, which are supplied by the -A parameter in sbatch command. Currently, accounts are named in the following manner:

suffix. Please mind that sbatch -A grantname won't work on its own. You need to add the -gpu-a100 suffix! Available computing grants, with respective account names (allocations), can be viewed by using the hpc-grants command.

Resource allocated on Athena doesn't use normalization, which was used on Prometheus. 1 hour of GPU time equals 1 hour spent using a GPU.on a GPU with a proportional amount of CPUs and memory (consult the table above). The billing system accounts for jobs that use more CPUs or memory than the proportional amount. If the job uses more CPU or memory for each allocated GPU than the proportional amount, it will be billed as it would have used more GPUs. The billed amount can be calculated by dividing the used number of CPUs or memory by the proportional amount per GPU and rounding the result to the closest and larger integer. Jobs on GPU partitions are always billed in GPU hours.

The cost can be expressed as a simple algorithm:

cost_gpu    = job_gpus_used * job_duration
cost_cpu    = ceil(job_cpus_used/cpus_per_gpu) * job_duration
cost_memory = ceil(job_memory_used/memory_per_gpu) * job_duration
final_cost  = max(cost_gpu, cost_cpu, cost_memory)

Storage

Available storage spaces are described in the following table:

...

• hpc-grants - shows available grants, resource allocations
• hpc-fs - shows available storage
• hpc-jobs - shows currently pending/running jobs
• hpc-jobs-history - shows information about past jobs

Software

Compilation should be done on a worker node inside of a computing job. It is most convenient to use an interactive job to do all compilation and application setup. Login node doesn't include development libraries!

Sample job scripts

...