[DOCUMENTATION] Consequences of setting "cells_auto_optimisation = false"

[ennoschaefer]

What is the documentation missing?
The missing information relates to MD simulations on GPU. In the documentation of oxDNA (as of 09.09.2024), the setting "cells_auto_optimisation = false" is proposed to improve performance. It is noted that to support this option, one needs to have enough memory. I am working on a cluster with access to A40 and A100 GPUs - however whenever I set cells_auto_optimisation to false, I instantaneously receive an error message: "CUDA error at /path/to/oxDNA/src/CUDA/Lists/CUDASimpleVerletList.cu 147: out of memory". I was wondering if I need to set any additional settings to manually control cells optimisation or if there is a way to check how much CUDA memory I would need to run my simulations with this option set.

Additional information
oxDNA version: v3.6.1

[lorenzo-rovigatti]

You are right, the docs are bit lacking on this part. Here is the text I have added to the docs (and will show up in the online documentation as soon as we release a new version):

There are some heuristics that attempt to limit the memory consumption of CUDA simulations. First of all, the given combination of parameters is used to evaluate the minimum size of the cells required to build neighbouring lists, $r_m$. In turn, $r_m$ is used to compute the number of cells along each coordinate $i$ (where $i = x, y, z$) as

$$ N_i = \max(\lfloor L_i / r_m \rfloor, 3), $$

where $L_i$ is the length of the box edge along the $i$-th direction. This value of $N_i$ is the number of cells used for the simulation if cells_auto_optimisation is set to false. However, if it set to true, which is the default, then the code checks whether $N_i > \lceil f L_i \rceil$, and if it is then sets

$$ N_i = f L_i, $$

where

$$ f = \left( \frac{2 N}{L_x L_y L_z} \right)^{1/3}. $$

The maximum number of particles that are in each given cell, $M$, is another important parameter that can be, to some extent, tuned to avoid crashes. It is defined at the beginning of the simulation, and also each time the total number of cells changes while the simulation is running, as

$$ M = f_\rho M_\text{max}, $$

where $f_\rho$ is a factor that can be set with the max_density_multiplier option and defaults to 3, while $M_\text{max}$ is the number of particles found in the cell containing the largest amount of particles in the current configuration.

On newer versions of oxDNA (> 3.6.1), setting debug = true will report in the log file (or on screen if log_file is not set) the amount of memory that is requested by each allocation on the GPU.