Augustus takes 420G ram +59h

[Sebastien-Raguideau]

Hello I have a new issue for you :)
Some of the genomes I'm looking at are taking forever to be profiled. When looking at it in detail augustus is being slow (50h+) as well as hogging ram (worst so far is 420G ram) on some genes. As I don't monitor all the time I don't know if some of theses managed to complete before.
From what Is running in my server I have 6 ELP3 and 3 RPO21 running for more than 40h.

I can share with you an example where it does not end so far.
Here is a command running forever:
augustus --optCfgFile=/mnt/gpfs/seb/Applications/UFCG_v1.0.1/UFCG/config/ppx.cfg --predictionStart=165215 --predictionEnd=185215 --proteinprofile=/mnt/gpfs/seb/Applications/UFCG_v1.0.1/UFCG/config/model/pro/ELP3.hmm /home/sebr/seb/Database/Fungus_DB/UFCG/tmp/Conco1/tmp/Conco1/Conco1_scaffold_37.fna

with genome and temp dir:
Conco1_issue.tar.gz

[endixk]

Hello again,

This is a weird behavior of augustus that I also witnessed before. I eventually failed to find the main cause or to determine the situation in which such behavior is observed.

As this issue will persist, from the next release, a timeout will be wrapped around the augustus predictions to prevent the pipeline from consuming unnecessary time or resources.

To root out the primary cause, the issue could be redirected to the Augustus repository, where the developers of augustus can actually deal with this unexpected behavior.

[endixk]

Now the same problem won't happen with the newest version, but will handle the 5-minute timeouts with messages something like this:

WARN  |:  Command timed out: augustus --optCfgFile=... --predictionStart=14960 --predictionEnd=34960
--proteinprofile=... /tmp/Conco1/Conco1_scaffold_199.fna > /tmp/Conco1/Conco1_scaffold_199_p24960_RPO21.gff