Flux launch commands supporting MPIR

[RichardBuntLinaro]

Hi All,

Is there a complete list of flux commands that can be used to start user applications and which of these support MPIR? I see that both flux run and flux job attach can both be used to launch jobs and expose the MPIR interface.

Many thanks,

Rich.

[grondo]

All Flux jobs support MPIR in a way. Unlike other RMs, Flux jobs do not require a "frontend process", which is typically the executable in which the MPIR interface is exposed. Instead, jobs in Flux run as a parallel program and the flux job attach utility can be used to interactively attach to the job and shepherd I/O, signals, etc. However, a job can run to completion without ever using flux job attach (e.g. if flux submit is used).

Since most submission cli tools are Python, the majority of the currently exposed MPIR interface is in the flux job attach utility. This utility can be used to attach to any running job and fetch the MPIR debug info (even if another flux job attach is currently running). I believe flux job attach will check for MPIR_being_debugged=1, which is also set with the --debug or --debug-emulate option, and will arrange to fetch MPIR information from he job shells if so.

There is one piece of MPIR support in the Python flux run submission tool. If MPIR_being_debugged is set in this process, then it will automatically set the stop-tasks-in-exec job shell option so that all job tasks are stopped when they call exec(2). flux run will then invoke flux job attach as usual. I think the debugger has to follow this exec and also set MPIR_being_debugged in flux job attach so that MPIR information is populated.