Implementation of delays of observations

[melecmat]

Hi, I would like to experiment with adding delays to the observations to bridge the sim2real gap. I found out they are not supported yet.

Are there some plans on adding this support? I saw a TODO in code regarding this, so I was wondering whether anyone has already tried something in this direction.

[alvarobelmontebaeza]

Not sure if this could work, but maybe you can set up an Event term that updates some variable which you can then query in your observation term.

Not the cleanest way of course, but could serve as a workaround...

[melecmat]

Interesting idea. However events seem to be tied to the environment step. If the delay needs higher resolution than that it would not work.

Right now I am thinking of extending the interface of the observation to enable to decide at which physics step the observation would be taken. This way the precision is determined by the size of the physics step. Furthermore one can buffer a few recent observations should the delay be larger than the environment step and pop out of a queue.

[KyleM73]

While not exactly what you're looking for, there is a DelayedPDActuator that has a configurable and randomizable amount of delay- this should be fairly easy to turn into an observation wrapper class that can take ObsTerms as inputs and apply fixed/variable delay. You just need to keep track of a pointer into some observation buffer that you update with each step.

https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab/omni/isaac/lab/actuators/actuator_pd.py

[dhajnes]

@melecmat Yes, I agree, events might not be the best idea. The delayed actuator implements this based on the simulation physics step, so it might be what you are looking for. I also got here because of the Sim-to-Real need for delayed observation. Did you perhaps manage to create such a wrapper? If yes, is it possible you share it somehow, for example fork?