Process Win32 input events through Natlink when possible

[drmfinlay]

Re: #11.

It occurs to me that most Dragonfly Key, Text and Mouse actions could be processed using the special natlink playEvents() function. Because DNS itself would "play" the events, this would allow indirect interaction with applications running in elevated mode.

This feature could either be optional or always used if the natlink module is available and DNS is running.

Assuming the documentation for playEvents() is complete, the following events can not be processed:

• Scroll wheel events
• Mouse 4 and 5 buttons events
• Unicode keystroke events

Dragonfly can will have to fallback on its own SendInput implementation for these. I do have hope, however, that the function may be used to type these events anyway; the accepted tuples are simply window message pairs (wParam and lParam) defined by Windows.

As is the case with DNS itself, this won't allow interaction with the UAC box.

[drmfinlay]

I have got the Key and Text actions to work as above. (Haven't committed the changes yet.) I am able to use these actions to interact with applications running in elevated mode. However, due to certain limitations of the interface, I think the feature should be disabled by default. It will be enableable like the Unicode keyboard.

Incidentally, this interface may be used in conjunction with WSR, or sphinx, as long as Natlink is available and DNS is running; it is not tied to the Natlink engine. Since 32-bit Python is required for Natlink, it can't be used directly with Kaldi.

[drmfinlay]

This interface doesn't work well with modifier keys and is therefore unsuitable as an alternative to SendInput. I'll mention the Natlink playEvents() and playString() functions in the FAQ answer for controlling administrative applications and leave it at that.

[drmfinlay]

Since the work on this is complete, I might just add it as a disabled feature with caveats mentioned in the docs. Better to include my implementation than to refer to the hard-to-use Natlink function. Some may find this feature useful despite the caveats.

[drmfinlay]

This feature is now ready on the master branch.

@quintijn, you may find this interesting.

[quintijn]

Thanks Dane, good to know. I agree that your Keys and Text (and also Mark Lillibridges sendkeys functions, which are now in the dtactions repository) are more easy, and can also be used when Dragon is not running, so also for other applications (as I happen to have).

[drmfinlay]

No worries, Quintijn. I can't take credit for the Key and Text actions though. Christo Butcher wrote those.

BTW, this feature uses a sort of replacement sendinput module: natlinkinput.py. It is not dependent on Dragonfly. I have no objection to it being used in dtactions, if that is desirable.