Proposal: Change Natlink to work out of process by default #198

drmfinlay · 2024-07-05T08:02:31Z

As discussed over e-mail and in the developer chat, changing Natlink to work exclusively in normal Python processes would resolve a number of open issues and simplify the software (natlink and natlinkcore) significantly.

quintijn · 2024-07-06T13:26:21Z

In principle, I support this idea, as you know. I hope, that this will make the install and maintenance procedure more simple. Starting a python process after starting Dragon is a minor extra thing when it makes other things more simple and robust.

OTOH, now the natlink subsystem is started with Dragon, and therefore integrated in the Dragon speech input cycle at the deepest possible place, and a supported way by Dragon.

I hope this close connection is not made weaker with your proposed solution. I think this needs to be discussed further!

drmfinlay · 2024-07-07T01:34:50Z

Thanks, Quintijn. I am confident that it will make installation, usage and maintenance simpler and more robust. I appreciate your caution here, but I don't believe the connection will be made weaker by my proposed solution. I've used Natlink and Dragon this way extensively. The connection is the same one used by the test code — Natlink's and Dragonfly's. If it does break, I take responsibility for fixing or reverting my changes.

…

On Sat, 06 Jul 2024 06:26:43 -0700 Quintijn Hoogenboom ***@***.***> wrote: In principle, I support this idea, as you know. I hope, that this will make the install and maintenance procedure more simple. Starting a python process after starting Dragon is a minor extra thing when it makes other things more simple and robust. OTOH, now the natlink subsystem is started with Dragon, and therefore integrated in the Dragon speech input cycle at the deepest possible place, and a supported way by Dragon. I hope this close connection is not made weaker with your proposed solution. I think this needs to be discussed further! -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

LexiconCode · 2024-07-07T14:03:23Z

This will solve a number issues that would be difficult or impossible fix.

PYD registration is problematic. When Enabling natlink com object in Dragons ini files it is a global registration for all users. This causes an error for other users that may not be utilizing natlink with Dragon.
Admin privileges are required for registration pyd and editing dragen ini files.
If there's something wrong with natlink dragon has to be restarted. This is a sore spot for many users. Some are left helpless depending on their accessibility. Out of process means Dragon can be running in the background and the subsystem can be restarted with natlink. Then users can still control their computer.

At the end of the day, this makes Natlink more portable and robust for the end user.

There's one downside with how things currently would be structured out of process. Natlink does not start with Dragon automatically. This is a desired feature by user's.

On the process of making a GUI that can display command history, rules and so forth there might be another opportunity. My thought is this GUI could be running in the system tray when OS starts. When Dragon is launched, it could also be watching for that process and automatically start natlink.

The tricky bit is making sure it starts late enough. That might be worthy of some discussion..

drmfinlay · 2024-07-07T23:42:56Z

We agree on most of the above, I think. An autostart feature goes beyond what I had in mind here. As Quintijn said, it is a minor extra thing for the user to do. Maybe it can be considered further down the road? I am still in the planning phase on this. A polished GUI like that might be good as a setuptools "extra" for 'natlinkcore', to be installed alongside it.

I can help you with the last part at least. Use the `natlink.isNatSpeakRunning()' function.

quintijn · 2024-07-08T14:02:34Z

Great, Dane, I did not think about the isNatspeakRunning function.

Eventually, you only need to start Natlink: if Natspeak is not running yet it is started from the script, with configurable user (in natlink.ini), wait for the load procedure of Natstpeak. After that, you can go ahead.

LexiconCode · 2024-07-09T18:46:27Z

Here's my attempt to change how PYD gets loaded.

Old way, registered COM object

natlink/pythonsrc/src/natlink/__init__.py

Lines 23 to 73 in a48702f

    
           def outputDebugString(to_show): 
        
               """  
        
               :param to_show: to_show 
        
               :return: the value of W32OutputDebugString 
        
               Sends a string representation of to_show to W32OutputDebugString  
        
               """ 
        
               return W32OutputDebugString(f"{to_show}") 
        
           clsid="{dd990001-bb89-11d2-b031-0060088dc929}"          #natlinks well known clsid 
        
           #these keys  hold the pyd name for natlink, set up through the installer run regsvr32 
        
           subkey1=fr"WOW6432Node\CLSID\{clsid}\InprocServer32" 
        
           subkey2=fr"CLSID\{clsid}\InprocServer32"  #only likely if not 64 bit windows 
        
           subkeys=[subkey1,subkey2] 
        
           default_pyd="_natlink_core.pyd"    #just a sensible default if one isn't registered. 
        
           path_to_pyd="" 
        
           #find the PYD actually registered, and load that one. 
        
           found_registered_pyd=False 
        
           for subkey in subkeys: 
        
               try: 
        
                   reg = winreg.ConnectRegistry(None,winreg.HKEY_CLASSES_ROOT) 
        
                   sk = winreg.OpenKey(reg,subkey) 
        
                   path_to_pyd = winreg.QueryValue(sk,None) 
        
                   found_registered_pyd = True 
        
                   break 
        
               except: 
        
                   pass 
        
           try: 
        
               pyd_to_load=path_to_pyd if found_registered_pyd else default_pyd 
        
               #if something goes wrong we will want these messages. 
        
               outputDebugString(f"Loading {pyd_to_load} from {__file__}") 
        
               loader=importlib.machinery.ExtensionFileLoader("_natlink_core",pyd_to_load) 
        
               spec = importlib.util.spec_from_loader("_natlink_core", loader) 
        
               _natlink_core=importlib.util.module_from_spec(spec) 
        
               from _natlink_core import * 
        
               import locale 
        
               from _natlink_core import execScript as _execScript 
        
               from _natlink_core import playString as _playString 
        
               from _natlink_core import playEvents as _playEvents 
        
               from _natlink_core import recognitionMimic as _recognitionMimic 
        
           except Exception: 
        
               tb_lines = traceback.format_exc() 
        
               outputDebugString(f"Python traceback \n{tb_lines}\n in {__file__}")

New way, finds PYD via path stored in environment.json instead of looking for registered COM object

natlink/pythonsrc/src/natlink/__init__.py

Lines 32 to 65 in 7bc3c13

    
           def get_config(): 
        
               """get the configuration information from environment.json 
        
               Returns: 
        
                   (dict): the configuration data 
        
               """ 
        
               try: 
        
                   # get the path to the environment.json file user directory .natlink 
        
                   path = os.path.join(os.path.expanduser("~"), ".natlink", "environment.json") 
        
                   with open(path) as file: 
        
                       data = json.load(file) 
        
                   return data 
        
               except Exception as e: 
        
                   outputDebugString(f"get_config: {e}") 
        
           try: 
        
               data = get_config() 
        
               path_to_pyd = data.get("natlink_pyd") 
        
               #if something goes wrong we will want these messages. 
        
               outputDebugString(f"Loading {path_to_pyd} from {__file__}") 
        
               loader = importlib.machinery.ExtensionFileLoader("_natlink_core", path_to_pyd) 
        
               spec = importlib.util.spec_from_loader("_natlink_core", loader) 
        
               _natlink_core = importlib.util.module_from_spec(spec) 
        
               from _natlink_core import * 
        
               import locale 
        
               from _natlink_core import execScript as _execScript 
        
               from _natlink_core import playString as _playString 
        
               from _natlink_core import playEvents as _playEvents 
        
               from _natlink_core import recognitionMimic as _recognitionMimic 
        
           except Exception: 
        
               tb_lines = traceback.format_exc() 
        
               outputDebugString(f"Python traceback \n{tb_lines}\n in {__file__}")

It seems the PYD is loaded successful but something still not right as evidenced by the following tested through dragonfly.

[24644] Loading C:\Users\Main\Desktop\natlink_vert_out_of_process\environment\DNS\venv\Lib\site-packages\natlink\_natlink_core.pyd from C:\Users\Main\Desktop\natlink_vert_out_of_process\environment\DNS\venv\lib\site-packages\natlink\__init__.py
[24644] RefCountedObject Create: ProfileInfoStatus addr 0x03F8AE48

========= Loading Dragonfly Example Rule ==========
INFO:engine:Initialized 'natlink' SR engine: NatlinkEngine().
DEBUG:command:Recognizing with engine 'natlink'
INFO:module:CommandModule('dragonfly_example_rule.py'): Loading module: 'C:\Users\Main\Desktop\natlink_vert_out\dragonfly_example_rule.py'
DEBUG:grammar.load:Grammar sample: adding rule MainRule.
DEBUG:grammar.load:Grammar sample: loading into engine NatlinkEngine().
DEBUG:grammar.load:Grammar sample: adding rule _IntegerRef_07.
DEBUG:engine:Engine NatlinkEngine(): loading grammar sample.
DEBUG:engine.compiler:NatlinkCompiler(): Compiling grammar sample.
DEBUG:engine.compiler:NatlinkCompiler(): Compiling rule MainRule.
DEBUG:engine.compiler:NatlinkCompiler(): Compiling rule _IntegerRef_07.
DEBUG:grammar.load:Grammar sample: activating rule MainRule.
DEBUG:engine:Activating rule MainRule in grammar sample.
DEBUG:grammar.load:Grammar sample: activating rule _IntegerRef_07.
DEBUG:grammar.load:Grammar _recobs_grammar: adding rule _anonrule_000_Rule.
DEBUG:grammar.load:Grammar _recobs_grammar: loading into engine NatlinkEngine().
DEBUG:engine:Engine NatlinkEngine(): loading grammar _recobs_grammar.
DEBUG:engine.compiler:NatlinkCompiler(): Compiling grammar _recobs_grammar.
DEBUG:engine.compiler:NatlinkCompiler(): Compiling rule _anonrule_000_Rule.
DEBUG:grammar.load:Grammar _recobs_grammar: activating rule _anonrule_000_Rule.
DEBUG:engine:Activating rule _anonrule_000_Rule in grammar _recobs_grammar.
Speech start detected.
DEBUG:grammar.begin:Grammar sample: detected beginning of utterance.
DEBUG:grammar.begin:Grammar sample: executable 'C:\Program Files\WindowsApps\Microsoft.WindowsNotepad_11.2405.13.0_x64__8wekyb3d8bbwe\Notepad\Notepad.exe', title 'Untitled - Notepad'.
DEBUG:grammar.begin:Grammar sample:     active rules: ['MainRule', '_IntegerRef_07']

See above after beginning of utterance grammar decode never happens above despite dragonfly showing an active rule!

Below is with the Dragonfly text engine

========= Loading Dragonfly Example Rule ==========
INFO:engine:Initialized 'text' SR engine: TextInputEngine().
DEBUG:command:Recognizing with engine 'text'
INFO:module:CommandModule('dragonfly_example_rule.py'): Loading module: 'C:\Users\Main\Desktop\natlink_vert_out\dragonfly_example_rule.py'
DEBUG:grammar.load:Grammar sample: adding rule MainRule.
DEBUG:grammar.load:Grammar sample: loading into engine TextInputEngine().
DEBUG:grammar.load:Grammar sample: adding rule _IntegerRef_07.
DEBUG:engine:Engine TextInputEngine(): loading grammar sample.
DEBUG:grammar.load:Grammar sample: activating rule MainRule.
DEBUG:grammar.load:Grammar sample: activating rule _IntegerRef_07.
hotel info
Speech start detected.
DEBUG:grammar.begin:Grammar sample: detected beginning of utterance.
DEBUG:grammar.begin:Grammar sample: executable 'C:\Program Files\WindowsApps\Microsoft.WindowsTerminal_1.20.11381.0_x64__8wekyb3d8bbwe\WindowsTerminal.exe', title 'C:\WINDOWS\system32\cmd.exe'.
DEBUG:grammar.begin:Grammar sample:     active rules: ['MainRule', '_IntegerRef_07'].
DEBUG:grammar.decode:   attempt: MainRule(MainRule)
DEBUG:grammar.decode:    -- Decoding State: ' >> hotel info'
DEBUG:grammar.decode:      attempt: Alternative(...)
DEBUG:grammar.decode:         attempt: Compound('hotel info')
DEBUG:grammar.decode:            attempt: Literal(['hotel', 'info'])
DEBUG:grammar.decode:            success: Literal(['hotel', 'info'])
DEBUG:grammar.decode:             -- Decoding State: 'hotel info >> '
DEBUG:grammar.decode:         success: Compound('hotel info')
DEBUG:grammar.decode:      success: Alternative(...)
DEBUG:grammar.decode:   success: MainRule(MainRule)
Recognized: hotel info
DEBUG:action.exec:Executing action: 'These types of hospitality services are not cheap.' ({'_grammar': Grammar(sample), '_rule': MainRule(MainRule), '_node': Node: Alternative(...), ['hotel', 'info'], 'n': 1, 'text': ''})
These types of hospitality services are not cheap.

It would be nice to get a simpler test just with natlink without dragonfly. I've sent through chat (It's too big to upload here) natlink_vert_out.zip which contains a virtual Python setup with the modified packages for testing purposes. make sure to unregister natlink before running natlink out of process.

One thing I'm not sure is how the pathway of natlink c++ is initialized out of process. For example relevancy of appsupp.h
https://github.com/dictation-toolbox/natlink/blob/virtual_python/NatlinkSource/COM/appsupp.h

dougransom · 2024-07-10T13:53:08Z

alternative ways to get dragon out of process include DLL Surrigates https://learn.microsoft.com/en-us/windows/win32/com/using-the-system-supplied-surrogate.

You can use the Running Object Table to host a singleton natlink object (or a bunch of COM objects as you desire) that can accessed by name.

You can also host natlink in an NT Service if you really wanted to, so it is always available for natlink to connect to.

I have no idea if any of the above are applicable or helpful to what you are working towards.

LexiconCode · 2024-07-10T18:11:37Z

Here's a quick easy way to test that have been successful on my system. The issue of recognizing commands during the Decode time at least in dragonfly is an issue. I have yet to test directly with natlink. Thank you @quintijn for simplifying this process!

Download natlink_vert_out_of_process.zip link is valid for 30 days

Uninstall existing Natlink
extract natlink_vert_out_of_process.zip
Run setup.bat - Sets up everything
Running tests:

Run test_dragonfly - natlink.bat to test dragonfly with natlink or test_dragonfly - text.bat with dragonfly's Text engine without natlink.
or
Execute python directly in python_environment.bat after typing python See Test Examples below:

Warning

There are issues with leaving Natlink processes in a dirty state! On error probably not being able to execute natlink.natDisconnect()?
- Workaround: Restart your computer but I'm looking for a better alternative like closing out certain processes. example: Python.exe/COM surrogate.exe
The dirty state looks like the following in two ways:
1. Dragon is running Test Example commands can freeze or or cause the GUI not to not respond in Dragon.
2. Run Test Example when Dragon is completely close yet these commands in Test Example still think Dragon is running!
Do not move the root folder as Python virtual environments rely on absolute paths unfortunately.
- Workaround: Re-create the Python environment by deleting environment folder. Rerun setup.bat

Reverting to Natlink installer

Make sure all processes are closed, that Natlink is not in a dirty state and PYD is not in use.
run Natlink installer
Done

Technically I'm not sure if it's even required to uninstall/deregister Natlink. However I have run into issues where the virtual Python tries to pick up on natlink site packages in C:\Program Files (x86)\natlink. For testing purposes use a clean system without Natlink. No need to uninstall system Python.

Test Example

(venv) C:\Users\Main\Desktop\natlink_vert_out_of_process>python
Python 3.10.14 (main, Apr 15 2024, 17:42:09) [MSC v.1929 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import natlink
>>> import natlinkcore
>>> dir(natlink)
['BadGrammar', 'BadWindow', 'DataMissing', 'DictObj', 'GramObj', 'InvalidWord', 'MimicFailed', 'NatError', 'NatlinkConnector', 'OutOfRange', 'ResObj', 'SyntaxError', 'UnknownName', 'UserExists', 'ValueError', 'W32OutputDebugString', 'WrongState', 'WrongType', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_execScript', '_natlink_core', '_original_natconnect', '_playEvents', '_playString', '_recognitionMimic', '_test_playEvents', 'addWord', 'contextlib', 'createUser', 'ctypes', 'data', 'deleteWord', 'displayText', 'execScript', 'ext_keys', 'finishTraining', 'getAllUsers', 'getCallbackDepth', 'getClipboard', 'getCurrentModule', 'getCurrentUser', 'getCursorPos', 'getDNSVersion', 'getMicState', 'getScreenSize', 'getTrainingMode', 'getUserTraining', 'getWordInfo', 'getWordProns', 'get_config', 'importlib', 'inputFromFile', 'isNatSpeakRunning', 'json', 'lmap', 'loader', 'locale', 'natConnect', 'natDisconnect', 'openUser', 'os', 'outputDebugString', 'path_to_pyd', 'playEvents', 'playEvents16', 'playString', 'recognitionMimic', 'saveUser', 'setBeginCallback', 'setChangeCallback', 'setMicState', 'setTimerCallback', 'setTrayIcon', 'setWordInfo', 'spec', 'startTraining', 'toWindowsEncoding', 'traceback', 'waitForSpeech', 'win32api', 'win32gui', 'winreg', 'wrappedNatConnect']
>>> natlink.natConnect(1)
<contextlib._GeneratorContextManager object at 0x00B070A0>
>>> natlink.getMicState()
'on'
>>> natlink.setMicState('off')
>>> natlink.setMicState('on')

This will execute the loader manually in natlinkcore
>>> dir(natlinkcore)
['Path', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', 'getThisDir', 'logname']

>>> from natlinkcore import loader
>>> loader.run()
>>> natlink.natDisconnect()

I will be codifying these into tests that can be executed by bat files running python scripts. If the culprit is figured out by closing processes I can create dedicated bat to forcibly clean the state. That way we can test figure out if anything specifically contributing to this bad state.

I will make a new post with the new zip file when these tests are ready! Currently utilize what's above.

LexiconCode · 2024-07-14T04:55:19Z

I've tried registering pyd without my changes natlink init and leaving the Dragon ini vanilla. Dragonfly called through the cli with a demo grammar (Although I don't see decode in debug mode) are recognized, but not executed. The same, calling through natlink!

drmfinlay · 2024-07-14T08:17:13Z

I have not had a chance to test this yet, but these results do not sound encouraging. I suspect it is a problem with Dragon 16. I'll look into it and see what, if anything, can be done.

…

On Sat, 13 Jul 2024 21:55:40 -0700 LexiconCode ***@***.***> wrote: I've tried registering pyd without my changes natlink init and leaving the Dragon ini vanilla. Dragonfly called through the cli with a demo grammar dommands ( Although I don't see the decode debug mode) are recognized, but not executed. The same, calling through nalink! -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

LexiconCode · 2024-07-14T08:55:52Z

natlink_vert_out_of_process_v2

Added more bat files to run .py file for tests

drmfinlay · 2024-07-24T01:44:59Z

The issue mentioned above — the broken recognition callback — appears to be Windows-related. I can reproduce it on Windows 10 with Dragon 15. It does not occur on Windows 7 with the same Dragon version. The problem may be fixable. I have been able to determine that the callback is invoked, but causes a fault. I'll see if anything can be done about that. I don't know about the dirty state issue. It does sound like what occurs when `natDisconnect()' isn't called.

drmfinlay · 2024-07-24T03:21:20Z

Okay, good news. I managed to fix the broken callback. My proposal now looks viable on Windows 10. The same binary also works on Windows 7. Dragonfly's test suite now passes expectedly on both. I did have to do some debugging and I can confirm it is easier this way. Simply attach Visual Studio's native debugger to your Python process and ensure the symbol file (natlink.pdb) is loaded.

drmfinlay · 2024-07-27T02:32:39Z

@LexiconCode The dirty state issue that occurs if one closes Dragon before Natlink is solvable by waiting for the shutdown event and disconnecting cleanly. I have some code for this that works and will include it in my changes for review. I assume you guys would like Natlink to remain open if Dragon is closed first and then reconnect when it can, rerunning the loader code. This seems intuitive to me.

LexiconCode · 2024-07-27T04:33:06Z

Yes, that seems intuitive to me. Is it a simple, wait for dragen.exe to load then execute loader?

drmfinlay · 2024-07-27T07:08:30Z

Okay great. Yes, basically. There is a Windows event for DNS initialization that the program can wait for.

dougransom · 2024-08-06T15:38:13Z

I think it is ok to make the user launch natlink, or to launch it at startup, or when dragon restarts (which is not that often).

drmfinlay · 2024-08-30T08:06:01Z

I was thinking about this yesterday. I think I can retain Natlink's in-process mode and have the software merely work out-of-process by default. This would be a more modest proposal. The in-process mode should be enableable via a post-install script for a single Python installation.

quintijn · 2024-08-30T10:26:34Z

Any way the inprocess variant is kept as a possibility seems good to me. But I would like to have a stable natlink installer (inprocess) before we move on to installers that do out of process (by default).

drmfinlay · 2024-08-30T22:29:52Z

All right then. I do think 'pip install natlink natlinkcore' followed by a post-install script for inprocess would be simpler. But, I'll have a look into what can be done to stabilise the current installer. Maybe it could run those commands, or similar ones.

…

On Fri, 30 Aug 2024 03:26:55 -0700 Quintijn Hoogenboom ***@***.***> wrote: Any way the inprocess variant is kept as a possibility seems good to me. But I would like to have a stable natlink installer (inprocess) before we move on to installers that do out of process (by default). -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

drmfinlay · 2024-08-31T01:34:26Z

This is somewhat tangential, but I think we should explicitly rule out supporting virtual environments in-process. That way, all the current sys.path hacks could be done away with and Natlink could simply rely on CPython's Windows registry mechanism (PEP 514).

LexiconCode · 2024-08-31T01:42:53Z

There are no hacks needed for virtual environments in-process. All that is needed
https://github.com/LexiconCode/natlink/blob/9761412be13e551947e71b6f9a9a600e96d6ce59/NatlinkSource/COM/appsupp.cpp#L208

We should use PyConfig_InitIsolatedConfig for in-process and out-of-process. The are problems with packages pollution from 32bit/64bit python. PyConfig_InitIsolatedConfig keeps that from happening.

drmfinlay · 2024-09-01T00:14:45Z

I'm sorry, I just don't see the value of this. The function you refer to is not invoked by Natlink out of process. In that case it is invoked by the interpreter, python.exe. With my proposal implemented, one could use Natlink in a virtual environment out of process without needing administrative access rights or a signed installer. As for in-process Natlink, the simplest way of using it is with a dedicated 32-bit installation, installed at the user or system level. This essentially already functions as an isolated environment, as I think it is fair to say most programmer users will do normal Python work with a 64-bit installation.

drmfinlay · 2024-09-01T00:49:21Z

As to package pollution, the 64-bit Python programmer user who uses in-process Natlink could do their 64-bit work in a virtual environment. We could advise this in the docs.

LexiconCode · 2024-09-03T09:10:01Z

I misspoke about PyConfig_InitIsolatedConfig for out of process. The virtual environment does solve other issues with how pip install packages through the installer. I'm not sure, but there might be another way around that issue.

drmfinlay · 2024-09-04T23:57:28Z

I now understand from our IM chat why you want this: in order to work around a separate problem, described in issues #178, #207, etc. There's a term for this I learned only recently: the XY problem. It is where one tries to solve X in order to solve Y. I'll see if I can figure out what's causing Y here.

…

On Tue, 03 Sep 2024 02:10:23 -0700 LexiconCode ***@***.***> wrote: I misspoke about PyConfig_InitIsolatedConfig for out of process. The virtual environment does solve other issues with how pip install packages through the installer. I'm not sure, but there might be another way around that issue. -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

drmfinlay · 2024-09-25T12:45:12Z

This proposal itself seems like an XY problem thing, too, now that I think about it. The problems with the installer, DLL registration, etc., are, I think, caused by Python 3 design changes and are solvable. Pywin32 is only a few years older than Natlink and has managed to solve similar problems. Still, there are one or two possibly useful ideas to come out of this. I think it would be nice if Natlink's message window were to be controllable via a function similar to the setTrayIcon() function. What do you think, Quintijn?

…

On Tue, 03 Sep 2024 02:10:23 -0700 LexiconCode ***@***.***> wrote: I misspoke about PyConfig_InitIsolatedConfig for out of process. The virtual environment does solve other issues with how pip install packages through the installer. I'm not sure, but there might be another way around that issue. -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

LexiconCode · 2024-09-25T13:10:55Z

Possibly the installer, inconsistent DLL registration, and some of the Python 3 package management could be solved. However in process requires editing dragon ini, and DLL registration. Fundamentally Natlink cannot be used in the multiuser system:

On a shared system:

Dragon maybe used with or without Natlink
- Editing dragon ini for COM compatibility are global which forces every user to use Natlink, if the pyd is not registered a error occurs. (not solvable) when Dragon starts.
Different Python environments with Natlink
- DLL registration is global and can only point to one Python environment for the pyd (possibly solvable)

Other issues:

IT departments have denied admin privileges to install Natlink which is often decided on a case-to-case basis.

This is not a hypothetical environment. In my support of caster this is has kept Natlink from being used in university accessibility resource centers, corporate and home environments where multiple rely on Dragon with shared/multiuser environment. Out of process allows install without admin privileges. This greatly simplifies the user experience anywhere including the home environment. I haven't brought this up in the past because out of process had some outstanding issues which now are being addressed.

The out of process solution fixes easily all of these issues. Most of us seem to be on board with out a process as the default moving forward. What is a technical reasoning for not following through with out of process as the default method?

dougransom · 2024-09-25T22:52:36Z

Maybe this will help:

from https://learn.microsoft.com/en-us/windows/win32/sysinfo/hkey-classes-root-key

dougransom · 2024-09-25T23:01:52Z

'IT departments have denied admin privileges to install Natlink which is often decided on a case-to-case basis.'

I don't even think we should try to address that. If an IT department wants to support people who want or need to use dragon for their work, accessibility needs, etc, with natlink they can make that call. Let their stakeholders lobby them.

LexiconCode · 2024-09-26T00:17:32Z

'IT departments have denied admin privileges to install Natlink which is often decided on a case-to-case basis.'

I don't even think we should try to address that. If an IT department wants to support people who want or need to use dragon for their work, accessibility needs, etc, with natlink they can make that call. Let their stakeholders lobby them.

As someone that is worked with many many people on this issue specifically I strongly disagree with this stance. We should be lowering the barriers for people when we have the ability to do so.

LexiconCode · 2024-09-26T00:23:47Z

As a concession we could do both in and out of process. If the following features were implemented for out of to process:

Having natlink start with Dragon out of to process.
Releasing natlink as a pip package
Signing the .pyd / installer

The installer could do in process and other methods could handle out of process.

drmfinlay · 2024-09-26T01:47:02Z

Thank you for writing this up, Aaron! I had not given enough thought to the multiuser scenario. You are quite right, Natlink's in-process mode would unusable on a multiuser system at the moment. If Python 3, Natlink and any dependency packages were installed at the system level and if the Natlink DNS compatibility module did basically nothing unless enabled for the current user, perhaps through an HKCU registry key or the natlink.ini file, then it could work. Having the message window controllable as I touched on above would make it easier for the in-process mode to do basically nothing for users who haven't enabled it. This change would also let us explore the out of process solution more gradually, if natlink is also put up as a package on PyPI. I agree, mostly, with Aaron's points on IT departments and his above concessional feature list. Signing the installer (#87) would help with the IT department issue. Having out of process natlink start with Dragon could be done, but seems unnecessary. But again, we can look into that more gradually. One unmentioned technical reason for not following through with this proposal is the growing complexity of my project plan for it. The plan is nearing the edge of intellectual manageability for me, has taken months and is still not finished. Natlink is an old, complex piece of software and I don't think we have another expert really capable of reviewing my changes to it. I don't want to cause headaches down the line. The installer is easier to fix. The Python code already exists in the historical natlink configuration functions to modify the ini file and registry keys appropriately and to register the DLL. There are a few other things, but they're manageable.

…

On Wed, 25 Sep 2024 17:24:08 -0700 LexiconCode ***@***.***> wrote: Alternatively having both in and out of process. if the following features were implemented for out of to process: - Having natlink start with Dragon out of to process. - Releasing natlink as a pip package - signing .pyd / installer The installer could do in process and other methods could handle out of process. -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

LexiconCode · 2024-09-27T00:46:48Z

Natlink DNS compatibility module did basically nothing unless enabled for the current user, perhaps through an HKCU registry key or the natlink.ini file, then it could work.

This is where I'm a little confused, I think DNS throws an error on startup with just the compatibility module edited ini. Try the following:

uninstall natlink,
edit manually file as admin
started Dragon
error looking for compatibility module, this will happen for all users who don't have the registered com?

I may be wrong but that's completely out of our control, right? I feel like I've missed something that conversation here. sorry!

One unmentioned technical reason for not following through with this proposal is the growing complexity of my project plan for it. The plan is nearing the edge of intellectual manageability for me, has taken months and is still not finished. Natlink is an old, complex piece of software and I don't think we have another expert really capable of reviewing my changes to it. I don't want to cause headaches down the line.

I can understand that, thank you for your help! We all have our limits both in time and energy.

drmfinlay · 2024-09-27T10:58:09Z

You haven't missed anything, I hadn't thought of it! That error message box does seem to be completely out of our control. However, it is always indicative of a setup error. It seems to me that this raises a solid argument for only supporting system-level installation: 1) Natlink's userbase includes users of multi-user systems. 2) Natlink's in-process server DLL and its dependency DLLs, including Python3X.dll, must be installed in globally accessible locations in order to prevent this error from occurring for other Dragon users on a multi-user system. 3) Software installed for all users covers the single-user system use case. 4) Therefore, Natlink and the Python 3 installation it uses should always be installed at the system-level for all users. Again, Python 3 appears to be the real problem here. Its per-user installation default is incompatible with Natlink's in-process mode and should not be supported in this way by the project. Thank you for your understanding and kind words! I think, given all of the above, a first-class out-of-process mode would still be useful, perhaps as a mode targeted more toward the programmer section of Natlink's userbase and downstream projects. I'll try to put together a list of the minimal changes necessary to allow gradual, collaborative exploration and development of a new out-of-process user mode. Fixing the installer should be our priority for the moment, however.

…

On Thu, 26 Sep 2024 17:47:08 -0700 LexiconCode ***@***.***> wrote: > Natlink DNS compatibility module did basically nothing unless enabled for the current user, perhaps through an HKCU registry key or the natlink.ini file, then it could work. This is where I'm a little confused, I think DNS throws an error on startup with just the compatibility module edited ini. Try the following: 1. uninstall natlink, 2. edit manually file as admin 3. started Dragon 4. error looking for compatibility module I may be wrong but that's completely out of our control, right? > One unmentioned technical reason for not following through with this proposal is the growing complexity of my project plan for it. The plan is nearing the edge of intellectual manageability for me, has taken months and is still not finished. Natlink is an old, complex piece of software and I don't think we have another expert really capable of reviewing my changes to it. I don't want to cause headaches down the line. I can understand that, thank you for your help! We all have our limits both in time and energy. -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

drmfinlay · 2024-09-27T11:43:05Z

I would like to add that my proposal from last year to use CPython's limited API (#171) would help us with both modes. It would take the pickiness about minor Python 3 versions away from the library and give it to the user as a freedom without us having to distribute more than one compiled installer/binary. A minimum minor version requirement would still be enforced, of course. Obviously one is needed.

…

On Thu, 26 Sep 2024 17:47:08 -0700 LexiconCode ***@***.***> wrote: > Natlink DNS compatibility module did basically nothing unless enabled for the current user, perhaps through an HKCU registry key or the natlink.ini file, then it could work. This is where I'm a little confused, I think DNS throws an error on startup with just the compatibility module edited ini. Try the following: 1. uninstall natlink, 2. edit manually file as admin 3. started Dragon 4. error looking for compatibility module I may be wrong but that's completely out of our control, right? > One unmentioned technical reason for not following through with this proposal is the growing complexity of my project plan for it. The plan is nearing the edge of intellectual manageability for me, has taken months and is still not finished. Natlink is an old, complex piece of software and I don't think we have another expert really capable of reviewing my changes to it. I don't want to cause headaches down the line. I can understand that, thank you for your help! We all have our limits both in time and energy. -- Reply to this email directly or view it on GitHub: #198 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

LexiconCode · 2024-09-27T12:40:33Z

All right sounds like we have pathway forward!

drmfinlay self-assigned this Jul 5, 2024

drmfinlay mentioned this issue Jul 5, 2024

File Reload of natlink should restart the whole python "subsystem" #28

Open

drmfinlay mentioned this issue Jul 31, 2024

make playEvents obsolete for Dragon16 #200

Open

drmfinlay mentioned this issue Aug 22, 2024

Grammar lists do not work correctly with Dragon 15 in some cases #82

Open

drmfinlay changed the title ~~Proposal: Change Natlink to work exclusively out of process~~ Proposal: Change Natlink to work out of process by default Aug 30, 2024

drmfinlay mentioned this issue Oct 16, 2024

Add a new natlink function for controlling the message window #213

Open

Proposal: Change Natlink to work out of process by default #198

Proposal: Change Natlink to work out of process by default #198

Comments

drmfinlay commented Jul 5, 2024

quintijn commented Jul 6, 2024

drmfinlay commented Jul 7, 2024 via email

LexiconCode commented Jul 7, 2024 • edited Loading

drmfinlay commented Jul 7, 2024

quintijn commented Jul 8, 2024

LexiconCode commented Jul 9, 2024 • edited Loading

dougransom commented Jul 10, 2024

LexiconCode commented Jul 10, 2024 • edited Loading

LexiconCode commented Jul 14, 2024 • edited Loading

drmfinlay commented Jul 14, 2024 via email

LexiconCode commented Jul 14, 2024 • edited Loading

drmfinlay commented Jul 24, 2024 via email

drmfinlay commented Jul 24, 2024 via email

drmfinlay commented Jul 27, 2024 via email

LexiconCode commented Jul 27, 2024

drmfinlay commented Jul 27, 2024

dougransom commented Aug 6, 2024

drmfinlay commented Aug 30, 2024 via email

quintijn commented Aug 30, 2024

drmfinlay commented Aug 30, 2024 via email

drmfinlay commented Aug 31, 2024

LexiconCode commented Aug 31, 2024 • edited Loading

drmfinlay commented Sep 1, 2024 via email

drmfinlay commented Sep 1, 2024 via email

LexiconCode commented Sep 3, 2024

drmfinlay commented Sep 4, 2024 via email • edited Loading

drmfinlay commented Sep 25, 2024 via email

LexiconCode commented Sep 25, 2024 • edited Loading

dougransom commented Sep 25, 2024

dougransom commented Sep 25, 2024

LexiconCode commented Sep 26, 2024 • edited Loading

LexiconCode commented Sep 26, 2024 • edited Loading

drmfinlay commented Sep 26, 2024 via email

LexiconCode commented Sep 27, 2024 • edited Loading

drmfinlay commented Sep 27, 2024 via email

drmfinlay commented Sep 27, 2024 via email

LexiconCode commented Sep 27, 2024

LexiconCode commented Jul 7, 2024 •

edited

Loading

LexiconCode commented Jul 9, 2024 •

edited

Loading

LexiconCode commented Jul 10, 2024 •

edited

Loading

LexiconCode commented Jul 14, 2024 •

edited

Loading

LexiconCode commented Jul 14, 2024 •

edited

Loading

LexiconCode commented Aug 31, 2024 •

edited

Loading

drmfinlay commented Sep 4, 2024 via email •

edited

Loading

LexiconCode commented Sep 25, 2024 •

edited

Loading

LexiconCode commented Sep 26, 2024 •

edited

Loading

LexiconCode commented Sep 26, 2024 •

edited

Loading

LexiconCode commented Sep 27, 2024 •

edited

Loading