Proper MacOS Support

[tryptech]

I'm going to use this as the master issue for triaging all of the Mac related issues as I start to build that functionality out properly. I will be addressing these issues as I go along in the separate branch mac-rewrite. Please note any issues in functionality here as I build out support in that branch if you choose to test it.

I'm basing all of this work around an M2 Mac Mini running macOS Ventura 13.2.1 so I appreciate your understanding if the script does not end up working properly on older versions of macOS or Intel-based Macs. I believe the OBS code itself branches at macOS version 12.5 with the introduction of M1 Macs, so I am looking for maintainers for Intel-based Macs below macOS 12.5.

Current Issues:

• Not detecting monitor sources (MacOS - TypeError: tuple indices must be integers or slices, not str #82, Zoom source list empty on MacOS 13.2 (22D49) Chip Apple M1 Pro #95)
• Slow loading of monitor sources (MacOS - TypeError: tuple indices must be integers or slices, not str #82)
• Improper mouse tracking at HiDPI (Zoom source list empty on MacOS 13.2 (22D49) Chip Apple M1 Pro #95)
• Improper mouse tracking across multiple monitors

[tryptech]

In doing a deep dive for properly detecting monitor sources, I'm finding that OBS does not properly report the settings for the macOS Screen Capture source.

In the case of Windows and Linux, there are separate sources for window capture and monitor capture. This makes it easy for me to determine what a user wants to do given a source selection and lets me branch the code appropriately. MacOS, from 12.5 onwards, uses a single source type for both monitor capture and window capture.

I'm going to leave the existing code as is for dealing with older Macs and I'm going to write separate handling methods for Window capture and Display capture for newer Macs to avoid breaking any existing functionality.

[tryptech]

I have preliminary support for macOS screen capture for macOS 12.5+ as of commit 3807d36. I also have tried to offset/avoid delays where I can for now: macOS will primarily only lag now on script initial load, script reload, source change, and source refresh. Changing source settings should no longer cause lag. This should also provide minor improvements to script performance on Windows and Linux. There is still a minor ~1 second lag on macOS when zooming in or out: I will have to revisit that later.

[CryptoChrisJames]

Current Setup:

• M1 MBP
• OS: Ventura v13.0.1 (22A400)

I pulled down the mac-rewrite branch and I've been able to successfully select my source and zoom in and out. My only issue now is that the mouse position doesn't seem to be tracked properly. In order to get the zoomed area to move, my mouse has to venture outside of it. This also prevents it from being able to reach the edges of the screen. I can't get the very top or the sides of the screens into the zoomed area.

[HeyBradyLang]

Current Setup:
M1 16" MBP
OS: Ventura v13.0.1 (22A400)
Secondary Display: 3440x1440

To add to @obsidiantech , I'm experiencing something similar, but different...

Using Display Capture on my Second Monitor,

✔️ I've also be able to select my source and zoom in and out.
❌ It's not zooming in on my mouse location, but instead zooming in on the same location of my screen.

✔️ It follows my mouse vertically from the top edge of my screen all the way to the bottom edge of my screen
❌ It will not follow my mouse horizontally, at all.

[tryptech]

@HeyBradyLang @obsidiantech Could I get an illustration from you both of how your monitors are laid out, with labels of each monitor's physical native resolution, the resolution reported by macOS (if possible similar to how #95 finds that OBS and pywinctl get a reported resolution of 1512*944), and which one is the primary monitor? I can't get to it really until next weekend, but I'd like some data on the issue.

[HeyBradyLang]

@tryptech

I pulled the most recent, updated commit of the script (68868e9) and the Zoom and Follow work, except, I'm facing this issues as well:

CryptoChrisJames: I pulled down the mac-rewrite branch and I've been able to successfully select my source and zoom in and out. My only issue now is that the mouse position doesn't seem to be tracked properly. In order to get the zoomed area to move, my mouse has to venture outside of it. This also prevents it from being able to reach the edges of the screen. I can't get the very top or the sides of the screens into the zoomed area.

So, my primary display is my Laptop Screen and Secondary is a LG Widescreen Monitor. It is arranged similarly, where x=0 is the leftmost side of my laptop and x=2658 is the rightmost side of my laptop. Canvas = 3840 x 2160.

1. Laptop = 3456x2234 according to System Preferences > About. This also reports as "2056x1329" in Display Capture Drop Down, and "4112x2658" in OBS Suggested Canvas size and Projector View... Lol WTF.
2. Widescreen Monitor = 3440x1440 according to System Preferences > About. This is universally accurate everywhere. In OBS Display Capture Drop Down, this is "3440x1440 @ 2056,0"
   For simplicty's sake, I'm just capturing the entire display and forcing it into my Canvas, leaving a theoretically strange Zoomed Out look.

Follow
✔️My Follow is able to reach all 4 corners of my widescreen display..
❌ However, it is not tracking the position of my mouse -- the X Axis is my Laptop Screen, so I can't follow anything on the Widescreen, however, moving my mouse over to the laptop screen will start moving the widescreen accordingly. Therefore, only the rightmost part of my display has the mouse in focus. See video:
https://youtu.be/1bj3ZdWBp0s

Zooming In/Out
❌ When "Zooming Out", my ZoomCrop Filter resets to size [X,Y] = [0,0] and Transform [x,y] = [1px,1px]
✔️ When "Zooming In", my Zoom Crop Filter goes to the expected size, and zooms in on the "Mouse Location", but because of the incorrect X-Axis mouse position (mentioned above), it's not always actually on my mouse. When hiding the "ZoomCrop" filter on my recorded source, it goes back to the expected size of the originally recorded source
See Video (Sorry, the way this is filmed is painful, but I hope it's clear):
https://youtu.be/Jh7jCAG0B64

I have set my manual monitor size (3440x1440) and the transform is set correctly. (Stretch to Outer Bounds, Center, etc.)

[Kalmat]

HI @tryptech,

I am working on a new version of pywinctl module (v0.1). Amongst other improvements, this new version fixes two issues which may affect your module, and especifically in macOS:

• Calling getClientFrame() from a non-main thread (fixed in a recent PR by super-ibby)
• Monitor list after calling getAllScreens() is overwritten in macOS in case the monitor names are identical (check this)
• I discovered that PyRect module (which was an important part of the original PyGetWindow module) is not properly handling multi-monitor setups (well, it was not intended to do so, actually). I replaced PyRect module by a custom class which is now (hopefuly) multi-monitor ready.

We can work together to check if this new version helps to solve some of your issues, before I upload it to PyPi (I have not uploaded it yet, I mean, and will wait until you have the time to check this new version if you intend to do so).

[tryptech]

Very good, I recently worked on some macOS code so any improvements there are much appreciated. I haven't delved too deeply into the multi monitor issues that have been reported since I don't use multiple monitors now, but I'm willing to test how it affects this modules multiple monitor support on Windows and Mac

[Kalmat]

Nice! I have the same problem: I don't own a "real" macOS and the VMs I use to test do not allow multi-monitor. We can perhaps try to find a workaround or any other user with a multi-monitor setup which mercifully wants to help! As a last resort, I think I can borrow "real" macOS for some days if needed.

Please, check this for some multi-monitor tips in macOS. Besides, you can download new version wheel from here and perform your own tests to check if some issues are already solved or I have to further work on them! Any help you need, just let me know.

[tryptech]

I have an actual Mac Mini and an iPad, so Sidecar should let me test multi monitor on macOS specifically, I just haven't had a chance to try it out yet. I'll grab the wheel and try to figure out what I can over this weekend.

[Kalmat]

good news! Of course, no hurries, don't worry.

Don't know if you are struggling with screen coordinates (if so, you have to be prepared to handle negative values in multi-monitor setups), or if you need to check whether a window has changed its position or status or has been moved to another monitor... Check the issue I was linking, as well as pywinctl features just in case it can help with that kind of things. Also, just ask me if you have a doubt (I don't have a clue about OBS, but I do know a few things about windows!).

[Kalmat]

Hi again, @tryptech. I've been making some additional tests, but still incomplete since I don't have access to the required full setup. By the moment, I think my assumptions, at least in macOS, were wrong. I will have to get at least 3 monitors and an actual macOS.

I will keep you informed on any progress. If, as a result of your own tests, you get any additional info or comment, I would really appreciate if you let me know.

[hqawasmi]

Hi Guys, I have tried the script on my Mac M1 today, its all working ,but when its zoom it it zoom to 0,0 dimension window, will issues be solved anytime soon ?!

[nishyp]

Multi-monitor test dummy reporting again for duty!
I left a few notes in #82 regarding macOS support, I've tried out the mac-rewrite branch with not much luck

Note: I'm currently on Catalina as I use this build primarily for music production and VST plugin support is shaky for older plugins that I would like to keep using.

System Info:
MacOS Catalina 10.15.7 - 9700k, 64GB RAM, RX 5700 XT
OBS 29.0.2 (Note: Catalina support dropped for newer versions 29.1+ more info here)
Python 3.10.9
PyWinCtl 0.1 (I've also left some test logs for you here @Kalmat)
obs-zoom-and-follow (mac-rewrite branch)

Here's my log when triggering a hotkey:

[Python] Python failure in hotkey_pressed:779:
[Python: zoom_and_follow_mouse.py] Traceback (most recent call last):
[Python: zoom_and_follow_mouse.py]   File "/Applications/OBS.app/Contents/PlugIns/frontend-tools.plugin/Contents/Resources/scripts/zoom_and_follow_mouse.py", line 1055, in toggle_zoom
[Python: zoom_and_follow_mouse.py]     zoom[attr]
[Python: zoom_and_follow_mouse.py] TypeError: 'CursorWindow' object is not subscriptable
[Python: zoom_and_follow_mouse.py] 
[Python: zoom_and_follow_mouse.py] During handling of the above exception, another exception occurred:
[Python: zoom_and_follow_mouse.py] 
[Python: zoom_and_follow_mouse.py] Traceback (most recent call last):
[Python: zoom_and_follow_mouse.py]   File "/Applications/OBS.app/Contents/PlugIns/frontend-tools.plugin/Contents/Resources/scripts/zoom_and_follow_mouse.py", line 1059, in toggle_zoom
[Python: zoom_and_follow_mouse.py]     zoom.update_source_size()
[Python: zoom_and_follow_mouse.py]   File "/Applications/OBS.app/Contents/PlugIns/frontend-tools.plugin/Contents/Resources/scripts/zoom_and_follow_mouse.py", line 391, in update_source_size
[Python: zoom_and_follow_mouse.py]     self.monitor_capture_mac(data)
[Python: zoom_and_follow_mouse.py]   File "/Applications/OBS.app/Contents/PlugIns/frontend-tools.plugin/Contents/Resources/scripts/zoom_and_follow_mouse.py", line 233, in monitor_capture_mac
[Python: zoom_and_follow_mouse.py]     if (monitor['id'] == monitor_index):
[Python: zoom_and_follow_mouse.py] TypeError: tuple indices must be integers or slices, not str

[Kalmat]

Hi everyone!

I eventually managed to test in an actual macOS with a multi-monitor setup (x2). Everything is working OK (at least in my system) and has been updated in new PyWinCtl v0.1 version (donwloadable directly from PyPi). This update is based in two new custom modules: PyWinBox (useful to define and control any rectangular area, including a window area) and PyMonCtl (aimed to get info on and control monitors, especially in multi-monitor setups, and including a watchdog which will track and notify monitors changes).

In case you find any issue, please let me know here

[tryptech]

Wonderful, I just updated last night to improve multimonitor support and HiDPI support, particularly on apple silicon based macs, as well as adding in that dependency on PyMonCtl. I'll have to look into using that watchdog for monitor updates later

[Kalmat]

@tryptech, just in case it is helpful in your case. In PyWinCtl you can set a watchdog passing a callback to be invoked when the window changes the monitor is in (as well as when it changes position, size, etc.). The callback will receive the new monitor name (you can also get it using getDisplay() method if you don't want to set a watchdog). This monitor name can be directly used within PyMonCtl, where you can also set a watchdog to track monitor changes. In short, combining both you can track/control every window and the monitor it is in.

Just let me know if you need help.