[macosx] Expose window position, size, screen info, and event notifications#31172
[macosx] Expose window position, size, screen info, and event notifications#31172alberti42 wants to merge 5 commits intomatplotlib:mainfrom
Conversation
…ckend FigureManager_init (line 611–616) — now uses PyArg_ParseTupleAndKeywords with optional x (default 100) and y (default 350) float kwargs, passed through to NSMakeRect. 5 new C functions (lines 813–873), inserted just before FigureManagerType: ┌────────────────────────────────────┬────────────────────────────────────────────────────────────────┐ │ Function │ NSWindow/NSScreen call │ ├────────────────────────────────────┼────────────────────────────────────────────────────────────────┤ │ FigureManager_get_window_frame │ [window frame] → (x, y, w, h) as 4 doubles │ ├────────────────────────────────────┼────────────────────────────────────────────────────────────────┤ │ FigureManager_set_window_frame │ [window setFrame: NSMakeRect(...) display: YES] │ ├────────────────────────────────────┼────────────────────────────────────────────────────────────────┤ │ FigureManager_get_screen_frame │ [[window screen] frame] → (x, y, w, h) │ ├────────────────────────────────────┼────────────────────────────────────────────────────────────────┤ │ FigureManager_get_window_screen_id │ NSScreenNumber from deviceDescription → CGDirectDisplayID │ ├────────────────────────────────────┼────────────────────────────────────────────────────────────────┤ │ FigureManager_set_window_level │ [window setLevel: NSFloatingWindowLevel / NSNormalWin dowLevel] │ └────────────────────────────────────┴────────────────────────────────────────────────────────────────┘ tp_methods table (lines 917–931) — 5 new entries added. lib/matplotlib/backends/_macosx.pyi Full type stubs for FigureManager covering all existing and new methods. lib/matplotlib/backends/backend_macosx.py FigureManagerMac.__init__ now accepts *, x=None, y=None and forwards them to the C __init__ when provided.
…t, _window_resize_event, _window_move_event, _focus_in_event, _focus_out_event. src/_macosx.m @interface Window — added - (PyObject*)pyManager; declaration. @interface View — added declarations for windowDidMove:, windowDidBecomeKey:, windowDidResignKey:. @implementation Window — added pyManager accessor (lines 1261–1264) between closeButtonPressed and close. windowDidResize: — augmented with a second PyObject_CallMethod on [window pyManager] calling _window_resize_event with (width, height), reusing the already-acquired GIL. Three new delegate methods added after windowDidResize:: - windowDidMove: — calls _window_move_event(x, y) with the window's new Cocoa origin - windowDidBecomeKey: — calls _focus_in_event() via gil_call_method - windowDidResignKey: — calls _focus_out_event() via gil_call_method backend_macosx.py FigureManagerMac.__init__ — initialises self._window_event_callbacks = cbook.CallbackRegistry(). Six new methods on FigureManagerMac: mpl_connect, mpl_disconnect, _window_resize_event, _window_move_event, _focus_in_event, _focus_out_event.
…ize_end_event and _window_move_end_event --- src/_macosx.m @interface View — two new ivars (BOOL _in_move, id _move_monitor) and windowDidEndLiveResize: declaration. initWithFrame: — initialises both to NO / nil. dealloc — safety cleanup: if a monitor is still installed when the View is deallocated (e.g. window closed during a drag), it is removed and released before [super dealloc]. windowDidMove: — on the first call (flag is NO), installs a local NSEventTypeLeftMouseUp monitor. The block uses __block View* blockSelf (not retained in MRC, avoiding a permanent retain cycle) and __block id monitor (so the block can reference the monitor to remove it). When the mouse-up fires, the block fires _window_move_end_event, removes and releases the monitor, and clears the flag. windowDidEndLiveResize: (new) — single gil_call_method call to _window_resize_end_event. No flag needed since macOS provides this natively. backend_macosx.py Two new dispatcher methods: _window_resize_end_event and _window_move_end_event. The mpl_connect docstring updated to document all six event names and their callback signatures. --- To test, after rebuilding: mgr.mpl_connect('window_resize_end_event', lambda: print("resize ended — final frame:", mgr.get_window_frame())) mgr.mpl_connect('window_move_end_event', lambda: print("move ended — final pos:", mgr.get_window_frame()[:2]))
Currently get_screen_frame() only tells you about the screen the window is already on. You need the frames of all screens before deciding where to place the window. The fix is to add a module-level function (not a method on FigureManager, since it doesn't depend on any window).
|
I am skeptical, but have become less skeptical as I tried to explain why I am skeptical. I have some knee-jerk concerns about window level events. We already have axes and figure in/out events and a notification on the canvas/figure resize, it is unclear to me what window level events would give you that you can not already get (e.g. we expect the UI system to propagate window resize events down to the widget layouts and then we respond to that). I'm struggling to think of a use case where you want something in the plot to be reactive to where on the screen the figure is! the macOS backend is a bit different than the other GUI backends in that we fully own the nativetoolkit to Python wrapping rather than relying on a (optional) dependency / CPython to provide it. To my understanding our wrapping of the native UI toolkit is more-or-less minimal to what it is we need to do our stand-alone figures and nothing else. Thus for other toolkits you can reach in and do what ever you want on the underlying objects where as for macOS you can't. A more minimal change would be to expose a UI-specific API on the macOS backend that brings it closer to parity with the other toolkits. I also do not have a good grasp of if we expose enough at the right level to allow users to build bigger applications embedding our widgets or not (if we don't that would also be interesting to add assuming it is not outlandishly complicated). The proposal here to add a second event system to the manager classes to handle window level events only for our stand-alone figures (which eliminates some of my initial concerns about what this would do for (complex) embedding use cases) is reasonable and seems technically sound. However, we have historically resisted turning into a multi-UI toolkit shim layer and it is not clear to me that if you want to be doing complex window managemen/layout work it is not better to pick a toolkit and do the direct embedding yourself (so you have full control of the windows in a native-to-the-toolkit way not laundered through a lowest-common-denominator shiim layer). If the goal is to make these management layers play nicely with |
Indeed, I would fear that if we start exposing shims beyond those strictly needed by Matplotlib, it becomes very hard to draw a line where to stop.
I would guess that the minimal solution is to expose the ObjC-level FigureManager->window (as a raw (int-casted) pointer to a NSWindow or as a PyObjC wrapper (picking up an optional dependency on pyobjc)) and let third-parties do whatever they need with it (with their own native code or via PyObjC). |
Addendum to b162944. Adds the missing getter counterpart to set_window_level() — returns True if the window is at NSFloatingWindowLevel (always on top), False if at NSNormalWindowLevel. - FigureManager_get_window_level in _macosx.m - Corresponding entry in tp_methods table - Type stub in _macosx.pyi
|
Thanks for taking the time to review and comment. I think my intent may not have come through in the PR description, and I likely did Let me give you some more background and clarify what this PR aims to achieve (and The real use case (and why it matters)This work comes from a very practical workflow: multi-monitor scientific/experimental In this setting, window layout reproducibility is a must-have:
The goal of this PR is to make standalone Matplotlib windows manageable enough that As a concrete reference point, I built a small proof-of-concept package that restores https://github.com/alberti42/matplotlib-window-tracker It relies heavily on the primitives proposed in this PR (window frame + screen queries Why Matplotlib-level primitives (vs external hacks)Yes, there are external ways to control windows on most OSes (window-manager tools on But these approaches tend to be:
What's missing is a backend-native, Matplotlib-level way to:
Scope: not a general window-management layerI agree with the concern about scope creep. This PR is intentionally limited to a
This PR is intentionally macosx-first: it is a concrete, self-contained contribution If the feedback is favorable and this direction is acceptable for Matplotlib, I am |
|
I’d like to advocate the philosophy that any windows we create and manage (via pyplot) are exclusively in our responsibility. They should be considered a black box application from the user side, because any control or change a user may make on the gui framework level may conflict with our current or future handling of the window. The clean way is to instead create and manage the window yourself and add a figure canvas to it. There may be some ways to ease transitions between those extreme cases, e.g. exposing a handle with the clear warning that changes in window handling on our side can happen without warning. Or a helper to create a preconfigured window with canvas and toolbar, but let the user run the event loop. |
|
I understand the philosophy "pyplot windows are owned by Matplotlib". However, users already change those windows all the time: they move them, resize them, focus them, and put them on different screens. Matplotlib backends also already expose some window controls in practice (e.g. resize, fullscreen, title, etc., backend-dependent). The primitives in this PR do not open a new category of interaction. They automate what is already possible and already happening manually, and they make it feasible to restore a complex multi-window layout deterministically across reruns. On macOS specifically, when creating multiple figures, new windows currently appear With many windows this becomes painful quickly. Being able to set/query window frames and save them at the end of a move/resize gesture solves that usability issue without requiring users to abandon pyplot and reimplement their workflow as an embedded GUI application. Just to be clear (and avoid misunderstandings): I do not want to customize pyplot The only extra feature I consider important is an always-on-top flag, which is very |
|
@alberti42 it would have helped if you had first created an issue to discuss the idea and intentions. I’m inclined to say, we should only provide the bare minimum that is needed to support such a use case, I.e. @anntzer’s suggestion of a pointer. Just forwarding gui framework functionality individually per backend is only an indirection (call x on the backend instead x on the native window) and is not worth it. OTOH defining a unified backend-independent API would have some user value. We have a minimal set of such functionality in FugureManager https://matplotlib.org/stable/api/backend_bases_api.html#matplotlib.backend_bases.FigureManagerBase, some additions could be made here, but we should really see that we’re not starting to proxy all possible gui functionality. |
|
@timhoffm thanks for the follow-up.
I'm not attached to this particular implementation; I'm attached to the idea and to getting a good solution merged. I'm new to the Matplotlib contributor workflow and did not realize this should start as an issue. I marked the PR as If you'd prefer, I can open an issue summarizing the motivation + proposed API and we can treat this PR thread as a first-pass design discussion (or move the design discussion to the issue and keep the PR focused on implementation). I agree a clear design thread is useful; the only downside is splitting context across threads.
I want to make sure I understand what is meant by "a pointer" here. If it's a raw int-casted NSWindow pointer (or a PyCapsule), then Python users can only do something useful with it if they also bring their own bridge layer (PyObjC or their own native code / ctypes glue). If an optional PyObjC wrapper is acceptable, then yes: with an NSWindow handle you can call the native APIs for geometry, level, etc., so I'd be happy with that direction. My motivation for the explicit primitives in this PR is that they are a small, stable, testable surface that works from plain Python without requiring users to adopt PyObjC or write native glue. This is also why I'm hesitant to make a raw handle the primary (or only) solution. It raises the bar a lot for "normal" Python users: once you require PyObjC / native glue, you effectively push the real work into a separate package and into per-user tooling. In contrast, a small set of explicit primitives on the manager (geometry queries, set frame, end-of-gesture move/resize notifications, always-on-top) is usable from plain Python and can later be mirrored across backends so users get one coherent API rather than one-off platform hacks. If there is still interest in an escape-hatch handle, I'm not opposed to adding it, but I would strongly prefer it as an optional add-on, not as a replacement for the basic Python-level primitives.
I agree the end goal should be a backend-independent API, likely on FigureManagerBase, once it exists across the major interactive backends. This is something I did mention in the original PR description, but perhaps went unnoticed since my text was very long ;-) The reason this PR starts with macosx is that it's already a significant chunk of work, and I didn't want to propose/freeze a cross-platform contract without first getting feedback on the shape of the API and whether it belongs in Matplotlib at all. If this direction is accepted, I'm happy to follow up with dedicated PRs for other backends on Linux/Windows (Qt/Tk/Gtk), and then we can discuss promotion to a cross-backend contract once we have multiple implementations. |
What I propose is essentially diff --git i/src/_macosx.m w/src/_macosx.m
index 9ca6c07493..f834c1544d 100755
--- i/src/_macosx.m
+++ w/src/_macosx.m
@@ -679,6 +679,12 @@ bool mpl_check_modifier(bool present, PyObject* list, char const* name)
Py_TYPE(self)->tp_free((PyObject*)self);
}
+static PyObject*
+FigureManager_get__raw_nswindow_ptr(FigureManager* self, void* closure)
+{
+ return PyLong_FromVoidPtr(self->window); // Maybe expose as a ctypes.c_void_p object instead.
+}
+
static PyObject*
FigureManager__show(FigureManager* self)
{
@@ -819,6 +825,11 @@ bool mpl_check_modifier(bool present, PyObject* list, char const* name)
.tp_dealloc = (destructor)FigureManager_dealloc,
.tp_repr = (reprfunc)FigureManager_repr,
+ .tp_getset = (PyGetSetDef[]){
+ {"_raw_nswindow_ptr", (getter)FigureManager_get__raw_nswindow_ptr}, // Name TBD.
+ {} // sentinel
+ },
+
.tp_methods = (PyMethodDef[]){ // All docstrings are inherited.
{"_show",
(PyCFunction)FigureManager__show,which can be used as
Hopefully everything can be done at the PyObjC layer and without requiring end-user native code (admittedly I'm not a specialist of PyObjC so I don't know its limitations) but yes, I would push that functionality into the hands of end users. |
|
Thanks for the concrete patch and example; it's very helpful to understand your proposal. I only have one concern with a "raw NSWindow pointer" as the primary solution: I feel A small, explicit set of Python-level primitives (geometry + end-of-gesture I could add However, I have the impression that what you are really asking for is to minimize the Finally: I genuinely want to thank you all for the amazing work you have been doing |
|
This proposal is maybe having issues because it is quite all encompassing and because "figure" is a mixed concept in Matplotlib, referring to both the figure canvas and in some backends the gui window. That said, I think the idea of making the mpl-gui able to at least query and change the position of the gui windows is a pretty reasonable feature request. In fact, I was a little surprised to realize it is not a current feature Rather than digging around in the internals of the Mac backend, which is a bit of a special flower, I wonder if it would make more sense to sketch this out using an easier universal backend like qt5/6? The implementation would then just be a couple of lines of python. Finally, we keep talking anout separating pyplot and the mpl-gui concept. I still feel that would make sense and perhaps this added functionality would be more at home with a more standalone module like mpl-gui? |
|
Thanks @jklymak I think I understand your point about reviewability and long-term maintainability. It sounds like part of the difficulty here is that the macOS backend is a bit of a special case, so it’s hard to quickly judge whether this is a direction we want Matplotlib to take. Would it help if I put together a small, equivalent Qt proof-of-concept for the core idea here (window geometry, and only whatever minimal event hooks you think are in-scope), purely so maintainers can evaluate it in a backend/toolkit that’s easier to review? If yes, what would you prefer?
If that would be useful, I’m happy to do it in the next days; otherwise, I’ll pause and wait until we have a better plan. |
|
I think the devs group should think about it before any contributors get involved. What programatic python-level control do we think is reasonable for the pyplot GUI? I agree with OP that set/get_position and raise are reasonable asks. Of course downstream users can write their own Qt app or macosx app, but normal python users could benefit from being able to place their plots programatically. |
This is exactly what I would support (push the remaining support in glue code that we don't want to take responsibility for -- maybe it would be more maintenance for end users, but less for matplotlib developers...). In any case, I think I explained my position so I won't insist more or block this if there's strong agreement among the other developers in the other direction. |
|
I think the minimum api would be:
However, with wayland this seems to be technically impossible (https://wayland-book.com/xdg-shell-in-depth/interactive.html) from inside the application and I am pretty 👎🏻 on growing either a local implementation of a wayland client or picking up a dependency on one (and it is not clear that would even work in a reliable way!). |
Notes for reviewers
The PR is structured as four self-contained commits that can be reviewed in series:
get/set_window_frame,get_screen_frame,get_window_screen_id,set_window_level; initial window position at creationmpl_connect/mpl_disconnectonFigureManagerMac; livewindow_resize_event,window_move_event,focus_in_event,focus_out_eventwindow_resize_end_event(nativewindowDidEndLiveResize:),window_move_end_event(NSEvent local monitor)_macosx.get_screens()We are happy to provide self-contained reproduction scripts for any part of the
implementation to assist with review or manual testing.
PR summary
Why is this change necessary?
The macOS backend (
_macosx.m/backend_macosx.py) had no way to query or controlthe position and size of the figure window, query screen geometry, or receive
notifications when the user resizes, moves, focuses, or unfocuses a window.
These primitives are needed for applications that manage window layout programmatically
(multi-window dashboards, restored window positions, always-on-top inspector panels, …).
No equivalent public API exists in any other matplotlib backend; this PR establishes
the pattern for macOS first, with the intention of porting it to other GUI backends in
separate PRs (see Future outlook).
What problem does it solve?
After this PR, users of the macOS backend can:
Positioning a window at a precise location before it appears (no flicker, because the
NSWindow is created with
defer=YESand only shown onplt.show()):Positioning on a specific screen:
What is the reasoning for this implementation?
All changes are confined to three files:
src/_macosx.mlib/matplotlib/backends/backend_macosx.pylib/matplotlib/backends/_macosx.pyiGeometry functions (
get_window_frame,set_window_frame,get_screen_frame,get_window_screen_id,set_window_level) are thin wrappers around standardNSWindow/NSScreenCocoa APIs. Coordinates are in Cocoa screen-space points(logical pixels, Y-up, origin at bottom-left of the primary screen) — exactly what
[NSWindow frame]and[NSScreen frame]return natively.get_screens()is a module-level function (not a method onFigureManager) sinceit does not depend on any window. It iterates
[NSScreen screens]and returns(CGDirectDisplayID, x, y, w, h)for each connected display.Window event callbacks follow the same two-layer pattern already used for the close
button: an
NSWindowDelegatemethod on the CocoaViewclass calls back into thePython
FigureManagerviaPyObject_CallMethod/gil_call_method, and the Pythonside dispatches to user callbacks via
cbook.CallbackRegistry.window_resize_end_eventuses the nativewindowDidEndLiveResize:delegate method.window_move_end_eventuses a temporaryNSEventMaskLeftMouseUplocal eventmonitor, installed on the first
windowDidMove:notification and torn downautomatically on mouse-release (or in
deallocas a safety net). This syntheticapproach — set a flag on move start, fire on mouse-up — is the pattern that can be
ported to other backends via their own mouse-release hooks.
A
pyManageraccessor was added to theWindowObjC class so thatViewdelegatemethods can reach the Python
FigureManagerwithout breaking encapsulation.Future outlook
Other GUI backends. We intend to implement the same API for Qt5, GTK3, and other
meaningful GUI toolkits, with the goal of eventually covering all interactive backends.
Each backend will be handled in its own dedicated PR to keep reviews focused.
Promotion to
FigureManagerBase. Once the API has been validated across allbackends, the relevant methods should be promoted to
FigureManagerBaseso theybecome part of matplotlib's official cross-platform contract. We deliberately hold off
on this until all backends are covered, to avoid freezing an interface based on a
single backend's constraints.
Tests. Tests for the new API will be added to
lib/matplotlib/tests/test_backend_macosx.py, following the existing pattern ofsubprocess_run_helper+@pytest.mark.backend('macosx', skip_on_importerror=True).Documentation. A dedicated API documentation page and a
What's Newentry willbe added once the API stabilises across backends.
PR checklist
— tests will be added to
lib/matplotlib/tests/test_backend_macosx.pyfollowingthe existing
subprocess_run_helperpattern before this PR is marked readywindow management primitives, not plotting features
directive and release note: will be done before this PR is marked ready
general and
docstring
guidelines: will be done before this PR is marked ready
AI assistance disclosure
See policy: