Fwiw the feature looks pretty natural to me, and it's certainly something I've been wanting to have before.

Now that #20054 is in I think this is now working properly on horizontal/vertical colorbars.

I didn't find any tests of the Zoom/Pan tools, am I missing those somewhere, or do we not test those ones and only test the widgets separately?

I agree with @jklymak concerns. We have two spaces, the data space (a, b) and the normalized "color" range (0, 1), which is associated with the color values. A the colored area in a colormap represents this color range and naively, zooming-in into the color range would result in a subrange and this a fraction of the color-range to be used, while the norm (i.e. the data-to-color relation) is unchanged. This however is less useful because you don't get additional detail. Actually, I'm wondering if it is useful at all.

What people usually want is to manipulate the norm. Technically, you would like to move the ticks on the colorbar, but that's not something we can directly support as a mouse interaction.

So two questions:

1. Is the naive color-range manipulation something that we consider supporting at all? If so, we cannot use zooming and panning tools for norm manipulation.
2. If not, is the changed semantics of the tools for colorbars as proposed here bearable in terms of user expectation or is that too confusing?

Mouse scrolling

There are standard conventions for mouse scrolling

scroll-up/down: vertical scroll
shift+scroll-up/down: horizontal scroll
ctrl+scroll-up/down: zoom in/out

For a regular axes the first to should translate to pan and the third should translate to zoom. (We don't have this implemented right now, but it would be a nice addition if anybody is interested in picking this up.)

We'd have to think if these could be resaonbly be adapted for colorbars in the above context.

Maybe a use-case for why I wanted this in the first place would be helpful. When making figures, coauthors will often say, well what if vmin/vmax were just a little higher/lower, how would that look? That could be done with adding widgets/sliders to control the vmin/vmax, but I don't want to be adding components, so I figured this was a good way to fit it in. Zoom/pan seemed natural to me, but if it isn't to other people we can consider some other interactions as well.

• set_xlim() - interesting, I hadn't considered that. It seems like set_xlim/set_ylim on a Colorbar axis should be dispatched to norm.vmin/norm.vmax and not zoomed in to a smaller data range. For example, that could be misleading if you have extensions on your colorbar, but the limits at the top aren't actually the norm limits...

• Zoom out / extend limits - This should be right click and drag I believe, the same as a normal axis because I just took most of the code from there. This also works with the home, back, and forward buttons too.

• Moving the ticks on the colorbar - I'm not sure I follow here? The ticks are moving in the above example. Do you mean you want to interact with just the ticks outside the axis and not inside the axis where the colors are?

As for the questions about whether this seems bearable for user expectations or not... I came up with this idea, so I'd say yes, but YMMV :) Another thing here is that you have to explicitly enter zoom/pan mode to get this available, so it isn't something that will just blindly be available for someone mousing over the image, so there is that safety net in a sense.

So maybe we start with set_xlim/set_ylim? Currently we do not have a colorbar.set_lim because we usually expect the user to change the colorbar by changing the vmin/vmax on the Norm. Hence the colorbar is a subordinate to the Norm. This PR inverts that and makes the colorbar actively control the norm. I will say this is also a confusion I see on stackoverflow that folks think the colorbar should somehow control the colors in the image. Currently, it is really easy currently to say "no it doesn't", the colorbar passively reflects the norm. If we no longer make that the case, then we need to think this through very carefully what the back and forth is and have thorough tests.

Again, maybe this is OK, but can we develop a programatic API first that doesn't rely on pan and zoom? i.e. colorbar.set_lim or colorbar.set_vlims, or?? That would make me a lot more comfortable rather than jumping to a GUI-only API, and would make the discussion a lot easier...

Some questions. If I set the vlim on the colorbar, and then I set it on the mappable, which is used at draw time? What if someone flips the limits? Norms often have other parameters than limits, in particulate everyone's favourite is symlognorm, or centered norm. How will this interact with those? Will the center move as well?

I'll put on the dev call for today. Feel free to stop by and pitch it. I don't know if we should discuss all the nitty gritty, but putting the PR on folks' radar and asking for comment here (or if you make a simpler colorbar.set_vlims PR) would be great.

I didn't find any tests of the Zoom/Pan tools, am I missing those somewhere, or do we not test those ones and only test the widgets separately?

Zoom is tested in lib/matplotlib/tests/test_backend_bases.py::test_interactive_zoom.

I guess the other UI for this is instead of mucking with the colorbar directly, a norm editor pops up in a separate helper window. This could also change the colormap.

@jklymak, you missed a good lengthy discussion today :) I think what you're proposing is essentially @anntzer's project here: https://github.com/anntzer/mplinorm

I'll be giving this some more thought over the next week and try to improve the UI.

OK, so if I zoom, I change the color limits of the ScalarMappable, but if I set the limits of the axis, i.e. with set_xlim it zooms in on part of the colorbar (at least after #20054) but leaves the mappable unchanged.

I think this is a mistake, will open a different issue about that next, but resolving that is probably a pre-req for merging this.

Further, I don't see how to expand the limits with this GUI API.

I would assume that right click zoom and right-click pan "zoom out" like they do with normal axes?

I think that the pan/zoom verbiage is semantically correct (if a bit jarring on first pass), but the UI needs a bit of tuning to make it clearer what is happening.

Although we implement the colorbar on an (increasingly standard) Axes, it is much more like the tick labels than it is like an normal "image". Under the hood we have transforms that go from data space -> x/y. The absolute position within the figure is meaningless, but by putting ticks on the Axes we can locally give meaning to the (relative) position. Even if we remove the ticks, we the relative positions with in an Axes still have meaning. When we adjust these transforms (by adjusting the x and y limits) we call this "panning" and "zooming" (depending on if we change the dynamic range or not).

Similarly, when we color map we have a transform (implemented in 2 parts: the norm and the colormap) that take data from dataspace -> RGB. The absolute color has no meaning (change the color map does not change the inherent meaning of the data any more than translating the whole axes around the figure changes the meaning!), but the relative colors tell us something about the relations between the data. If we then add a colorbar then we can get back access to the absolute values the same way ticks do.

In the images below, I think it is un-controversial to say "the y-axis was zoomed" or "the y-axis was panned". Doing exactly the same thing to the clim I think makes sense to use the same wording (there are some slight differences like when the curve goes out of the bounds it gets clipped but the color mapping saturates, could set over/under colors to transparent to get the same effect). Given the limitations of screens (we can not get a color axis sticking out of the screen!), I think implementing pan/zoom on the colorbar to "pan" and "zoom" in norm-space is the next best thing.

import matplotlib.pyplot as plt
import numpy as np

th = np.linspace(start := 0, stop := 2 * np.pi, N := 1024)
data1d = np.sin(th)

data2d = np.sin(th[:, np.newaxis]) * np.cos(th[np.newaxis, :])

curve_axes = ["1d", "y zoom in", "y zoom out"]
image_axes = ["2d", "color zoom in", "color zoom out"]

fig, ax_dict = plt.subplot_mosaic(
    [
        curve_axes,
        image_axes,
    ],
    constrained_layout=True,
)

for an in curve_axes:
    ax = ax_dict[an]
    ax.plot(th, data1d)
    ax.set_title(an)
    if an == "y zoom in":
        ax.set_ylim(-0.01, 0.01)
    elif an == "y zoom out":
        ax.set_ylim(-100, 100)
    else:
        ax.set_ylim(-1, 1)

for an in image_axes:
    ax = ax_dict[an]
    im = ax.imshow(
        data2d,
        extent=[start - (stop - start) / (2 * N), stop + (stop - start) / (2 * N)] * 2,
    )
    ax.set_title(an)
    fig.colorbar(im, ax=ax, aspect=7)
    if an == "color zoom in":
        im.set_clim(-0.01, 0.01)
    elif an == "color zoom out":
        im.set_clim(-100, 100)
    else:
        im.set_clim(-1, 1)

fig.savefig("/tmp/zoom.png")


curve_axes = ["1d", "y pan up", "y pan down"]
image_axes = ["2d", "color pan up", "color pan down"]

fig, ax_dict = plt.subplot_mosaic(
    [
        curve_axes,
        image_axes,
    ],
    constrained_layout=True,
)

for an in curve_axes:
    ax = ax_dict[an]
    ax.plot(th, data1d)
    ax.set_title(an)
    if an == "y pan up":
        ax.set_ylim(0, 2)
    elif an == "y pan down":
        ax.set_ylim(-2, 0)
    else:
        ax.set_ylim(-1, 1)

for an in image_axes:
    ax = ax_dict[an]
    im = ax.imshow(
        data2d,
        extent=[start - (stop - start) / (2 * N), stop + (stop - start) / (2 * N)] * 2,
    )
    ax.set_title(an)
    fig.colorbar(im, ax=ax, aspect=7)
    if an == "color pan up":
        im.set_clim(0, 2)
    elif an == "color pan down":
        im.set_clim(-2, 0)
    else:
        im.set_clim(-1, 1)
        
fig.savefig("/tmp/pan.png")

plt.show()

As mentioned on the call, this only makes sense for continuous (i.e. not categorical) norms.

I don't have any problem with the semantics of zooming. I have practical concerns about the complexity of two-way linking of the colorbar to the mappable/norm it represents.

I do have another issue with the GUI aspect of zooming on a colorbar. @greglucas has a nice hefty colorbar up there, and you can still see him struggle to get the drag box to stay in the colorbar. I almost never have such large colorbars as I consider them a waste of space so I would be even more screwed. Below is a published example:

struggle to get the drag box to stay in the colorbar.

I don't think you need to keep the cursor in the colorbar; as long as the initial click is in it, the drag box will just be clipped to the axes.

Thats good - but it's still a little fiddly...

@jklymak, the colorbar and mappable are currently directly linked. cb.norm is cb.mappable.norm So, if you update the vmin/vmax of either the colorbar or the image mappable, that state should get propagated. I think this is the way it should be as well, I want my colorbar to represent the data in my image when I've updated the state and not lag behind or be out of sync somehow.

I suppose if you make your colorbar too small and can't interact with it, then that just means you can't take advantage of this feature as easily, it doesn't adversely affect anything. There was a discussion about trying to interact with the ticks instead of the axis/colored area to make it more explicit as well what we are changing.

@jklymak, the colorbar and mappable are currently directly linked. cb.norm is cb.mappable.norm So, if you update the vmin/vmax of either the colorbar or the image mappable, that state should get propagated. I think this is the way it should be as well, I want my colorbar to represent the data in my image when I've updated the state and not lag behind or be out of sync somehow.

Sure. But that does not necessarily have anything to do with the axes min/max. You are proposing that it does, and @tacaswell is proposing that zooming in on the colorbar is a "mistake" for some reason I've not heard. I'm not 100% against either proposal, but it needs some discussion. And just to reiterate, folks have asked to be able to zoom in on the colors on the colorbar, but to preserve the norm/colormap mapping. The ability to do so is new in #20054, so now is definitely the time to discuss if we don't want that behaviour, but except to implement these semantics, I'm not sure what the argument is against it.

I still feel the way forward here is if we are going to make this an action on the colorbar, we should first define the colorbar method that will do this, and leave the GUI discussion out of it.

Just a note about the limits corresponding to vmin/vmax. This is in the comments, which seems to indicate that a larger colorbar axes than just vmin and vmax is explicitly allowed in the old API (imagine contours on a plot, saturated at high values, but you still want them labeled on the colorbar.

        # copy the norm and change the vmin and vmax to the vmin and
        # vmax of the colorbar, not the norm.  This allows the situation
        # where the colormap has a narrower range than the colorbar, to
        # accommodate extra contours:

I guess I still just don't quite understand this use case and wonder if it can't be solved in a different manner... Is the real request to be able to use less of the colormap than is available by default because you want it to saturate at a different value? I've used this crude hack in the past to change the saturation point, so maybe a public function on a colormap to do something similar would be the better place for it than messing with limits on an axes?

cmap = mpl.cm.get_cmap('inferno')
# Remove the bottom 10% of the cmap
cmap_small = mpl.colors.LinearSegmentedColormap.from_list('inferno_small', cmap(np.linspace(0.1, 1.0, 256)))

pc = ax.contour(np.arange(1, 11), np.arange(1, 11), np.arange(100).reshape(10, 10), 
                levels=np.arange(0, 100, 10), vmin=0, vmax=50)
cb = fig.colorbar(pc)

is a use case where the norm goes from 0 to 50 but the colorbar goes to 100.

BTW, just to be clear, I'm not saying the above functionality should trump the proposed functionality, just that it is something we will break...

I was mostly thinking about the case where you make the range of the color bar narrow than the vmin/vmax which seems like a miss-feature to me (because then we have put colors in the figure that do not have a key to indicate what that color means), but making the top/bottom of the color bar bigger than the norm (and basically implementing the extend arrows by having a constant area in the color bar).

I think I am going to back away from "the color bar limits should always exactly match the norm limits" to "the norm limits should be contained in the color bar limits".

The first thing that occurred to me under that framing is we have a norm_tracks_limits setting on the color bar. If True we get the behavior with pan/zoom that @greglucas is proposing here, if it is False, then when the user sets the limits on the color bar we expand them to include vmin/vmax (there is precedent for this with the nonsingular step) and if they are bigger just let it happen. In all of these cases the "extend" arrows will still be consistent.

I think the constraints are:

• every possible color in the cmap should be shown with a correct value in the color bar
• ever tick / value along the axis of the color bar should be matched with the color that value would be in the data

every possible color in the cmap should be shown with a correct value in the color bar

That seems overly prescriptive. Folks have specifically asked to have the limits tighter than vmin and vmax. Why would we forbid that?

My understanding is that @greglucas was hoping to come up with a different UI here, so putting in draft

I did take a look at trying to create a better UI, but ran into some issues that I'll have to look at in more detail. I'll put some notes here primarily for myself so I don't forget later. But, this is definitely still in a draft state :)

1. The Zoom box is drawn as a rubberband in backend-specific implementations. So, if I wanted to override that, I may have to do it on each backend. Alternatively, I could possibly shortcut out early if it is a ColorbarAxes and add two ax.vlines() (or a tickline-like extension) where the positions get updated as you drag and then removed once Zoom is off. The second suggestion is complicated by 2 below.
2. The interaction currently takes place on the outer_ax, which means it doesn't know it is a Colorbar, so we would need to add a private attribute identifier for this. However, then we don't want to do the draw on the outer_ax, but rather the inner_ax, so really I need to look at how to make all of the interaction take place on the base ColorbarAxes and dispatch the methods properly from the base class instead.

As mentioned in the dev call, I achieved something like this in my project using just the existing colorbar API and the picker functionality. The relevant functions are below:

https://github.com/richardsheridan/magic-afm/blob/c39b6e135263cc8d9b30b7e20fd8c1e9416257ae/magic_afm/gui.py#L1395-L1419

customize_colorbar is triggered in response to pick events on the canvas and (among other customizations) sets the clim directly on the underlying solid. This allows a user to simply click where they would like the top and bottom ends of their colorbar range.

expand_colorbar essentially only calls the latter part of Colorbar.update_normal() so that the actual gradient part of the colorbar is allowed to "expand" to the full range of the axes. This is a helpful mode to have available if the desired range of the image is much less than the total dynamic range of the image.

My point being, I think we can already produce functionality that covers the use case of the present PR with existing APIs, which -- if true -- would be a strong argument against invasive changes. My solution is not perfectly polished, as the ticks have some slightly weird but not terrible behavior by default. I will try to post a PR with a recipe suitable for discussion and review this weekend, hopefully to be converted into an example for the gallery.

@richardsheridan I think you'll find some of the tick issues etc are ameliorated by the recent changes to colorbar, but maybe not?

I think you'll find some of the tick issues etc are ameliorated by the recent changes to colorbar, but maybe not?

I pinned my app's dependency versions a while back but I'll try out the latest commits sometime soon since you bring it up

@richardsheridan, that sounds like another good possible solution! I had another meeting during the call today, but I should be able to make next week's dev call and I'd enjoy seeing a demo of that in action.

To me it still seems like something along these lines of "interactive manipulation of colorbar range" is desirable and now we need to figure out the best strategy of how to do it within MPL. I personally want it to be more than just an example of how to do it, I'm looking for this to be a keypress/event away within the GUI so I don't have to write my own click handler and attach it. I can already add a widget range slider to get a similar effect, but that doesn't help if I'm just doing quick-look plots.

Now that #20501 is in, I came back to this and it is simpler now to just have to interact on one main axes.

Short summary of updates:

• Removed interactivity from Categorical norms and contour(f)
• Added tests for horizontal/vertical, zoom-in/zoom-out/pan
• Refactored out some common helper methods to get axes bounds so the logic isn't duplicated here
• Extended the rubberband selector on the zoom tool all the way to the horizontal/vertical bounds

For the UI/UX, I decided to try out the simple just extend the rubberband (drawn by the backends already with the zoom tool) all the way out to the edges of the rectangle in one direction, and I think that looks pretty good. The axvline/axhline idea would complicate things a bit by having to keep track of those artists and remove them after the zoom release. This keeps it consistent with the core "zoom" rubberband implementation on each backend.

I'm marking this as ready for review again. I updated the gif in the original post showing the rubberband fully extended along the axis while zooming and that contour(f) colorbars not having this capability.

Just a friendly ping on this to see if there are any other comments/questions on this.

As discussed on the call today, I updated the API release note and the final commit also adds a way to get cla() to remove the colorbar interactivity.