WIP classify to rgba

[knaaptime]

this adds a function to generate an array of colors from an array of values. I've been toying with pydeck lately, which, basically, requires a column of colors on the df. Looking a bit closer, that's how most (all?) of the viz libraries work, so this is a shot at refactoring geopandas's logic for generating colors.

I can add tests, but wanted to open first to see if folks find this valuable and if it should live here

[codecov]

Codecov Report

Attention: Patch coverage is 91.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 89.6%. Comparing base (af76ab3) to head (0f64631).

Additional details and impacted files

@@          Coverage Diff          @@
##            main    #211   +/-   ##
=====================================
  Coverage   89.6%   89.6%           
=====================================
  Files          8       9    +1     
  Lines       1101    1125   +24     
=====================================
+ Hits         986    1008   +22     
- Misses       115     117    +2     

Files	Coverage Δ
mapclassify/__init__.py	100.0% <100.0%> (ø)
mapclassify/util.py	91.3% <91.3%> (ø)

[martinfleis]

I am think whether this should not be a class or a method on MapClassifier (probably the latter). Because to use it with a legend, you will normally be interested in other information apart from the array of colours.

[knaaptime]

i was curious about that, but it seemed like classify was doing the heavy lifting in geopandas, even with the legend?

probably no real reason we couldn't do both. The difference is a classifier class doesn't require a camp (edit: though, I guess if its a method it just consumes the cmap)

[knaaptime]

ah. but classify gives back the fitted classifier whereas this gives back the array.

Agree this makes more sense as a method, will move around

[knaaptime]

nevermind. If this gets pushed inside the class, there's nowhere to include the nan logic

I am think whether this should not be a class or a method on MapClassifier (probably the latter). Because to use it with a legend, you will normally be interested in other information apart from the array of colours.

so I think I would just create a classifier and the rgbas if you need both at the moment

[martinfleis]

That is a bit unfortunate to do the classification twice. Especially given some of them can be quite costly. Maybe include NaN tracking in the class? Like your legit_indices here?

[knaaptime]

yeah, agree. I think the nan-handle logic is only like those three lines. Without touching any of the classification code, we could maybe sneak it into the first step of the binning function so nans are ignored from the outset?

[sjsrey]

I'm not clear why there is a classifier called prior to here.
It seems to me this is trying to do a classless choropleth map [1].
If that is true, then the first classifier can be omitted and line 258 could become
norm = Normalize(v.min(), v.max())

But maybe I'm missing something here in my understanding?

[1] Tobler, W. (1973). Choropleth maps without class intervals. Geographical
Analysis, 5:262-265

[knaaptime]

no the whole idea is to use the classifier to get colors. It might not need that call to Normalize, but we're taking the value, discretizing it to the class bins, then using the bins to get colors from a matplotlib colormap

[sjsrey]

So do I have this right?

This was classified k=8, fisher jenks. So I'm thinking there should only be 8 colors in the figure, but the additional z axis allows for differentiation between units in the same bin but with different values. The color of the hex fill would be whatever class the hex value is placed into.

[knaaptime]

Yeah exactly. Z and color are disconnnected there. Color is based on Jenks but height is continuous

[knaaptime]

(i.e. all this does is take a cmap and translate bins --> colors)

so if you remove the z-dimension, this function gives you the same choropleth geopandas would give you. But then you can also use extrusion to encode the same or a different variable. A bit more at the end of this

[sjsrey]

this adds a function to generate an array of colors from an array of values. I've been toying with pydeck lately, which, basically, requires a column of colors on the df. Looking a bit closer, that's how most (all?) of the viz libraries work, so this is a shot at refactoring geopandas's logic for generating colors.

I can add tests, but wanted to open first to see if folks find this valuable and if it should live here

This is definitely useful functionality.

I'm not sure that mc is the place for this, for a couple of reasons. First, it makes matplotlib a hard dependency. Second, it seems to consume mc, rather than extend mc , for a particular use case.

I can see a few options:

Keep in it mc, but move it into a utility module that makes matplotlib a soft dependency.

Keep it as suggested with the hard dependency

Don't include it in mc but have it be part of a downstream package.

Other?

[knaaptime]

yeah agree. I'm happy to stick it downstream, but when @martinfleis and I discussed briefly on the last pysal call, we thought it might be useful more broadly (e.g. for geopandas to consume, since its coloring logic is currently a bit arcane).

would be easy to make matplotlib a module-level import if we want to go that route

[martinfleis]

I've discussed this with @sjsrey in Basel and came to a conclusion that it won't help geopandas at this point. We are planning on refactoring (or at least having a deep dive) of the plotting code for 1.1 or 1.2 though, so it may become more useful then. The issue is that we will always have mapclassify as optional dependency only and when no binning is applied we need to deal with missing values anyway.

[knaaptime]

cool. nice work on 1.0 btw!

i still think this has value here, e.g. for legendgram and such (would also give a logical place to live alongside #173), but lmk what you think. Happy to put elsewhere if you prefer

[martinfleis]

I am perfectly fine with keeping it here as I can see it can be useful. Just wanted to close the discussion about usage of this in geopandas.

[knaaptime]

I vote to put this into a util module with mc as a soft dep. It's not central to classification, but often useful alongside classification.

So it's really useful to keep around with the same group of tools, but shouldn't induce a new dependency

[martinfleis]

Shall we not require k and pass through all kwargs to support all classifiers and their options?

[knaaptime]

oh, dur. good point

[martinfleis]

If you make pre-commit happy lgtm.

[martinfleis]

kwargs should probably be mentioned here.

[knaaptime]

cool, i'm always unsure what to do in the docstrings when there's a catchall

[martinfleis]

I always mention where are they passed to.