feature: multiple axes in charts

[AlexMooney]

Tracking more than one value axis is necessary for e.g. scatter plots and manipulating charts with a secondary axis.

In the oxml/chart/chart.py file, the catAx and valAx should be ZeroOrMore arrays of axes instead of ZeroOrOne values that store only the first axis in a chart. I think it would be nice to preserve the behavior today where you get the first axis with e.g. chart.value_axis and only expose the multiple axes via some new API like chart.value_axes_list.

There will need to be provisions for working with axis IDs and assigning the axis IDs for each plot within a chart. The pptx.chart.axis._BaseAxis will need to be able to set c:crossAx, c:crosses, and c:axPos values.

[AlexMooney]

I've started implementing this in my multiple_axes branch. So far I have the chart.value_axes_list and chart.category_axes_list working as well as ValueAxis.axisId.

[scanny]

Have you given any thought to what the API would look like? I've found that folks who start with the implementation usually end up with something that needs to be rewritten from scratch before commit.

Not intending to spoil the fun though :) There's nothing wrong with working your own fork for your own purposes. Just wanted to provide an early heads up that if you wanted to contribute your work back to the main branch you might do better with an outside-in approach :)

[scanny]

Read your second post first, just now getting to the first one, thought it was one I read from earlier, sorry about that :)

I'm thinking the right api for the axis access would be:

Chart.value_axis
# and
Chart.secondary_value_axis

where each would return either the appropriate axis object or None. Pie charts for example have no value axis IIRC, so there is at least one case where both would return None.

The category axis access API would look the same.

Chart.category_axis
# and
Chart.secondary_category_axis

The only question in my mind is whether there could possibly be more than four, in particular in the case of a 3D chart. I haven't looked into those seriously yet, but it probably bears an initial investigation at least.

@AlexMooney it probably makes sense to get a feature analysis page started in your branch we can collaborate on to noodle out and document these decisions. The key success factor for getting a pull committed turns out to be whether it started with an analysis page. There are just so many small detail to be decided up front that become big factors in the design and implementation that it's statistically very improbable to get it right unless some up-front "pre-factoring" gets done :)

Let me know if you want more on how to get one started. Here's an example:
https://github.com/scanny/python-pptx/blob/master/docs/dev/analysis/features/cht-axis-has-gridlines.rst

[huandzh]

Hi, here is my solution at https://github.com/huandzh/python-pptx/compare/chart_cat_multiLvl. It worked for my working scripts for sometime.

• Provided an additional ChatData class called ChartDataMoreDetails, and support multilvl categories, blank categories and blank values, format_code etc.
• Modified .replace_data without breaking compatibility.

I hope it is helpful.

Sample code：

from pptx import Presentation
from pptx.enum.chart import XL_CHART_TYPE
from pptx.util import Inches
from pptx.chart.data import ChartDataMoreDetails
# create presentation with 1 slide ------
prs = Presentation('chart-01test.pptx')
chart = prs.slides[0].shapes[0].chart
# define chart data ---------------------
chart_data = ChartDataMoreDetails()
#one line of categories
# chart_data.categories = [#[(0,'top3'),(4,'others')],
#                          [(0,1),(1,2),(2,3),(4,4),(5,5)]]
# categories and values in (idx, value) pair
chart_data.categories = [[(0,'top3'),(4,'others')],
                         [(0,1),(1,2),(2,3),(4,4),(5,5)]]
#chart_data.values_len = 6
chart_data.add_series('Score>90', 
                      [(0,98.32),(1,97.12),(2,95.54),(4,92.12)],
                        format_code='0.0')

chart.replace_data(chart_data)

prs.save('chartmore-02.pptx')

[AlexMooney]

@huandzh, those are some great changes but as far as I can tell, there's no implementation of secondary axes. Can you create charts like the one I pasted below?

@scanny, this scatter plot has 4 <c:valAx>s (with 2 <c:scatterChart>s) and no category axes:

This surface plot has a <c:catAx>, a <c:valAx>, and a <c:serAx> (which are not yet supported).

I'll start writing a feature analysis...

[AlexMooney]

I've uploaded a draft of an analysis page. It's here.

[scanny]

Hi Alex, this looks great :)

I'm traveling this week, but will take a closer look over the weekend and provide some comments.

[scanny]

Hi @AlexMooney, I read through your draft feature analysis. My main first reaction is that I'm rethinking my idea of the axis access API being Chart.value_axis, .secondary_value_axis, .category_axis, .secondary_category_axis. That would be fine if that were the most there could be, but your example showing four value axes demonstrates that design is not going to be general enough.

Taking a closer look at the Microsoft API it looks like the possible axis configurations for a chart are a partial cover of a three-dimensional space where axis-type(value, category, series), axis-group(primary, secondary), and orientation(horizontal, vertical) are the three dimensions.

I'm thinking the best next step is a reasonably thorough case analysis about what configurations are possible. Those would inform the API design and then go on to be test cases.

I'm starting to like the idea of having an Axis collection on Chart that would allow iteration over the various axes, whatever they happen to be. Then having .type, .axis_group, and .orientation properties on each axis so they could each be well characterized. The .value_axis and .category_axis properties on Chart could stay since they're so handy for the most common cases.

I suppose the next step after that would be to have a way to add an axis. The MS API uses the Chart.HasAxis(type, group) method/property for this, but I think we'd want something a bit more Pythonic in this case; it would be odd to see something like this in Python code although it might be possible to make it work with descriptors or something:

chart.has_axis(XL_AXIS_TYPE.VALUE, XL_AXIS_GROUP.SECONDARY) = True

Off the top of my head I'd be thinking something more like:

chart.add_axis(XL_AXIS_TYPE.VALUE, XL_AXIS_GROUP.SECONDARY)

I don't see how they manage to cover all the possibilities with just the two parameters though. I suspect they overload XlValue and XlCategory to mean VALUE+VERTICAL and VALUE+HORIZONTAL respectively in the case of an XY Scatter plot. It would take some experimentation with the MS API to discover that I expect.

Anyway, it looks like it's shaping up to be a fairly big job to handle the general case. I think the following list of features would be a good place to start because 1) they don't break anything, 2) they would be required infrastructure for the rest of it, and 3) they would provide an opportunity to uncover more of the details required for the other bits like adding a new axis:

• Chart.Axes -- collection of all axes in chart, having list semantics I think for a start, but possibly using dictionary-style lookup on a 3-tuple (type, orientation, group).
• _BaseAxis.type, .orientation, and .group

On the issue of validation, I think we'd need to have the case analysis worked out pretty firmly so we could keep the user from doing something that PowerPoint would barf on. It creates a support problem when newbie users can generate a presentation that won't load. Much better to just raise an exception when they try something we know won't work. Of course that means we'd need to do the analysis to know what will work and what won't.

[AlexMooney]

In the XML, orientation and group are not directly represented. Instead of orientation there is position (top, bottom, right, left; presumably others for 3d charts), although we could make a lookup to do something like 'v' if position in ['l', 'r'] else 'h'. The group seems to be determined by which axes come first in the XML. I was able to swap which horizontal axis was primary by rearranging the c:valAxs with a text editor. I'll work on orientation and group based on the order the axes appear in.

_BaseAxis.type and _BaseAxis.position have already been added in my multiple_axes branch.

[AlexMooney]

I added the _BaseAxis.orientation and Chart.axes which supports the same indexing stuff that Chart.plots does.

Not sure how to implement _BaseAxis.group without butchering things. Each axis would have to know where it fell in the list of all axes to figure out if it was the first or second instance of a particular orientation. Since an instance of _BaseAxis isn't aware of all axes involved in a chart, it seems like it has to involve a nasty hack. Do you have any advice?

[scanny]

I don't have my head freshly wrapped around this, so take that into account, but off-hand I'd say that .group would be a collection responsibility, most naturally belonging to _Axes. Usually I've accomplished this sort of thing by an axis, in this case, keeping a reference to the collection it belongs to, generally named ._parent

This would require that all axes be constructed by an _Axes object. That might not be a bad idea anyway.

So the operation might look vaguely like this:

class _BaseAxis(...):
    @property
    def group(self):
        return self._parent.axis_group(self)

class _Axes(...):
    def axis_group(axis):
        # work out which group this axis belongs to here
        return _AxisGroup(...)

The protocol would be reminiscent of how list.index(item) works in Python.

What do you think?

[AlexMooney]

Worked like a charm, thanks! That's committed but breaks most of the axis tests since making an axis has a different signature now. Should the tests be rewritten or should there be a default ._parent of None and error handling code around that?

To summarize, we now have Chart.axes, along with _BaseAxis.type, .orientation, and .group as well as _BaseAxis.orientation, since that's what's in the raw xml.

[scanny]

Hi Alex, sorry for the delay in responding, been a crazy week or so :)

In general, I'd say just update the tests using None for the parent parameter of the _BaseAxis() call or whatever, wherever that will work. I expect that will be most of them if not all.

If there are any that actually require the ._parent field, they'll need a suitable mock or perhaps a stub like a string or int, depending.

I haven't taken a look at them, but if you fix the ones where a None will get it done and still have any you need me to look at I'll be happy to. If you can send me a link to the right line(s) in the repo that will make it quick for me to review.

I've encountered this situation before as an object gets to the point where it needs to delegate something "upward" and it seems like almost all of the tests just need a working constructor call and not an actual parent. Makes sense of course since none of the prior tests could have used a parent that wasn't there and usually that field is only used by the new method you're adding :)

Interestingly, the practice of using None wherever possible is actually good testing practice as it provides some measure of assurance the code is not depending on a value you're not expecting it to :)

[AlexMooney]

Hi Steve, no worries!

None fixed all of the tests in the test_axis.py but there are a couple of failing tests in test_chart.py that I'm not sure how to deal with.

tests/chart/test_chart.py lines 34 and 45 are running CategoryAxis_.assert_called_once_with(catAx) and ValueAxis_.assert_called_once_with(valAx), respectively. Here's the error that the Category Axis test fails:

E           AssertionError: Expected call: CategoryAxis(<Element {http://schemas.openxmlformats.org/drawingml/2006/chart}catAx at 0x7f88e5a2f8e8>)
E           Actual call: CategoryAxis(<Element {http://schemas.openxmlformats.org/drawingml/2006/chart}catAx at 0x7f88e5a2f8e8>, <pptx.chart.chart._Axes object at 0x7f88e5a46d50>)

The cat_ax_fixture starts on line 110 , but I'm not sure how to modify it to work properly. I think like 112 is the problem.

The corresponding val_ax_fixture starts on line 205.

[scanny]

Ah, right, that makes sense.

So the first one you can fix by adding ', chart.axes' to line 34:

def it_provides_access_to_the_category_axis(self, cat_ax_fixture):
    chart, category_axis_, CategoryAxis_, catAx = cat_ax_fixture
    category_axis = chart.category_axis
    CategoryAxis_.assert_called_once_with(catAx, chart.axes)  ## <<< here
    assert category_axis is category_axis_

The same approach should fix the second test as well.

It points up an interesting question though, and that is whether the implementation of .category_axis and .value_axis are still appropriate when there is an _Axes object available.

I think before long we'll want an implementation more like this:

@property
def category_axis(self):
    """
    The first category axis of this chart. Raises |ValueError| if no
    category axes are defined.
    """
    try:
        return self.axes.list_by_type(MSO_AXIS_TYPE.CATEGORY)[0]
    except IndexError:
        raise ValueError('chart has no category axis')

but that can wait until after you get it working :)

Basically the idea being there shouldn't be more than one way to get the first category axis because it would represent duplication in the code. We would want the Axes object to be in charge of all axis getting and the .category_axis property of chart would delegate that job to its Axes object instead of handling it directly as it does today.

So we'll need to noodle a bit on what the broader API of Axes will be. The .list_by_type() method is just off the top of my head.

Anyway, we can get to all that in due time, step by step :)

[AlexMooney]

Thanks to that snippet, the tests are all passing again.

Instead of .list_by_type(), we could simply change the code to something like this:

@property
def category_axis(self):
    """
    The first category axis of this chart. Raises |ValueError| if no
    category axes are defined.
    """
    try:
        return next((ax for ax in chart.axes if ax.type == 'category'))
    except IndexError:
        raise ValueError('chart has no category axis')

It's really a matter of how useful the .list_by_type would be, since that's just as easily implemented in Chart.axes.

What do you think should be next?

[scanny]

Oooh, that's an idea. I think the except IndexError would have to become except StopIteration, but I really like the idea. We could just change it to that and then worry about any .list_by_type() method later when and if we needed it for something else. :)

[AlexMooney]

Changing the category and value axis methods (with except StopIteration, of course) causes failures in the same two tests.

tests/chart/test_chart.py:34: 
[...]
E           AssertionError: Expected to be called once. Called 0 times.

I don't suppose you know how to rewrite those tests off the top of your head..? 😕

[scanny]

Well, they would need to be rewritten. They use mocks so changing the interaction used to get the result will need a different test. A mocked test tests the interaction rather than the final result.

First, it would take two tests, one for the success case and one for the error case.

For the success case you need to mock the Chart.axes property to return an Axes mock, then have the Axes mock return a type of CATEGORY and have its __iter__() method return the right axis object.

I know that's a little bit tricky probably if you're not used to working with the Python mock library :)

Also I think there will need to be some sort of AXIS_TYPE enumeration, we wouldn't want to use a raw XML attribute string value in there.

It would be perfectly fine to leave it the way it was (calling CategoryAxis() directly) for now and we can get it upgraded when we actually close in on a set of commits. I'll leave that up to you. I think if we can get it working that's probably the first priority for most folks. We can attend to the careful craftsmanship of it as a second step. We might need more of my attention for that, but the actual incorporation of these into the main branch always seems to take a lot of that one way or the other :)

[zkn365]

Is the API for the second y axis avaiable now? I checked the chart and series source withtout clue.
Thanks.

[klucord]

Is this feature something that I can use? If so, is there anyway you could post an example of how to plot on the secondary axis or direct me to some documentation. Thank you!

[scanny]

No. This is a feature request. No one's working on it as far as I know. Looks like Alex made some progress on it a couple years ago. You can check out his fork, although it's 400-odd commits behind this master, so a lot of the latest features would be missing.

[anekix]

@scanny is there any work around to get two axis charts ? large part of our application depends upon python-pptx.i will be able to look into more detail of a permanent fix once i have some time off from work. Thanks

[scanny]

@anekix I don't know of any. There was some work done on this as you can see from the prior posts, so you might be able to pull from there. If your team has budget to devote to it you might consider sponsoring this feature.

[aishwarya91]

Hi @scanny

I have a simple problem , i am trying to add a reference line on the head of a bar keeping my value axis line same. For example

I want to add that blue line on the top of my reference bar.

[scanny]

@aishwarya91 Please post support questions on StackOverflow, using the 'python-pptx' tag.

[aishwarya91]

Hi

Have posted On StackOverflow.

Thanks

[TylerTCF]

I would like to try and workout a solution for the secondary axis on a simple combo chart. Is there any documentation you are aware of that could get me up to speed on:

1. the xml chart coding for the PPTX files
2. how I would add custom xml to the charts already generated by @scanny api

Sorry for a newbie dev post, but this feature would really help me out, so I want to try and work it out for a simple bar/line combo chart (bars on one axis, lines on other).

Thanks.

[scanny]

The analysis pages on Charts here are probably a good place to start:
https://python-pptx.readthedocs.io/en/latest/dev/analysis/index.html

[ownicn]

So, do you plan to implement it in the future ？

[scanny]

No current plans to implement.