Providing visual bouds of SVG content

[dirkschulze]

For some use cases it is required to provide the visual bounds of the SVG content so that a big-enough image buffer can get created before rendering the SVG content.

Skia, CG and Cairo Graphics do support mechanisms to compute the visual bounds of the content. However, for some it might be required to traverse the entire tree while others provide a "recording" mechanism.

Cairo Graphics and Skia (example 1, API) do have a "recording mechanism". It means that drawing commands get "recorded" and "replayed" on another rendering context.
Both seem to have the possibility to provide the "fill bounds", the visual area.

We should consider computing at least a rough estimation of the bounds as well. The rendering port would at least (minimum requirement) need to provide the tight bounding box of paths. In this case we traverse the rendering tree just like we would for actual rendering. However, instead of painting we would simply compute the union of the bounding box of each path including stroke and excluding the clipping area.

[dirkschulze]

@moazin @mpsuzuki any thoughts? Would you rather want to rely on recording for Skia and Cairo or let SVG Native Viewer compute the bounds? The latter might be more expensive with regards of CPU cycles. But it requires performance testing.

Note that I haven't tested SkPicture_Recorder yet.

[moazin]

@dirkschulze A rough estimation of the bound shall suffice. The method FreeType uses for traditional glyphs also calculates a rough estimation because it's faster and almost always very close to the actual bound.

As I understand it, relying on Cairo/Skia for the bound calculation will limit the users of FreeType to compile svg-native-viewer with only Cairo/Skia ports and the feature becomes unavailable if the user doesn't want to compile with these backends. @lemzwerg How could this affect the users of FreeType (OT-SVG)?

[lemzwerg]

Sorry for being so late in replying. I'm slowly recovering from a conference, a severe low back pain, teaching, and concerts...

I prefer if svgnative is really stand-alone with everything to properly provide all data necessary for SVG rendering, independent of the backend. So yes, I favour a built-in recording mechanism.

[moazin]

@dirkschulze Any update on this?

[apodtele]

The true tight bounding box might be very hard to compute short of actual rendering. Instead, it should be rather straightforward to compute the control box which encapsulates all control points for all paths, perhaps clipped using clipPath control box as well. This is just something quick and dirty.

[moazin]

@dirkschulze I'm trying to work on this myself and I'm thinking to do it in a port-agnostic manner using @apodtele's approach. At the moment, I'm experimenting with how one can create a PathRecorder interface between SVGDocumentImpl and a Path instance to record all the draw calls and based on those create a bounding box which you can later apply a transform on or adjust for stroke if needed. Ultimately, one can sum these bounding boxes to find the bounding box of the whole document.

If you have ideas/suggestions/comments, I'd love to hear them. :-)

[dirkschulze]

@moazin All graphical implementations have a way to "record" paths. And svg-native-viewer has this as a requirement already kind of. I would prefer if we could use the native support of each port so that we can re-use the recorded paths for rendering right away.

Though I am more sceptical of just using the tight (transformed and clipped) bounding box. More often than not, it would provide a too-small box. Getting the stroke bounding box (even a rough estimation that isn't too far off from the tight stroke bounding box) is not trivial.

The idea of a rendering recorder (based on native support by rendering ports) would provide the best results and probably still be performant enough (due to display lists or other internal optimisations).

[moazin]

@dirkschulze While I've used the mechanism for recording and bound retrieval in Cairo already and they work great, I just built Skia on my machine and tried using the recorder mechanism and it doesn't seem to provide the "fill" bound. The steps I followed were:

1. Create an instance of SkPictureRecorder.
2. Begin recording and get the canvas by skPictureRecorder->beginRecording(...).
3. Draw whatever we want.
4. Get the drawable by skPictureRecorder->finishRecordingAsDrawable().
5. drawable->getBounds()
   The resultant bounds just seem to be the dimensions that I pass when calling beginRecording.

I was wondering if you might know the correct way to do this?
@mpsuzuki , any experience with this?

[dirkschulze]

@schenney-chromium @CoreyDotCom would you know how to get the visual bounds from an SkPictureRecorder? Do you have an idea who to ask?

[CoreyDotCom]

Not off the top of my head but this thread might help - e.g. record with an RTree and then query the results post record. https://bugs.chromium.org/p/skia/issues/detail?id=5974 ... otherwise you can inquire directly with the team here via the skia google discussion group. https://groups.google.com/g/skia-discuss.

[schenney-chromium]

I agree that skia-discuss would be the best place to ask. I believe the point of passing in the bounds to the SkPictureRecorder is explicitly to limit the drawable size. Tracking and unioning the bounds as you add things to the recorder might be the best approach to get the actual bounds of the content you are drawing, but that won't change the bounds in the the recorder.

Regarding fill, there is code in Blink (chromium's SVG rendering implementation) that computes bounds to answer Javascript API queries for bounds and compute visual overflow. That must include fill somehow thought I don't recall the details. Blink implicitly does the "union bounds of all drawn content" well before it creates any actual recording of the content.

[CoreyDotCom]

To clarify you need to provide an RTree factory that starts out with inflated bounds and they will ultimately be deflated, here is cookbook code. https://github.com/google/skia/blob/375e1f6a6486a1e423f61d221bc39d81a2aaf6a0/tests/PictureBBHTest.cpp#L97

[dirkschulze]

Thanks a lot @schenney-chromium and @CoreyDotCom! @moazin do you think you have enough info to experiment?

[sairuspatel]

Note that the font bounding box (including the SVG glyphs) is in the head table: https://docs.microsoft.com/en-us/typography/opentype/spec/head#xMin.

It sounds like a rough estimation of the glyph’s bbox will suffice for this issue, at least for the use case where performance is critical. Thus, one or more “sides” of the font’s bbox could in some cases be consulted, when doing the quick-and-dirty calculation for a glyph bbox since the font’s bbox will provide the overall bounds beyond which no glyph (including any stroking) should extend.

Just an idea to add to the mix.

[moazin]

@dirkschulze Yes, this should be enough for me to start experimenting. Thank you so much @schenney-chromium @CoreyDotCom.

@sairuspatel Thanks for chiming in. So, this "global" bbox in the head table is basically the union of all of the individual bounding boxes of glyphs and what you're basically saying is that this can provide a lenient bound that can be shrunk to find the "tight" bounding box of any particular SVG glyph?

[lemzwerg]

Yes, the bbox in the head table is ideally the union of all glyph bboxes. Note, however, that this value might be inaccurate (or even zero) for older fonts.

@sairuspatel, are you sure that the head value table also covers SVG glyphs? I don't see this mentioned in the OpenType spec. Can you post a reference?

[apodtele]

It cuts both ways too. Should you rely on global bbox, you must clip your glyphs just in case. Nobody does that. The global bbox is just auxiliary information.

[sairuspatel]

@lemzwerg https://docs.microsoft.com/en-us/typography/opentype/spec/head#xMin says for "all" glyphs (except those without contours). In the OT spec process, clarifying this was considered but the "all" part was seen as sufficiently clear. The only wrinkle in this one could imagine is animated SVG glyphs, but SVG Native Viewer by definition doesn't support animation.

Yes the font bbox value could be incorrectly set (hopefully unlikely, since OT-SVG fonts have been produced only in the past decade by a small number of tools), but yes like with any other value read from the font, code should be defensive and guard against bad things like crashes happening. If clipping happens because the font bbox is incorrectly set, that's acceptable IMO. The font is badly made and should be fixed.

(As a side note, I wonder whether those glyph bboxes recorded in the 'glyf' table are seen as trustworthy, or if font engines tend to ignore them and simply compute the boxes as they have to do with CFF.)

@moazin Yes, that's correct. And in the cases where only rough glyph bboxes are needed (to prevent clipping, say), perhaps the font bbox itself could be used if it's within some reasonable size, in those situations where computing the glyph's bounds is too onerous. The font bbox is there and we should feel free to rely upon it when appropriate, is my main point here.

[apodtele]

People do creative stuff with fonts. Some even want to pre-allocate memory for a whole atlas texture in graphics memory before the glyphs are directly rendered there without copying. They would really appreciate a quick but reasonable bbox estimate. It is a good thing to afford them this facility.

Emojis might be roughly the same size, but 'I' and 'W' are vastly not the same.

[moazin]

@dirkschulze I'd like to move forward with this and proceed towards submitting a PR, maybe just a draft PR for now but one that is clean and can be merged soon.

The approach that I've decided on is to rely on the port specific functions to retrieve the fill bounding box of each individual paths and doing a union on them to get the bounding box for the whole document. Of course when there is a stroke present, I use the corresponding functions to get the stroke bounding box too and I take care of clipping and transforms as well.

While ultimately we should support the bounding calculation feature for all four ports and even the upcoming D2D one, I wonder if we can for now just proceed with the Skia port and keep the rest unimplemented (by an exception or assertion perhaps).

For the API design, I'm thinking to have a simple Bounds function that has been overloaded with two implementations. One that takes in an id and the other one that doesn't. The Bounds function doesn't care about the ViewBox or any transformations applied on the Canvas, all of those are simply ignored. It'll simply return the bounding box in the SVG coordinates.

[HinTak]

@dirkschulze While I've used the mechanism for recording and bound retrieval in Cairo already and they work great, I just built Skia on my machine and tried using the recorder mechanism and it doesn't seem to provide the "fill" bound. The steps I followed were:

1. Create an instance of SkPictureRecorder.
2. Begin recording and get the canvas by skPictureRecorder->beginRecording(...).
3. Draw whatever we want.
4. Get the drawable by skPictureRecorder->finishRecordingAsDrawable().
5. drawable->getBounds()
   The resultant bounds just seem to be the dimensions that I pass when calling beginRecording.

I was wondering if you might know the correct way to do this? @mpsuzuki , any experience with this?

Just reading past issues - skPictureRecorder->finishRecordingAsPicture()->cullRect() calculates the bounds correctly. It is used that way in the skia-port.c in my freetype svghook.

[HinTak]

The oldest skia I looked at was m87 (ported skia-python from m87 to m88, first one when SVGDOM not experimental), and mostly used m110 binary with main - m116 source code + history as reference. So it might be a lot better than you guys were using -m80(?)-ish?The bound calculation with skia's SkPictureRecorder works quite well and possibly faster than cairo's recording surface, too.