Flutter inline widget sizing

[matthew-carroll]

Super Editor would like users to be able to insert bitmaps, GIFs, and other visuals within text (#8).

These inline widgets should respect the baseline and line height of the surrounding text content.

Does Flutter support such behavior?

WidgetSpans

Flutter provides WidgetSpans, which display arbitrary widgets in the middle of text.

WidgetSpans, support baseline alignment.

However, WidgetSpans don't support line-height constraints. The WidgetSpan constructor doc says "Child widgets heights are unconstrained...". This behavior is confirmed by inspecting RenderParagraph, which is responsible for running layout for the inline widgets.

From RenderParagraph:

// Layout the child inline widgets. We then pass the dimensions of the
  // children to _textPainter so that appropriate placeholders can be inserted
  // into the LibTxt layout. This does not do anything if no inline widgets were
  // specified.
  List<PlaceholderDimensions> _layoutChildren(BoxConstraints constraints, {bool dry = false}) {
    if (childCount == 0) {
      return <PlaceholderDimensions>[];
    }
    RenderBox? child = firstChild;
    final List<PlaceholderDimensions> placeholderDimensions = List<PlaceholderDimensions>.filled(childCount, PlaceholderDimensions.empty);
    int childIndex = 0;
    // Only constrain the width to the maximum width of the paragraph.
    // Leave height unconstrained, which will overflow if expanded past.
    BoxConstraints boxConstraints = BoxConstraints(maxWidth: constraints.maxWidth);  // <-- UNCONSTRAINED HEIGHT
    // The content will be enlarged by textScaleFactor during painting phase.
    // We reduce constraints by textScaleFactor, so that the content will fit
    // into the box once it is enlarged.
    boxConstraints = boxConstraints / textScaleFactor;
    while (child != null) {
      double? baselineOffset;
      final Size childSize;
      if (!dry) {
        child.layout(
          boxConstraints,
          parentUsesSize: true,
        );
        childSize = child.size;
        switch (_placeholderSpans[childIndex].alignment) {
          case ui.PlaceholderAlignment.baseline:
            baselineOffset = child.getDistanceToBaseline(
              _placeholderSpans[childIndex].baseline!,
            );
            break;
          case ui.PlaceholderAlignment.aboveBaseline:
          case ui.PlaceholderAlignment.belowBaseline:
          case ui.PlaceholderAlignment.bottom:
          case ui.PlaceholderAlignment.middle:
          case ui.PlaceholderAlignment.top:
            baselineOffset = null;
            break;
        }
      } else {
        assert(_placeholderSpans[childIndex].alignment != ui.PlaceholderAlignment.baseline);
        childSize = child.getDryLayout(boxConstraints);
      }
      placeholderDimensions[childIndex] = PlaceholderDimensions(
        size: childSize,
        alignment: _placeholderSpans[childIndex].alignment,
        baseline: _placeholderSpans[childIndex].baseline,
        baselineOffset: baselineOffset,
      );
      child = childAfter(child);
      childIndex += 1;
    }
    return placeholderDimensions;
  }

Custom spans?

If WidgetSpans won't work, can we create our own specialized InlineSpan?

Super Editor could create a custom InlineSpan but it wouldn't have any effect on this problem. The root problem is RenderParagraph, which hard-codes a decision to leave all PlaceholderSpans vertically unconstrained.

Estimating height

Perhaps Super Editor could assign a height to a WidgetSpan with an approximation based on the incoming line height.

Unfortunately, it doesn't look like a given WidgetSpan is notified of the incoming/preceding text style. One would expect to receive that information in the WidgetSpan's build() method, but that information isn't provided.

Conclusion

Super Editor can't impose line-height restrictions on inline widgets without implementing an alternative to RenderParagraph.

Super Editor can't approximate a height by inspecting the preceding text style, because the preceding text style is not provided to a given WidgetSpan.

At this time, the only control over inline widget height looks to be an explicit height that's chosen by the widget that's being displayed.

[justinmc]

Would it be acceptable if the height constraint was the preferredLineHeight? It might be tough if you need the actual line height calculated after laying out all other content on the line. The size of the WidgetSpan itself would affect what ends up on each line.

[matthew-carroll]

I think the approach needs to make it possible to receive actual line height. It's true that the WidgetSpan might alter the line height. It might also alter the line break point. It may also want to see what's coming downstream, too.

Text layout use-cases can get very complicated, but these are the things that people do with text. So, as I always suggest, I think we need to embrace the yak shave on this and figure out how to support the gritty details. And all the options should be on the table. For example, maybe a single representation of text layout is insufficient. If you're worried about losing performance in the average case because of an elaborate protocol for the document case, then perhaps there should be an alternative to RenderParagraph that's aimed specifically at these use-cases. I can appreciate that a button label doesn't need to waste CPU cycles on this stuff.