Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inline Images in Text Layouts? #490

Open
terhechte opened this issue Jan 30, 2022 · 4 comments
Open

Inline Images in Text Layouts? #490

terhechte opened this issue Jan 30, 2022 · 4 comments
Labels
enhancement New feature or request text

Comments

@terhechte
Copy link

I was wondering if there was interest in having inline images support. I think it would be reasonably easy for the CoreGraphics backend by adding a NSTextAttachment with the attachment property.

However, I don't know how difficult this would be on the other platforms. I did a brief search for Cairo and found this. However, as with the blurred rectangles maybe it is fine to start just with one (well, two if counting iOS) platform(s) here?

@cmyr
Copy link
Member

cmyr commented Jan 30, 2022

My sense is that this is out of scope for the time being, as doing it correctly would require us to basically implement our own run-based layout API, which is a lot of work.

I think of the current version of piet as "good enough", in that it can handle the 2D drawing requirements of most simple applications, and once we get into fancier stuff then the capabilities and API of the underlying platforms start diverging rapidly, and there is diminished return on our work. In the long term, our vision for cross-platform 2D graphics is a single unified API that talks directly to the GPU, and avoids intermediate platform API completely; this would give us control over the API we could expose, without worrying about the particulars of the platform.

@raphlinus
Copy link
Contributor

I definitely would like to have a way to do this, but to say it's a complex topic is an understatement. Is it just images, to be drawn when the text layout is drawn, or is it a more general mechanism, for example analogous to replacement spans in Android? Does it allow for animation of the drawn graphic? At some point, you'll also want more sophisticated layout than just inline, for example floats.

As Colin says, one way to address this is to expose a lower level API at the piet level, so the upper layer can draw the runs and the graphics wherever they like. That was originally the idea of skribo, but now I'm not so sure; there are a lot of complexities, and platform text drawing APIs are not designed around to expose this. We will probably be moving away from platform text and toward parley, so it might be worth addressing at that level.

If I had to do something right now, it'd probably be introducing a very simple replacement span concept, where essentially just a box is provided, which would take space in the layout, then the client would be responsible for drawing the contents of the box.

@terhechte
Copy link
Author

Thanks for the insightful responses. Just for context, I'm using piet in a new project where (in the long run) I'd need inline images support - if the project bears fruit. Hence my interest in the topics. I don't need it right now though, so I might just as well wait for Piet-gpu. If the need arises I might just fork Piet for the time being and add rudimentary, temporary support to the CoreGraphics backend.

@cmyr
Copy link
Member

cmyr commented Feb 3, 2022

I think that a coregraphics-specific extension would be the right choice. I also think that the piet API should be quite stable for the foreseeable future, so maintaining a fork shouldn't be that hard.

@xStrom xStrom added enhancement New feature or request text labels Jun 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request text
Projects
None yet
Development

No branches or pull requests

4 participants