systemfonts::system_fonts crashes (segfault) when used in future #41
Comments
|
macos. 10.14.6 |
|
Hmm... it seems to be happening deep inside CoreText so I'm unsure how to approach this. Maybe CoreText is simply not safe to run from forked processes. @HenrikBengtsson would you know anything about this? |
|
Just for the record, it does not seem to happen on linux. Seems to be a mac specific problem indeed. |
|
It also does not crash on mac if instead of multiprocess I use multisession. It does crash with multicore |
|
Thanks for confirming that this is indeed something to do it's calling into CoreText from forked processes. Really not sure what to do about this 😕 |
|
Does the following also segfault on macOS? > f <- parallel::mcparallel(systemfonts::system_fonts())
> v <- parallel::mccollect(f)If so, that's a minimal reproducible example using forked processing. I don't have access to macOS, so I cannot test. It works on Ubuntu. |
|
@HenrikBengtsson Crashes immediately after the first line is executed |
|
Interesting. Not that it would be a solution, but it would be interesting to know if preloading {systemfonts} as in: v0 <- systemfonts::system_fonts()
f <- parallel::mcparallel(systemfonts::system_fonts())
v <- parallel::mccollect(f)behaves differently, or possibly even work. |
|
@HenrikBengtsson yes, that works. Interesting! |
|
Yes, surely interesting - and one of the obscure properties of forked parallel processing in R (which certainly is not stable). If so, I guess that: loadNamespace("systemfonts")
f <- parallel::mcparallel(systemfonts::system_fonts())
v <- parallel::mccollect(f)might also work, because that also triggers: > systemfonts:::.onLoad
function (...)
{
.Call("sf_init_c", asNamespace("systemfonts"))
load_emoji_codes()
}One hypothesis could be that you trigger the necessary initialization in the main R session, which is then inherited by the forked child processes, so it just works there. In contrast, if you initialize in the child process, then that is lost when the fork is terminated leaving you with an object with broken references/pointers. That's just a guess. I would definitely not rely on this to work. It could just be that it happens to work in this simple example but if you "hit it hard enough" the problem will show up again elsewhere. The problem need to be well understood in order to consider it solved. |
|
systemfonts does some caching which means that the forked process inherits the cache and never call into CoreText. My hypothesis is still that it is simply not safe to call into CoreText from a forked process. There is nothing in the systemfonts code that should illicit this behaviour (I think) |
|
For the record, I tried to workaround the issue in my application by adding the systemfonts call earlier in the calling process, and I get a different segfault later |
|
Thanks for these follow-ups - they're helpful for this package but also as a reference/example to others elsewhere. @thomasp85, I don't know how it can be done, but if we could come up with a way for a package to detect when it runs in a forked process, then we might be able to give a run-time error rather than random crashes. The next level up would be to declare whether a package, or a specific function, can be forked or not, e.g. |
|
Oh, I forgot that we might be able to detect whether we're in a fork or not via |
|
I don't think it's metadata worthy. I suspect it's a bug somewhere, possibly even in the Core libraries. It feels kind of weird that you can't perform the operation in a forked process. After all, all processes are forked. |
|
I'd say it is "metadata worthy" because this is not the only case where code/packages/libraries cannot be forked. For example, there are various examples where multi-threading combined with forked processing wreak havoc. Forked processing is really unstable and I and others often advise against it because of the risk(*) that comes with it. Here is what the author of
(*) ... and maintenance costs - this issue being just one of many examples. |
|
If you add a metadata about it, it needs to be platform dependent. In this case, it works well on both linux and (probably) windows. It's just not working on windows, and metainfo cannot know how you are going to use the library you install anyway. As far as I am concerned, the main problem I am facing is that there's no other way to generate reports using flextable and officer under shiny, but to use a future with subprocess. That's how I found the issue. If you don't use a future, it will lock the shiny instance for all users (not only the current user, all of them). |
|
@stefanoborini the second segfault you reported doesn't seem to be related to systemfonts, correct? You can always use multisession to avoid forking, but I agree that this is all quite annoying |
|
@thomasp85 it was generated by a later call after I tried to force the initialisation as suggested by @HenrikBengtsson, but apparently it then fails later in other calls. I am not sure I can use multisession in shiny unfortunately, but I haven't checked, and I am not a big expert in R multithreaded execution (or a big expert in R, I'm mostly a python programmer) |
|
But the traceback does not seem to indicate that the segfault has anything to do with systemfonts, so maybe you stumbled on yet another issue with forking in another package...? |
Do we actually know that, or could it be that we just haven't hit the problem there?
Not sure why you can't user 'multisession', which parallelizes via R processes running in the background. It might be the firewall doesn't allow you to open ports needed to communicate with the workers. Regardless, you can also try: plan(future.callr::callr)which works very similarly to the multisession backend (but doesn't require ports). It's built on top of the callr package. PS. The term 'multi-threading' is different from 'multi-processing'. R does not support multi-threading, which takes place at a very low level orchestrated by the operating system. Formally, we say that threads belong to and can only run within a process. Almost all parallelization with do in R is done by running multiple processes. There many packages that implement multi-threading, e.g. data.table, but that is done in native code. |
|
@thomasp85 I don't know. I would have to do some more tracing, but since the call is about font metrics I suspect that it's using the same structures returned by systemfonts, and that's where it barfs. @HenrikBengtsson I tried on linux and I had no problem at all. It works well. I don't have a windows machine to test. So far, the only platform that seem to be affected is macos. I'll try more tomorrow, now it's late. Thank you both for your help. |
This code produces a hard segfault:
R 3.6.0, systemfonts 0.2.3 on macos.
lldb backtrace
The text was updated successfully, but these errors were encountered: