Merge pull request #218 from sogaiu/process-mgmt

Add process management docs

content/docs/documentation.mdz

@@ -1,6 +1,6 @@
 {:title "Documentation"
  :template "docpage.html"
- :order 22}
+ :order 23}
  ---
 
 Documenting code is an important way to communicate to users how and when to use

content/docs/ffi.mdz

@@ -1,6 +1,6 @@
 {:title "Foreign Function Interface"
  :template "docpage.html"
- :order 25}
+ :order 26}
 ---
 
 Starting in version 1.23.0, Janet includes a Foreign Function Interface module on x86-64, non-Windows

content/docs/jpm.mdz

@@ -1,6 +1,6 @@
 {:title "jpm"
  :template "docpage.html"
- :order 23}
+ :order 24}
 ---
 
 JPM is a build tool that can be installed along with Janet to help build and install

content/docs/linting.mdz

@@ -1,6 +1,6 @@
 {:title "Linting"
  :template "docpage.html"
- :order 24}
+ :order 25}
 ---
 
 The @code`janet` binary comes with a built in linter as of version 1.16.1. This

content/docs/process_management/execute.mdz

@@ -0,0 +1,97 @@
+{:title "Executing a Process"
+ :nav-title "Executing"
+ :template "docpage.html"}
+---
+
+In the simpler approach using @code`os/execute`, once the program is
+started to create a subprocess, the function does not return or error
+until the subprocess has finished executing.  Technically,
+@code`os/execute` \"waits" on the created subprocess.  Further details
+regarding waiting will be covered when discussing @code`os/spawn`.
+The return value for @code`os/execute` is the exit code of the created
+subprocess.
+
+### @code`args`
+
+The only required argument, @code`args`, is a tuple or array of
+strings that represents an invocation of a program along with its
+arguments.
+
+@codeblock[janet]```
+# prints: I drank what?
+# also returns 0, the exit code
+(os/execute ["janet" "-e" `(print "I drank what?")`] :p) # => 0
+```
+
+### @code`flags`
+
+If there is a second argument, @code`flags`, it should be a keyword.
+@code`flags` can affect program launching and subprocess execution
+characteristics.
+
+Note in the example above, @code`flags` is @code`:p`, which allows
+searching the current @code`PATH` for a binary to execute.  If
+@code`flags` does not contain @code`p`, the name of the program must
+be an absolute path.
+
+If @code`flags` contains @code`x`, @code`os/execute` will raise an
+error if the subprocess' exit code is non-zero.
+
+@codeblock[janet]```
+# prints: non-zero exit code
+(try
+  (os/execute ["janet" "-e" `(os/exit 1)`] :px)
+  ([_]
+    (eprint "non-zero exit code")))
+```
+
+### @code`env`
+
+If @code`flags` contains @code`e`, @code`os/execute` should take a
+third argument, @code`env`, a dictionary (i.e. table or struct),
+mapping environment variable names to values.  The subprocess will be
+started using an environment constructed from @code`env`.  If
+@code`flags` does not contain @code`e`, the current environment is
+inherited.
+
+@codeblock[janet]```
+# prints "SITTING"
+# also returns 0
+(os/execute ["janet" "-e" `(pp (os/getenv "POSE"))`]
+            :pe {"POSE" "SITTING"}) # => 0
+```
+
+The @code`env` dictionary can also contain the keys @code`:in`,
+@code`:out`, and @code`:err`, which allow redirecting standard IO in
+the subprocess.  The associated values for these keys should be
+@code`core/file` values and these should be closed explicitly (e.g. by
+calling @code`file/close`) after the subprocess has completed.
+
+Note that the @code`flags` keyword argument only needs to contain
+@code`e` if making use of environment variables.  That is, for use of
+any combination of just @code`:in`, @code`:out`, and @code`:err`, the
+@code`flags` keyword does not need to contain @code`e`.
+
+@codeblock[janet]```
+(def of (file/temp))
+
+(os/execute ["janet" "-e" `(print "tada!")`]
+            :p {:out of}) # => 0
+
+(file/seek of :set 0)
+
+(file/read of :all) # => @"tada!\n"
+
+(file/close of) # => nil
+```
+
+### Caveat
+
+Although it may appear to be possible to successfully use
+@code`core/stream` values with @code`os/execute` in some cases, it may
+not work in certain situations (e.g. with another operating system,
+different programs, varied arguments, phase of the moon, etc.).  It is
+not a reliable choice and is thus not recommended.  The
+@code`os/spawn` function is better suited for use with
+@code`core/stream` values.
+

content/docs/process_management/index.mdz

@@ -0,0 +1,14 @@
+{:title "Process Management"
+ :template "docpage.html"
+ :order 22}
+---
+
+Janet has support for starting, communicating with, and otherwise
+managing processes primarily via two approaches.  One is a simpler
+synchronous method provided by @code`os/execute`.  Another is a more
+complex way that gives finer-grained control via @code`os/spawn` and
+some other @code`os/` functions.
+
+The @code`os/execute` function will be covered first as it is simpler
+but also because @code`os/spawn` may be easier to understand once one
+has a sense of @code`os/execute`.

content/docs/process_management/spawn.mdz

@@ -0,0 +1,243 @@
+{:title "Spawning a Process"
+ :nav-title "Spawning"
+ :template "docpage.html"}
+---
+
+Unlike @code`os/execute`, which returns an exit code (or errors) after
+the started subprocess completes, @code`os/spawn` returns a
+@code`core/process` value.  This value represents a subprocess which
+may or may not have completed its execution.  In contrast to
+@code`os/execute`, which "waits" for the subprocess, @code`os/spawn`
+does not and it is recommended to explicitly arrange for this
+"waiting" for resource management and other purposes ("waiting" will
+be covered in more detail below).
+
+The arguments for @code`os/spawn` are the same as those of
+@code`os/execute`.  Again, the required argument, @code`args`, is a
+tuple or array representing an invocation of a program with its
+arguments.
+
+@codeblock[janet]```
+# prints: :salutations
+# the returned value, proc, is a core/process value
+(let [proc (os/spawn ["janet" "-e" "(pp :salutations)"] :p)]
+  (type proc)) # => :core/process
+```
+
+## @code`core/process` and @code`os/proc-wait`
+
+Passing the @code`core/process` value to the @code`os/proc-wait`
+function causes Janet to "wait" for the subprocess to complete
+execution.  This is sometimes referred to as "rejoining" the
+subprocess.
+
+The main reason for "waiting" is so that the operating system can
+release resources.  Not "waiting" appropriately can lead to resources
+remaining unreclaimed and this can become a problem for a running
+system because it may run out of usable resources.
+
+Once "waiting" has completed, the exit code for the subprocess can be
+obtained via the @code`:return-code` key.  Accessing this key before
+"waiting" will result in a @code`nil` value.
+
+@codeblock[janet]```
+# prints: :relax
+(def proc (os/spawn ["janet" "-e" "(pp :relax)"] :p))
+
+(get proc :return-code) # => nil
+
+(os/proc-wait proc) # => 0
+
+(get proc :return-code) # => 0
+```
+
+The @code`os/proc-wait` function takes a single argument, @code`proc`,
+which should be a @code`core/process` value.  The return value is the
+exit code of the subprocess.  If @code`os/proc-wait` is called more
+than once for the same @code`core/process` value it will error.
+
+@codeblock[janet]```
+# prints: :sit-up
+(def proc (os/spawn ["janet" "-e" "(pp :sit-up)"] :p))
+
+(os/proc-wait proc) # => 0
+
+# prints: cannot wait twice on a process
+(try
+  (os/proc-wait proc)
+  ([e]
+    (eprint e))) # => nil
+```
+
+Note that the first call to @code`os/proc-wait` will cause the current
+fiber to pause execution until the subprocess completes.
+
+## @code`core/process` keys and the @code`env` argument
+
+As seen above, once "waiting" for a subprocess has completed, the exit
+code of a subprocess becomes available via the @code`:return-code`
+key.  Other information about a subprocess can also be accessed via
+the keys of an associated @code`core/process` value such as
+@code`:in`, @code`:out`, and @code`:err`.  On UNIX-like platforms,
+there is an additional @code`:pid` key.
+
+The @code`:in`, @code`:out`, and @code`:err` keys for a
+@code`core/process` value provide access to the standard input, output,
+and error of the associated subprocess provided that corresponding
+choices were made via the @code`env` dictionary (i.e. table or struct)
+argument to @code`os/spawn`.  This means that, for example, if there
+is no key-value pair specified via @code`:in` in the @code`env`
+dictionary, then the standard input of the subprocess will not be
+programmatically accessible via the associated @code`core/process`'s
+@code`:in` key.
+
+The values associated with the @code`:in`, @code`:out`, and
+@code`:err` keys of the @code`env` argument can be @code`core/file` or
+@code`core/stream` values.
+
+@codeblock[janet]```
+(def out-file (file/temp))
+
+(type out-file) # => :core/file
+
+(def proc
+  (os/spawn ["janet" "-e" `(print "again") (flush)`]
+            :p {:out out-file}))
+
+# only standard output of the subprocess is accessible
+(proc :in) # => nil
+(type (proc :out)) # => :core/stream
+(proc :err) # => nil
+
+(os/proc-wait proc) # => 0
+
+(file/seek out-file :set 0)
+
+# prints: again
+(print (file/read out-file :all))
+
+(file/close out-file)
+```
+
+Note that in the example above, the standard input and error of the
+subprocess were not accessible -- attempts to access them resulted in
+@code`nil` values -- because corresponding keys for the @code`env`
+argument were not specified.  In contrast, because a key-value pair
+for @code`:out` and @code`out-file` were included in the @code`env`
+struct, the standard output of the subprocess could be accessed
+programmatically.
+
+## @code`:pipe` and @code`ev/gather`
+
+For each of the @code`:in`, @code`:out`, and @code`:err` keys of the
+@code`env` argument to @code`os/spawn`, the associated value can be
+the keyword @code`:pipe`.  This causes @code`core/stream` values to be
+used that can be read from and written to for appropriate standard IO
+of the subprocess.
+
+In order to avoid deadlocks (aka hanging), it's important for data to
+be transferred between processes in a timely manner.  One example of
+how this can fail is if output redirected to pipes is not read from
+(sufficiently).  In such a situation, pipe buffers might become full
+and this can prevent a process from completing its writing.  That in
+turn can result in the writing process being unable to finish
+executing.  To keep data flowing appropriately, apply the
+@code`ev/gather` macro.
+
+In the following example, @code`ev/write` and @code`os/proc-wait` are
+both run in parallel via @code`ev/gather`.  The @code`ev/gather` macro
+runs a number of fibers in parallel (in this case, two) on the event
+loop and returns the gathered results in an array.
+
+@codeblock[janet]```
+(def proc
+  (os/spawn ["janet" "-e" "(print (file/read stdin :all))"]
+            :p {:in :pipe}))
+
+(ev/gather
+  (do
+    # leads to printing via subprocess: know thyself
+    (ev/write (proc :in) "know thyself")
+    (ev/close (proc :in))
+    :done)
+  (os/proc-wait proc)) # => @[:done 0]
+```
+
+## @code`os/proc-close`
+
+The next example is a slightly modified version of the previous one.
+It involves adding a third fiber (to @code`ev/gather`) for reading
+standard output from the subprocess via @code`ev/read`.
+
+@codeblock[janet]```
+(def proc
+  (os/spawn ["janet" "-e" "(print (file/read stdin :all))"]
+            :p {:in :pipe :out :pipe}))
+
+(def buf @"")
+
+(ev/gather
+  (do
+    (ev/write (proc :in) "know thyself")
+    (ev/close (proc :in))
+    :write-done)
+  # hi! i'm new here :)
+  (do
+    (ev/read (proc :out) :all buf)
+    :read-done)
+  (os/proc-wait proc)) # => @[:write-done :read-done 0]
+
+(os/proc-close proc) # => nil
+
+buf # => @"know thyself\n"
+```
+
+Beware that if pipe streams are not closed soon enough, the process
+that created them (e.g. the @code`janet` executable) may run out of
+file descriptors or handles due to process limits enforced by an
+operating system.
+
+Note the use of the function @code`os/proc-close` in the code above.
+This function takes a @code`core/process` value and closes all of the
+unclosed pipes that were created for it via @code`os/spawn`.  In the
+example above, @code`(proc :in)` had already been closed explicitly,
+but @code`(proc :out)` had not and is thus closed by
+@code`os/proc-close`.
+
+The @code`os/proc-close` function will also attempt to wait on the
+subprocess associated with the passed @code`core/process` value if it
+has not been waited on already.  The function's return value is
+@code`nil` if waiting was not attempted and otherwise it is the exit
+code of the subprocess corresponding to the @code`core/process` value.
+
+## Effects of @code`:x`
+
+If the @code`flag` keyword argument contains @code`x` for an
+invocation of @code`os/spawn` and the exit code of the subprocess is
+non-zero, calling a function such as @code`os/proc-wait`,
+@code`os/proc-close`, and @code`os/proc-kill` \(if invoked to wait)
+with the subprocess' @code`core/process` value results in an error.
+
+@codeblock[janet]```
+(def proc
+  (os/spawn ["janet" "-e" "(os/exit 1)"] :px))
+
+(defn invoke-an-error-raiser
+  [proc]
+  (def zero-one-or-two
+    (-> (os/cryptorand 8)
+        math/rng
+        (math/rng-int 3)))
+  # any of the following should raise an error
+  (case zero-one-or-two
+    0 (os/proc-wait proc)
+    1 (os/proc-close proc)
+    2 (os/proc-kill proc true)))
+
+# prints: error
+(try
+  (invoke-an-error-raiser proc)
+  ([_]
+    (eprint "error")))
+```
+