Improve dataflow arrows

[AndreVanDelft]

Data flow arrows ~~> and ~/~> (for Throwables) are rather new in SubScript.
A recent use case revealed that the operator priority needs to change:

query.findInBackground 

~~task:Task[List[ParseObject]]~~> 

for (post <- task.getResult) 
& (  post.put("published", true)
     post.saveInBackground
  )

With the current high priority the RHS of the arrow would be the for-construct, but it should extend to the end of the code fragment. It seems that the arrow's operator priority should be very low; maybe the lowest possible priority, lower even than the newl line. At least it should be lower than the priority of ==.

Another issue: currently there is one main dataflow operator: `x ~~> y /> z', where either the left or right arrow is optional. This is a bit ambiguous, as the following expressions are all different:

x ~~> y ~/~> z
x ~~> (y ~/~> z)
(x ~~> y) ~/~> z

Moreover we would like to be able to chain such expressions without worrying.
E.g.,

`x1 ~~> x2 ~~> x3 ~/~> z'

should be equivalent to

`((x1 ~~> x2) ~~> x3) ~/~> z'

i.e. a dataflow pipe with z as fallback in case the pipe breaks down somewhere. Then we must have different syntax for the ternary use coming down to do x then y else z.

Onother idea of specifying parameters inside the arrows has led to another idea: multiple alternatives, using extra arrows that start with +, as in:

x  ~~p1:P1~~> y1
+~~p2:P2~~> y2           

Now we can use the same plus symbol for the ternary dataflow operator: `x ~~> y +/> z'.

In general the dataflow operator looks like:

x ~~a1:A1~~> y1
 +~~a2:A2~~> y2
 +~~a3:A3~~> y3
+~/~e1:E1~~> z1
+~/~e2:E2~~> z2
+~/~e3:E3~~> z3

Then this would translate to

x ~~>  PartialFunction {
                    case a1:A1 ==> y1
                    case a2:A2 ==> y2
                    case a3:A3 ==> y3
                    case _ ==> {fail(new MatchException(a))}
             }
 +~/~>  PartialFunction {
                  case e1:E1 ==> z1
                  case e2:E2 ==> z2
                  case e3:E3 ==> z3
                  case _ :Throwable  ==> {fail(e)}
            }

This transformation would be done by the compiler. It also translates the arrows into call scripts in DSL.scala by the names of _dataflow_then_else, _dataflow_then, _dataflow_else.

The PartialFunction on the RHS is in fact a partial script; its treatment would be much like the current treatment of partial functions that handle incoming actor messages.

Note that the result of the partial script should be the result of the chosen branch. And other rules are:

• The result of x ~~> y +~/~> z is the result of y or z, whenever either one of these two succeeds or fails (note that y may be chosen multiple times, and z at most once).
• The result of x ~~> y is set to the result of y when that succeeds or fails, or to the result of x when that fails.
• The result of x ~/~> z is set to the result of x when this succeeds or of z when that succeeds or fails.

Most of this work seems relatively simple; I am not sure about the last parts (translating match expressions and setting the result value).

The gain would be big: dataflow operators would work almost completely (except for some compile time type checking), and lots of nice use cases will benefit, such as the Twitter Search example.

[AndreVanDelft]

Later we may increase the expressiveness for the arrows holding parameters. Since these are translated into cases in a match statements, all things should be allowed that are normally allowed between the word case and =>. In particular: extractions, the bar symbol |, and conditions.
Simple parameters may be enclosed in parentheses, other must be enclosed.

E.g.,

x ~~(1 | 2) ~> y1
 +~~(i:Int if i>2) ~> y2

[AndreVanDelft]

When inspecting the parser code I noticed that the if and do constructs have been programmed wrong. It is currently possible to write a + if c then b whereas that should not be acceptable; a + (if c then b) is ok.

[AndreVanDelft]

Re: Priorities.
I tried the Capturing Variables use case from Bolts Framework. This is the code that I would like to be able to write:

var successfulSaveCount = (val successfulSaveCount = 0
      saveAsync(obj1) ~~~> let successfulSaveCount += 1
      saveAsync(obj2) ~~~> let successfulSaveCount += 1
      saveAsync(obj3) ~~~> let successfulSaveCount += 1
      saveAsync(obj4) ~~~> let successfulSaveCount += 1
                     ~~/~> println:"error"
      )
      println: "done"

So, just like in the first use case, at the top of this issue, the arrows should have very low priority.
However, I then remembered the Exit process use case:

doExit = exitCommand areYouSure ~(ok:Boolean)~> while:!ok

or

doExit = exitCommand areYouSure ~~> while:!_

Here the dataflow arrow must have a very high priority; higher than the white space operator, since the arrow should be here an operand of the white space operator.

In fact, it seems that we could well use 2 variations of the data flow arrows, like with sequence operators: long arrows having low priority (even lower than new lines), and short arrows with high priority (higher than white space).
If ever one of these appears superfluous we can always drop it.

[AndreVanDelft]

RE: the first use case in this thread.
The complete SubScript code was:

val query = ParseQuery.getQuery("Post")
query.whereNotEqualTo("published", true)
query.findInBackground 

~~~task:Task[List[ParseObject]]~~~> 

for (post <- task.getResult) 
& (  post.put("published", true)
     post.saveInBackground
)

@Gui: mPostStatusTextView.setText(String.format("All Posts Published!")) 

The problem is the structure of the LHS of the arrow: a sequence; we could define that the result value by default of such a sequence is the result of the last element, and then this should be propagated to the lambda that is implicitly around this sequence, passed as a parameter to the method that is generated for the arrow.
Maybe that is a bit too arbitrary. Another option would be to say that a script's result value is by default set by any leaf node that is known to have a result value (as opposed to Unit); this may be overridden by explicitly appending a caret (^) to such an operand.