mouse shake events

[ghost]

hi Tomas, does reactFX support or enable detecting mouse shake, and press-and-shake events on FX nodes ? I haven't made myself well aquanted with the library as of yet, but wanted to check if this is doable. if so, where should I start-- how can one go about describing such an event.

[TomasMikula]

Hi Maher,

as you probably know, "shake" is not a primitive event in JavaFX. However, it is not too difficult to write a method to recognize such gestures. A classical approach to recognize a pattern in a stream of events is to define a state machine: 1. a data-type representing the state you keep track of and 2. state transition function (that updates the state when an event arrives). ReactFX lets you define a state machine. Below is a sample application that recognizes mouse shakes.

I decomposed the problem into two subproblems:

1. recognize "mouse turns", i.e. when the mouse changes its horizontal direction. This state machine operates on a stream of MOUSE_MOVED events and emits an event whenever it recognizes a "turn". The mouse x-coordinate and timestamp is part of the emitted event.
2. recognize shakes. This state machine operates on the stream of mouse turns and when enough turns occur, each within a certain delay and distance from the previous, an event is emitted.

To recognize press-and-shake events, you would change the MOUSE_MOVED event to MOUSE_DRAGGED.

import static org.reactfx.util.Tuples.*;

import java.util.Optional;

import javafx.application.Application;
import javafx.scene.Node;
import javafx.scene.Scene;
import javafx.scene.input.MouseEvent;
import javafx.scene.layout.StackPane;
import javafx.stage.Stage;

import org.reactfx.EventStream;
import org.reactfx.EventStreams;
import org.reactfx.StateMachine;
import org.reactfx.util.Tuple2;

public class MouseShakeDemo extends Application {

    private static enum Dir { LEFT, RIGHT }

    public static void main(String[] args) {
        launch(args);
    }

    @Override
    public void start(Stage stage) {
        StackPane pane = new StackPane();

        shakes(pane, 4, 100, 200).subscribe(shake -> System.out.println("SHAKE!!!"));

        stage.setScene(new Scene(pane, 400, 400));
        stage.show();
    }

    /**
     * Emits an event when a mouse shake occurs on the given node.
     * @param node
     * @param n The number of mouse turns that need to occur before recognize the gesture.
     * Must be at least 2.
     * @param maxDist maximum distance, in pixels, that the mouse has to travel between turns
     * @param maxDelay maximum delay, in milliseconds, between turns
     */
    private EventStream<?> shakes(Node node, int n, double maxDist, long maxDelay) {
        // we're going to track the number of turns and the last turn
        // as the state of a state machine
        return StateMachine.init(t(0, (Tuple2<Double, Long>) null)) // start with no turns
                .on(mouseTurns(node))
                .transmit((state, turn) -> state.<Tuple2<Tuple2<Integer, Tuple2<Double, Long>>, Optional<Object>>>map((cnt, lastTurn) -> {
                    if(lastTurn == null) {
                         // track the first turn, don't emit anything
                         return t(t(1, turn), Optional.empty());
                    } else {
                        double dist = Math.abs(turn._1 - lastTurn._1);
                        long delay = turn._2 - lastTurn._2;
                        if(dist <= maxDist && delay <= maxDelay) { // we are within the allowed delay and distance
                            if(cnt + 1 == n) { // reached the required number of turns
                                // emit an event and start counting from 0
                                return t(t(0, null), Optional.of(turn));
                            } else {
                                // increase the counter, but don't emit yet
                                return t(t(cnt + 1, turn), Optional.empty());
                            }
                        } else { // too much delay or too big distance, start counting from one
                            return t(t(1, turn), Optional.empty());
                        }
                    }
                }))
                .toEventStream();
    }

    // emits x-coordinate and timestamp (in milliseconds) of mouse turns
    private EventStream<Tuple2<Double, Long>> mouseTurns(Node node) {
        // we're going to track last mouse position and direction
        // as the state of a state machine
        return StateMachine.init(t((Double) null, (Dir) null)) // start with no value
                .on(EventStreams.eventsOf(node, MouseEvent.MOUSE_MOVED))
                .transmit((state, evt) -> state.<Tuple2<Tuple2<Double, Dir>, Optional<Tuple2<Double, Long>>>>map((lastX, lastDir) -> {
                    if(lastX == null) {
                        // start recording the x position. Can't determine direction yet.
                        return t(t(evt.getX(), (Dir) null), Optional.empty());
                    } else {
                        Dir dir = evt.getX() - lastX > 0 ? Dir.RIGHT : Dir.LEFT;
                        if(lastDir == null || lastDir == dir) { // direction unchanged
                            // record the new position and direction, don't emit anything
                            return t(t(evt.getX(), dir), Optional.empty());
                        } else {
                            // record the new position and direction and emit the turn
                            return t(t(evt.getX(), dir), Optional.of(t(lastX, System.currentTimeMillis())));
                        }
                    }
                }))
                .toEventStream();
    }
}

[ghost]

Thank, Tomas! This is a great example. It works well. I believe it would be a nice example to add to the demos folder.

[JordanMartinez]

I haven't made myself well aquainted with the library as of yet

In case you haven't already, I'd recommend reading through the wiki pages about EventStreams and how to use them well to get more familiar with this library. I wrote the pages under the EventStream section, so please also give some feedback about where it could be clearer if you find such a case.

[ghost]

Thanks Jordan, great! will do.

On Mon, Feb 8, 2016 at 1:18 PM, JordanMartinez [email protected]
wrote:

I haven't made myself well aquainted with the library as of yet

In case you haven't already, I'd recommend reading through the wiki pages
about EventStreams and how to use them well
https://github.com/TomasMikula/ReactFX/wiki to get more familiar with
this library. I wrote the pages under the EventStream section, so please
also give some feedback about where it could be clearer if you find such a
case.

—
Reply to this email directly or view it on GitHub
#52 (comment).

[ghost]

Hi Jordan, I read through it, thanks. It's a very good starter for me but I'll definely need to do some reading on FRP. In the current project, I utilise a lot of bindings to synchronise changes across many parts. My hack was to listen to integer properties that represent "completed update cycles", to which the other parts listen. Also as the code is MVC, my controller also samples MouseDrag events before it sends values to the model, and then runs a update once MouseReleased. Those helped with redraw speed, but I still get a hit in performance when there are many objects in the scene, even after i remove most of them from the scene. I believe this might be due to bindings. I am currently refactoring huge parts of the code to modularise things better. Once done, should be more ready to start figuring out how go FRP about it!

[JordanMartinez]

In the current project, I utilise a lot of bindings to synchronise changes across many parts. My hack was to listen to integer properties that represent "completed update cycles", to which the other parts listen.

Yeah... I'm pretty sure Reactive Programming / FRP would help with that. It sounds like your integer properties are intermediate bindings that hold a value only to notify some other bindings that actually do stuff. That definitely sounds like it would better function as streams.

I'll definitely need to do some reading on FRP

In case you don't already have a few things to read, please allow me to provide some for your consideration. Another link in the wiki pages is the Helpful Reactive Programming Resources. I read through those articles before looking at Tomas library and they helped a lot (aside from the MOOC because that's taught using Scala). Manning.com also has a MEAP on FRP. The first chapter is free, but I have not bought it and read through it myself.

[TomasMikula]

Hi, just to avoid potential confusion, ReactFX is not exactly FRP (just like ReactiveX/RxJava is not FRP), but some ideas in ReactFX are inspired by it. Nevertheless, reading on FRP can still help you get a better understanding of ReactFX.

[ghost]

Thanks for pointing this out. Changing code structure with lots of bindings
will be tricky for me :) Hoping to start on it once done with current
refactoring task.
On Feb 9, 2016 9:34 AM, "Tomas Mikula" [email protected] wrote:

Hi, just to avoid potential confusion, ReactFX is not exactly FRP (just
like ReactiveX/RxJava is not FRP), but some ideas in ReactFX are inspired
by it. Nevertheless, reading on FRP can still help you get a better
understanding of ReactFX.

—
Reply to this email directly or view it on GitHub
#52 (comment).

[JordanMartinez]

I believe this issue can be closed.