Fixes #337 : multiple fixes in the code and modernize the style

[Sharcoux]

This is not 100% complete, as I don't understand very well the build process. I'd need some enlightenment about the www part in order to finish this MR. However, it solves numerous issues reported here along with other ones I noticed on the fly.

[Sharcoux]

I need to make some fixes first. Don't merge anything yet. I'll comment all the changes to justify them

[jeffhuang]

Will do, thank you

[Sharcoux]

I commented the changes, but I would still like to go a bit further with the www folder. I will see if I have the time.

[Sharcoux]

@jeffhuang I finished the complete refacto of the project. I'd be very happy to discuss any change as intensively as needed, but I'm 100% convinced that this structure will be way easier to maintain and evolve in the future. You can test it. I made some small visual changes in the test project and added new buttons, but I can go back to how it was before. I simplified the API a lot for new users of the project. Each function details the available parameters.

I had to switch the src to typescript because of the test project. Importing the types there was too hard otherwise, and securing the calls would have been too hard without any type checks.

[jeffhuang]

Amazing, looking into this and testing...

[jeffhuang]

Sorry for the delay. I'll review this at the end of the week.

[wendesongomes]

Hey @Sharcoux! I saw that MR of yours, checked the code but I'm still wondering if you have fixed the threadRidge bug, did you? Im asking it cause if you did I'll not even open a MR with the fix I did 😄

[Sharcoux]

Hey @Sharcoux! I saw that MR of yours, checked the code but I'm still wondering if you have fixed the threadRidge bug, did you? Im asking it cause if you did I'll not even open a MR with the fix I did 😄

I don't know what bug you're talking about, but you can test my branch. From what I tested, it worked fine...

[wendesongomes]

The build is not working.

[Sharcoux]

I'll check if there might be a problem with the build. How did you run it? Anyway, you need to build the webgazer package before building the www part

[wendesongomes]

i use npm install after npm run build

[Sharcoux]

Do you have errors during the npm run build?

[Sharcoux]

Try with this new commit maybe

[wendesongomes]

u have errors during the npm

this

[wendesongomes]

same error.

your symlink points to /home/victus, could that be the problem?

[Sharcoux]

Ouch... They are not supposed to be part of the files. Let me fix

[Sharcoux]

Done. Sorry for the mess!

[wendesongomes]

It's right now, thanks.

[wendesongomes]

Suggestion for WebGazer: I created a function that sets the FPS of WebGazer to improve performance, allowing the user to choose what suits them best.

webgazer.setFps = function (fps) {
  params.fps = fps

  return webgazer
}

use in loop:

async function loop() {
  if (!paused) {

    // [20200617 XK] TODO: there is currently lag between the camera input and the face overlay. This behavior
    // is not seen in the facemesh demo. probably need to optimize async implementation. I think the issue lies
    // in the implementation of getPrediction().

    // Paint the latest video frame into the canvas which will be analyzed by WebGazer
    // [20180729 JT] Why do we need to do this? clmTracker does this itself _already_, which is just duplicating the work.
    // Is it because other trackers need a canvas instead of an img/video element?
    paintCurrentFrame(videoElementCanvas, videoElementCanvas.width, videoElementCanvas.height);

    // Get gaze prediction (ask clm to track; pass the data to the regressor; get back a prediction)
    latestGazeData = getPrediction();
    // Count time
    var elapsedTime = performance.now() - clockStart;

    // Draw face overlay
    if (webgazer.params.showFaceOverlay) {
      // Get tracker object
      var tracker = webgazer.getTracker();
      faceOverlay.getContext('2d').clearRect(0, 0, videoElement.videoWidth, videoElement.videoHeight);
      tracker.drawFaceOverlay(faceOverlay.getContext('2d'), tracker.getPositions());
    }

    // Feedback box
    // Check that the eyes are inside of the validation box
    if (webgazer.params.showFaceFeedbackBox)
      checkEyesInValidationBox();

    latestGazeData = await latestGazeData;

    // [20200623 xk] callback to function passed into setGazeListener(fn)
    callback(latestGazeData, elapsedTime);

    if (latestGazeData) {
      // [20200608 XK] Smoothing across the most recent 4 predictions, do we need this with Kalman filter?
      smoothingVals.push(latestGazeData);
      var x = 0;
      var y = 0;
      var len = smoothingVals.length;
      for (var d in smoothingVals.data) {
        x += smoothingVals.get(d).x;
        y += smoothingVals.get(d).y;
      }

      var pred = util.bound({ 'x': x / len, 'y': y / len });

      if (webgazer.params.storingPoints) {
        drawCoordinates('blue', pred.x, pred.y); //draws the previous predictions
        //store the position of the past fifty occuring tracker preditions
        webgazer.storePoints(pred.x, pred.y, k);
        k++;
        if (k == 50) {
          k = 0;
        }
      }
      // GazeDot
      if (webgazer.params.showGazeDot) {
        gazeDot.style.display = 'block';
      }
      gazeDot.style.transform = 'translate3d(' + pred.x + 'px,' + pred.y + 'px,0)';
    } else {
      gazeDot.style.display = 'none';
    }

    setTimeout(() => {
      requestAnimationFrame(loop);
    }, 1000 / params.fps)
  }
}

[jeffhuang]

I've been going through these changes for a whlie, and I really appreciate the work @Sharcoux but it's honestly pretty hard to review thoroughly. There's changes to the build process, linting, and small fixes. I really wish each of these could be separate.

Is there a way to pull out the pure linting changes into a separate PR? Some aren't really necessary, like linting the d3. Maybe an alternative is if the www stuff goes into a separate repository at this point, so the library is one repo, and the www is a repo. It's less important that the www is reviewed as thoroughly, as long as it works and showcases the library.

I really want to merge this. It's clearly an improvement. And @xanderkoo is helping out as well. But there's just too many lines changed to go through in a useful way. Thoughts?

[jeffhuang]

Another option that's not very satisfying is if there are any bug fixes, we can review those separately and incorporate them into the main branch. And the rest, we can point to your branch as the more modern style.