gaze tracking with poser

[Sohalt]

Do you think it would be possible to expand the poser case to do rudimentary gaze tracking?

[zaidalyafeai]

I need more details about the task.

[Sohalt]

What I was thinking about was roughly guessing the position on the screen a person is looking at from the webcam image, similar to the service offered by https://xlabsgaze.com.
My specific use case is, that I would like to automatically focus the window I am looking at. The accuracy for this does not need to be very high (10-20cm should be enough), but I do not want to involve additional cameras, other than the webcam already set up.

[zaidalyafeai]

It will be difficult because you might have your head still but look at different places in the screen.

[Sohalt]

While the head might stay still, the pupil will look in different directions however. So what I was thinking was, using the poser network to locate the eyes, then feed a high resolution part of the image cropped to the area around the eyes into a network, together with the overall position and direction of the head, as determined by the poser network and try to correlate it with a position on the screen. To train the network one would capture video of the person in front of the camera while asking them to follow a moving dot on screen with their eyes.

[zaidalyafeai]

Seems like a great idea.