- Emotiv Xavier Tools
- 1.1 Connecting XavierEmoKey to Emotiv EmoEngine
- 1.2 Configuring XavierEmoKey Rules
- 1.3 XavierEmoKey Keyboard Emulation
- 1.4 Configuring XavierEmoKey Rule Trigger Conditions
- 1.5 Saving Rules to an XavierEmoKey Mapping file
-
- 2.1 Interactive Mode
- 2.2 EmoScript Mode
This section explains the software utilities provided with the Emotiv SDK: XavierEmoKey and XavierComposer. XavierEmoKey allows you to connect detection results received from the EmoEngine to predefined keystrokes according to easy-to-define, logical rules. This functionality may be used to experiment with the headset as an input controller during application development. It also provides a mechanism for integrating the Emotiv neuroheadset with a preexisting application via the application’s legacy keyboard interface.
XavierComposer emulates the behavior of the EmoEngine with an attached Emotiv neuroheadset. It is intended to be used as a development and testing tool; it makes it easy to send simulated EmoEngine events and request responses to applications using the Emotiv API in a transparent and deterministic way.
XavierEmoKey translates Emotiv detection results to predefined sequences of keystrokes according to logical rules defined by the user through the XavierEmoKey user interface. A set of rules, known as an “EmoKey Mapping”, can be saved for later reuse. XavierEmoKey communicates with Emotiv EmoEngine in the same manner as would a third-party application: by using the Emotiv API exposed by the library.
By default, XavierEmoKey will attempt to connect to Xavier Control Panel when the application launches. If Xavier Control Panel isn’t running, then XavierEmoKey will display a warning message above the system tray. The reason XavierEmoKey connects to the Control Panel, instead of connecting directly to the EmoEngine and the neuroheadset, is to allow the user to select the user profile, configure the detection suite settings, and get contact quality feedback through the Control Panel interface.
XavierEmoKey can also be connected to XavierComposer . This is useful when creating and testing a new EmoKey Mapping since the user can easily generate EmoState update events that satisfy the conditions for rules defined in XavierEmoKey. Refer to Section 1.2 for more information about XavierComposer.
Figure 1: XavierEmoKey Connect Menu
XavierEmoKey’s connection settings can be changed by using the application’s Connect menu. If XavierEmoKey is not able to connect to the target application (usually because the target is not running), then the XavierEmoKey icon in the system tray will be shown as gray, instead of orange). If this occurs, start the desired application (either Xavier Control Panel or XavierComposer ) and then select Reconnect from the Connect menu.
Figure 2: Example XavierEmoKey Mapping
Figure above shows an example of an EmoKey Mapping as it might be configured to communicate with an Instant Messenger (IM) application. In this example, XavierEmoKey will translate Blink events generated by Emotiv’s Facial Expression suite to the text “LOL” (as long as the Instantaneous Excitement is also reporting a score > 0.3), which causes the IM program to send “LOL” automatically when the user blinks.
The topmost table in XavierEmoKey contains user interface controls that allow you to define rules that specify which keystrokes are emulated. Rules can be added by clicking on the Add Rule button. Existing rules can be deleted by selecting the appropriate row and pressing the Delete Rule button. In Figure 2, two rules, named “LOL” and “Wink”, were added. Rules can be edited as outlined below, in Table 1.
| Field | Description | Notes |
|---|---|---|
| Enabled | Checkbox to selectively enable or disable individual rules | The indicator “light” will turn green when the rule conditions are satisfied. |
| Player | Identifies the neuroheadset that is associated with this rule | Player 1 corresponds to user ID 0 in Xavier Composer and Xavier Control Panel. |
| Name | Custom rule name | Edit by double clicking on the cell. |
| Key | Keystroke sequence to be sent to the OS input queue | Edit by double clicking on the cell. |
| Behavior | Checkbox to control whether the key sequence is sent only once, or repeatedly, each time an EmoState update satisfies the rule conditions | If checked, then XavierEmoKey must receive an EmoState update that does NOT satisfy the rule’s conditions before this rule can be triggered again. |
Table 1: XavierEmoKey Rule Definition Fields
XavierEmoKey emulates a Windows-compatible keyboard and sends keyboard input to the OS input queue. The application with the input focus will receive the emulated keystrokes. In practice, this often means that XavierEmoKey is running in the background. Please note that closing the XavierEmoKey window will only hide the application window and that it will continue to run. When running, the XavierEmoKey/Emotiv icon will be visible in the Windows system tray. Double-click on this icon to bring XavierEmoKey back to the foreground. Choose Quit from the Application or system-tray menu to really quit the application.
Figure 3: XavierEmoKey System Tray Icon
Double-clicking in the Key field of a rule will bring up the Keys dialog as shown in Figure 4.
Figure 4: Defining Keys and Keystroke Behavior
The Keys dialog allows the user to specify the desired keystrokes and customize the keystroke behavior. The customizable options include:
-
Holding a key press: hold the key down for the duration of the rule activation period. The Hold the key checkbox is only enabled when a single key has been specified in the keystroke edit box.
-
Hot keys or special keyboard keys: any combination of control, alt, shift, the Windows key, and another keystroke. You may also use this option if you need to specify special keys such as Caps Lock, Shift, or Enter.
-
Key press duration and delay times: some applications, especially games, are sensitive to the timing of key presses. If necessary, use these controls to adjust the simulated keyboard behavior.
The Trigger Conditions table in XavierEmoKey contains user interface controls that allow you to define logical conditions that determine when the corresponding rule is activated. Clicking on a new rule in the Rules table will refresh the contents of the Trigger Conditions table, causing it to display only the conditions associated with the selected rule.
Conditions can be added by clicking on the Add Condition button. Existing rules can be deleted by selecting the appropriate condition and pressing the Delete Condition button. In Figure 2, two conditions, which examine the state of the Laugh and the Instantaneous Excitement detection, respectively, are associated with the LOL rule. All enabled conditions must be satisfied by the most recent EmoState update received from Xavier Control Panel or Xavier Composer for a rule to be triggered.
The fields of the Trigger Conditions table are described below, in Table 2.
| Field | Description |
|---|---|
| Enabled | Checkbox to selectively enable or disable individual trigger conditions |
| Action | The detection that links to this condition |
| Trigger | Description of the trigger condition being evaluated |
| Value | For non-binary trigger conditions, the value being compared to the action score returned by the designated detection |
Table 2: XavierEmoKey Trigger Condition Fields
Double-clicking on any of the fields of a condition in the Trigger Conditions table will reveal the Configure Condition dialog box, as shown in Figure 5. Use the controls on this dialog to specify an action (or detection) name, a comparison function, and a value, that must evaluate to true for this condition to be satisfied.
Figure 5: Defining an XavierEmoKey Condition
XavierEmoKey allows you to save the current set of rule definitions to an XavierEmoKey Mapping file that can be reloaded for subsequent use. Use the appropriate action in XavierEmoKey’s Application menu to rename, save, and load EmoKey mapping files.
Xavier Composer allows you to send user-defined EmoStates to Xavier Control Panel, XavierEmoKey, or any other application that makes use of the Emotiv API. Xavier Composer supports two modes of EmoState generation: Interactive mode and EmoScript mode. In addition to generating EmoStates, Xavier Composer can also simulate Emotiv EmoEngine’s handling of profile management and training requests.
SDK users can rely on Xavier Composer to simulate the behavior of Emotiv EmoEngine and Emotiv neuroheadsets without connecting to a real one. It is a very useful tool for all Emotiv SDK developers, allowing for easy experimentation with the Emotiv API early in the development process, and facilitating testing later in the development cycle.
Figure 6: Xavier Composer Interactive Mode
Xavier Composer Interactive mode allows you to define and send specific EmoState values to any application using the Emotiv SDK. The user interface settings are described below.
-
Player: choose the player number who’s EmoState you wish to define and send. The player number defaults to 0. When the player number is changed for the first time, an application connected to XavierComposer will receive an EE_UserAdded event with the new player number reported as the user ID.
-
Wireless: sets the simulated wireless signal strength. Note: if the signal strength is set to “Bad” or “No Signal” then XavierComposer simulates the behavior of the EmoEngine by setting subsequent EmoState detection values and contact quality values to 0.
-
Contact Quality: this tab allows you to adjust the reported contact quality for each sensor on the Emotiv neuroheadset. When the Contact Quality tab is active, all other EmoState detection values are set to 0. You can choose between typical sensor contact quality readings by selecting a preset from the General Settings drop-down box. If you choose the Custom preset, each sensor value can be controlled individually by clicking on a sensor and then selecting a new CQ Status value in the Sensor Details box. Note that the sensors above and behind the ears, correspond to the reference sensors on the Emotiv neuroheadset, will always report a CQ value of "Good" and cannot be adjusted.
-
Detection: this tab allows you to interactively control EmoState values and training result. When the Detection tab is active, the contact quality values for generated EmoStates will always be set to "Good".
-
EmoState: define the detection values in an EmoState by selecting the particular event type for each detection group in the appropriate drop-down list box. Set the event value in the spin box adjacent to the event name. You can define your own time value in the Time edit box, or allow XavierComposer to set the value automatically by incrementing it by the value of the EmoState Interval spin box after each EmoState is sent. The Excitement state is unique in that the EmoEngine returns both short-term (for Instantaneous Excitement) and long-term values. XavierComposer simulates the long-term value calculation adequately enough for testing purposes but does not reproduce values from the exact algorithm. Note that the value for the eye detections are binary (Active or Inactive) and that they are automatically reset to be Inactive after an EmoState is sent. If "Auto Repeat" mode is active, then you can press and hold the Activate button to maintain a particular eye state across multiple time intervals. Also note that the value for a neutral Mental Commands detection is automatically set to 0.
-
Training Results: specify the desired return value for EmoEngine requests generated for the current player by the IEE_CognitivSetTrainingControl and IEE_FacialExpressivSetTrainingControl functions.
-
EmoEngine Log: contents are intended to give developers a clearer picture about how the EmoEngine processes the requests generated by various Emotiv API functions. The log displays 4 different output types: "Request", "Reply", "MCResult", and "FEResult". An API function call that results in a new request to the EmoEngine will cause a Request output line to be displayed in the log. The multitude of API functions are translated to roughly a dozen different strings intended to allow the Emotiv SDK developer to see that an API function call has been serviced. These strings include: PROFILE_ADD_USER, PROFILE_CHANGE_USER, PROFILE_REMOVE_USER, PROFILE_LIST_USER, PROFILE_GET_CURRENT_USER, PROFILE_LOAD, PROFILE_SAVE, FACIALEXPRESSION_GET, FACIALEXPRESSION_SET, MENTALCOMMAND_SET and MENTALCOMMAND_GET. Because of the comparatively complex API protocol used to facilitate training the Mental Command algorithms, we display additional details when we receive training control messages generated by the IEE_MentalCommandSetTrainingControl API function. These strings are: MC_START, MC_ACCEPT, and MC_REJECT, which correspond to the IEE_MentalCommandTrainingControl_t constants exposed to developers in the SDK header files. Similar strings are used for the equivalent Facial Expression messages. All other request types are displayed as API_REQUEST. The Reply output line displays the error code and is either green or red, depending on whether an error has occurred (i.e. the error code != 0). The MCResult and FEResult outputs are used to inform the developer of an asynchronous response sent from the EmoEngine via an EmoState update as the result of an active Mental Command or Facial Expression training request.
-
Start: sends the EmoState to the connected application at the time interval specified in the EmoState Interval spin box. Use the Start/Stop button to turn the automated send on and off. You may interact with the EmoState controls to dynamically change the EmoState values while the automated send is active. Note that switching between the Contact Quality and Detection tabs in Interactive mode will automatically stop an automated send, and Xavier Control Panel will display “Cannot Acquire Data” until the first EmoState is received from XavierComposer.
-
Auto Reset: check this box to tell XavierComposer to automatically reset EmoState after sending the current one.
Figure 7: XavierComposer EmoScript Mode
XavierComposer EmoScript mode allows you to playback a predefined sequence of EmoState values to any application using the EmoEngine. The user interface settings are described below. EmoScript files are written in EML (Emotiv Markup Language). EML syntax details can be found in this document.
-
Player: choose the player number to associate with the generated EmoStates.
-
File: click the “...” button to select and load an EmoScript file from disk. If the file loads successfully then the timeline slider bar and Start button will be activated. If an error occurs, a message box will appear with a description and approximate location in the file.
-
Timeline Slider: move the slider control to see the EmoState and contact quality values for any point on the timeline defined in the EmoScript file.
-
Start/Stop button: starts or stops the playback of the EmoState values generated by the EmoScript file.
-
Wireless: the wireless signal strength setting is disabled while in EmoScript mode and it is always set to “Good.”
-
Contact Quality: the indicators on the head model correspond to the values defined by the contact_quality tag at specific time code in the EmoScript file. If no contact_quality tag has been specified then the contact quality values in the generated EmoStates default to "Good".
-
Detection: this tab allows you to view EmoState detection values and provides interactive control over training result values. Unlike Interactive mode, the contact quality and detection values are determined entirely by the contents of the EmoScript file and you can switch between the Contact Quality tab and the Detection tab to view the appropriate data during playback or timeline navigation.
-
EmoState: the values displayed correspond to the EmoState values for a particular point in time as defined by the EmoScript file. Note that these EmoScript values are not interactive and cannot be modified by the user (use the Interactive mode for this instead).
-
Training Results and EmoEngine Log: these controls operate exactly the same as they do in Interactive mode. See the Interactive mode above for more details.