-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathpatents.html
More file actions
1567 lines (1532 loc) · 237 KB
/
patents.html
File metadata and controls
1567 lines (1532 loc) · 237 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE HTML>
<html>
<head>
<title>VisualQ - Patent </title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" />
<link rel="stylesheet" href="assets/css/main.css" />
<noscript><link rel="stylesheet" href="assets/css/noscript.css" /></noscript>
</head>
<body class="is-preload">
<!-- Page Wrapper -->
<div id="page-wrapper">
<!-- Header -->
<header id="header">
<h1><a href="index.html">VisualQ</a></h1>
<nav id="nav">
<ul>
<li class="special">
<a href="#menu" class="menuToggle"><span>Menu</span></a>
<div id="menu">
<ul>
<li><a href="index.html">Home</a></li>
<li><a href="patents.html">Generic</a></li>
</ul>
</div>
</li>
</ul>
</nav>
</header>
<!-- Main -->
<article id="main">
<header>
<h2>Hand sanitation compliance enforcement systems and methods</h2>
<p>An electronic processor is operatively connected with at least one video camera, or additionally or alternatively a non-camera-based sensor, and at least one annunciator to process real time video acquired by the at least one video camera
in real time to perform a hand hygiene compliance enforcement method for encouraging use of a hand sanitation station.</p>
</header>
<section class="wrapper style5">
<div class="inner">
<div class="primary-content">
<div class="title-wrapper">
<h1 class="heading-1"> Hand sanitation compliance enforcement systems and methods </h1>
</div>
<div class="wrap">
<div id="byline" class="small-font">
<time datetime="2021-04-21" class="color-dove-gray">Apr 21, 2021</time>VISUALQ
</div>
<div id="abstract" class="to-xlarge-font">
<p>An electronic processor is operatively connected with at least one video camera, or additionally or alternatively a non-camera-based sensor, and at least one annunciator to process real time video acquired by the at least
one video camera in real time to perform a hand hygiene compliance enforcement method for encouraging use of a hand sanitation station. The real time video is analyzed to determine an occupancy time interval over which
a sanitation zone imaged by the real time video is occupied. Whether a hand sanitation event occurs is determined based on images of a wash station zone extracted from the real time video acquired during the occupancy
time interval. The at least one annunciator is controlled to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.</p>
</div>
<div id="content-navigation">
<span class="right-10">Skip to:</span>
<span>
<a href="#description">Description</a>
</span>
<span> · </span><span>
<a href="#claims">Claims</a>
</span>
<span> · </span><span>
<a href="#citations">References Cited</a>
</span>
<span> · </span><span><a href="#history">Patent History</a></span>
<span> · </span><span>
<a href="#history">Patent History</a>
</span>
</div>
<div id="description" class="wrapper">
<div class="title-wrapper">
<strong class="heading-2">Description</strong>
</div>
<div class="wrap">
<strong class="heading-4">CROSS REFERENCE TO RELATED APPLICATION(S)</strong>
<p id="p-0002" num="0001">This application claims the benefit of U.S. application Ser. No. 17/067,172 (filed on Oct. 9, 2020) and U.S. Provisional Application No. 62/912,787 (filed on Oct. 9, 2019). The entirety of each of the foregoing applications
is incorporated by reference herein.</p> <strong class="heading-4">FIELD OF THE DISCLOSURE</strong>
<p id="p-0003" num="0002">The present disclosure generally relates to hand sanitation compliance enforcement systems and methods, and, more particularly, to hand sanitation and/or hygiene compliance enforcement systems, devices, and/or methods
relating to the hand sanitation arts and to related arts such as medical sanitation arts, culinary sanitation arts, and the like.</p> <strong class="heading-4">BACKGROUND</strong>
<p id="p-0004" num="0003">Hand sanitation is a critical component in areas such as healthcare, food preparation/handling, child care, restaurants, elderly care, or the like. For example, a hospital ward housing critically ill patients may have
a hand sanitation station located outside the entrance door, including a wash basin, sanitizing soap, and/or the like. Doctors, nurses, or other medical personnel are expected to follow a hand hygiene protocol which
includes sanitizing their hands at the hand sanitation station prior to entering the hospital ward. In some protocols, gloves may be donned after performing hand sanitation, for example in accordance with a “gown
and glove” procedure such as is employed in a clostridium difficule (C-DIFF) protocol. A second hand sanitation station may be located inside the hospital ward proximate to the exit door, and the hand hygiene protocol
further calls for medical personnel to sanitize their hands at the second hand sanitation station prior to exiting the hospital ward. Following the prescribed hand hygiene protocol can prevent contact-mediated spread
of infectious diseases (e.g., coronavirus disease 2019 aka “COVID-19”) to and/or from critically ill patients. Similar hand hygiene protocols may be established for restaurant workers or others employed in the food
preparation/handling industries, for teachers who come into frequent contact with children, for workers at assisted care facilities who come into frequent contact with elderly residents, or so forth.</p>
<p id="p-0005" num="0004">Unfortunately, compliance with a hand hygiene protocol is imperfect. This can lead to unnecessary spread of infectious diseases (e.g., COVID-19). Noncompliance has been addressed by educational/training programs, and
by use of prominent signage to remind personnel to use the hand sanitation station. Education or training, while helpful, does not effectively prevent noncompliance due to forgetfulness or simple laziness. Prominent
signage may reduce forgetfulness, but signage can easily become overfamiliar if overused. For example, a hospital may have reminder signs posted at the entrance to every patient ward; over time, hospital personnel
come to tune out these ubiquitous signs thereby reducing their effectiveness. Furthermore, prominent signage is not a deterrent to noncompliance due to laziness.</p>
<p id="p-0006" num="0005">For the foregoing reasons, certain improvements are disclosed herein, as there is a need for systems, devices, and methods for hand hygiene compliance and enforcement.</p> <strong class="heading-4">BRIEF SUMMARY</strong>
<p id="p-0007" num="0006">In accordance with some illustrative embodiments disclosed herein, a hand hygiene compliance enforcement device comprises at least one video camera, at least one annunciator, an electronic processor; and a non-transitory
storage medium that stores instructions that are readable and executable by the electronic processor to process real time video acquired by the at least one video camera in real time. The instructions include occupancy
detection instructions readable and executable by the electronic processor to analyze the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real time video is occupied.
The instructions further include sanitation detection instructions readable and executable by the electronic processor to determine whether a hand sanitation event occurs based on images of a wash station zone extracted
from the real time video acquired during the occupancy time period and to control the at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation
event occurs.</p>
<p id="p-0008" num="0007">In accordance with some illustrative embodiments disclosed herein, a hand sanitation station includes a hand sanitation station including at least a sanitation fluid dispenser, and a hand hygiene compliance enforcement
device as set forth in the immediately preceding paragraph, in which the at least one video camera is arranged such that at least a portion of the hand sanitation station coincides with the wash station zone in
the real time video acquired by the at least one video camera.</p>
<p id="p-0009" num="0008">In accordance with some illustrative embodiments disclosed herein, a non-transitory storage medium stores instructions that are readable and executable by an electronic processor operatively connected with at least
one video camera and at least one annunciator to process real time video acquired by the at least one video camera in real time to perform a hand hygiene compliance enforcement method comprising: analyzing the real
time video to determine an occupancy time interval over which a sanitation zone imaged by the real time video is occupied; determining whether a hand sanitation event occurs based on images of a wash station zone
extracted from the real time video acquired during the occupancy time period; and controlling the at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand
sanitation event occurs.</p>
<p id="p-0010" num="0009">In accordance with some illustrative embodiments disclosed herein, a hand hygiene compliance enforcement method comprise: using at least one video camera, acquiring real time video of a field of view that includes a
sanitation station having a wash station zone; using an electronic processor, determining whether a hand sanitation event occurs based on images of the wash station zone extracted from the real time video; and using
an annunciator, annunciating a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.</p>
<p id="p-0011" num="0010">As described herein, a hand hygiene compliance enforcement device is disclosed that comprises: at least one video camera; at least one annunciator; an electronic processor; and a non-transitory storage medium storing
instructions that are readable and executable by the electronic processor, wherein the instructions, when executed by the electronic processor, cause the electronic processor to: process real time video acquired
by the at least one video camera in real time, analyze, by execution of occupancy detection instructions, the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real
time video is occupied, determine, by execution of sanitation detection instructions, whether a hand sanitation event occurs based on images of a wash station zone as extracted from the real time video acquired
during the occupancy time interval, and control the at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.</p>
<p id="p-0012" num="0011">In additional embodiments, as described herein, a non-transitory storage medium is disclosed for storing instructions for performing a hand hygiene compliance enforcement, the instructions being readable and executable
by an electronic processor, wherein the instructions, when executed by the electronic processor, cause the electronic processor to: receive real time video acquired by at least one video camera in real time; analyze
the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real time video is occupied; determine whether a hand sanitation event occurs based on images of a wash station
zone as extracted from the real time video acquired during the occupancy time interval; and control at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a
hand sanitation event occurs.</p>
<p id="p-0013" num="0012">In still further embodiments, as described herein, a hand hygiene compliance enforcement method is disclosed that comprises: acquiring, via a at least one video camera, real time video of a field of view that includes
a sanitation station having a wash station zone; determining, via an electronic processor, whether a hand sanitation event occurs based on images of the wash station zone as extracted from the real time video; and
annunciating, via an annunciator, a compliance status based on the determination of whether a hand sanitation event occurs.</p>
<p id="p-0014" num="0013">In additional embodiments, as described herein, a hand hygiene compliance enforcement device is disclosed that comprises: at least one non-camera-based sensor; at least one annunciator; an electronic processor; and
a non-transitory storage medium storing instructions that are readable and executable by the electronic processor, wherein the instructions, when executed by the electronic processor, cause the electronic processor
to: process real time data output of the non-camera-based sensor in real time, analyze, by execution of occupancy detection instructions, the data output to determine an occupancy time interval over which a sanitation
zone determined by the data output is occupied, determine, by execution of sanitation detection instructions, whether a hand sanitation event occurs based on the data output corresponding to a wash station zone,
the data output acquired during the occupancy time interval, and control the at least one annunciator to annunciate a hand hygiene compliance status based on the hand sanitation event.</p>
<p id="p-0015" num="0014">As described herein, the embodiments of the disclosure may comprise one or more video cameras and/or, additionally or alternatively, one or more non-camera based sensors. For example, while the disclosure herein refers,
in many embodiments, to use of a video camera, it is to be understood that, additionally or alternatively, other motion or video-like devices could be used as a substitute for a video camera. For example, for embodiments
using video camera(s), one or more images or image frames (e.g., comprising a video) as captured by the video camera(s) may turned into, or used to create, arrays of digits that represent the image. Additionally,
or alternatively, for embodiments using non-camera based sensor(s) the same (or similar) array structure(s) (as created for video camera embodiments) can be emulated by non-camera based sensor(s). Such non-camera
based sensor(s) can comprise, by way of non-limiting example, microwave sensor(s), depth sensor(s), tomographic sensor(s), and/or infrared (passive and active) sensor(s), or other such similar or related sensors
or technologies. These non-camera based sensor(s) can provide stream(s) of data, or otherwise inputs that emulate the use of a video camera. Just as video camera input, such non-camera based sensor(s) inputs or
streams of data can also serve as the base for artificial intelligence based model(s) (e.g., machine learning model(s) and/or CNN model(s)), and related model pattern recognition, as described herein. In this way,
non-camera-based sensors can create (or emulate) the same (or similar) array data structure(s) that can then be used to generate tensors for pattern recognition, just as for video camera related embodiments.</p>
<p id="p-0016" num="0015">In accordance with the above, and with the disclosure herein, present disclosure includes improvements in computer functionality or in improvements to other technologies at least because the claims recite that, e.g.,
a hand hygiene compliance enforcement device is improved by implementation of machine learning to improve the prediction accuracy of the hand hygiene compliance enforcement device to detect hand sanitation event(s)
in real time. That is, the present disclosure describes improvements in the functioning of the computer itself or “any other technology or technical field” because a hygiene compliance enforcement device is better
able to predict or detect hand sanitation event(s). This improves over the prior art at least because either no such device(s) existed or were limited to non-machine learning based approaches.</p>
<p id="p-0017" num="0016">The present disclosure includes applying certain of the features described herein with, or by use of, a particular machine, e.g., a video camera and/or a non-camera based sensor for processing real time video or sensor
data acquired by a video camera or non-camera based sensor in real time.</p>
<p id="p-0018" num="0017">The present disclosure includes effecting a transformation or reduction of a particular article to a different state or thing, e.g., transforming or reducing video images and/or sensor data to actual image or audible
output to indicate whether or not a hand sanitation event occurs.</p>
<p id="p-0019" num="0018">The present disclosure includes specific features other than what is well-understood, routine, conventional activity in the field, or adding unconventional steps that confine the claim to a particular useful application,
e.g., hand hygiene compliance enforcement devices and methods, as described herein.</p>
<p id="p-0020" num="0019">Advantages will become more apparent to those of ordinary skill in the art from the following description of the preferred embodiments which have been shown and described by way of illustration. As will be realized,
the present embodiments may be capable of other and different embodiments, and their details are capable of modification in various respects. Accordingly, the drawings and description are to be regarded as illustrative
in nature and not as restrictive.</p>
<description-of-drawings> <strong class="heading-4">BRIEF DESCRIPTION OF THE DRAWINGS</strong>
<p id="p-0021" num="0020">The Figures described below depict various aspects of the system(s), devise(s), and method(s) disclosed therein. It should be understood that each Figure depicts an embodiment of a particular aspect of the disclosed
system and methods, and that each of the Figures is intended to accord with a possible embodiment thereof. Further, wherever possible, the following description refers to the reference numerals included in the
following Figures, in which features depicted in multiple Figures are designated with consistent reference numerals.</p>
<p id="p-0022" num="0021">There are shown in the drawings arrangements which are presently discussed, it being understood, however, that the present embodiments are not limited to the precise arrangements and instrumentalities shown, wherein:
</p>
<p id="p-0023" num="0022">
<figref idref="DRAWINGS">FIG. 1</figref> is a perspective view that illustrates a hand sanitation station in context, along with a hand hygiene protocol enforcement system.</p>
<p id="p-0024" num="0023">
<figref idref="DRAWINGS">FIG. 2</figref> is a flow diagram that diagrammatically illustrates operation of the hand hygiene protocol enforcement system of
<figref idref="DRAWINGS">FIG. 1</figref>.</p>
<p id="p-0025" num="0024">
<figref idref="DRAWINGS">FIG. 3</figref> is a flow diagram that diagrammatically illustrates operation of the sanitation state determination operation of
<figref idref="DRAWINGS">FIG. 2</figref>.</p>
<p id="p-0026" num="0025">
<figref idref="DRAWINGS">FIG. 4</figref> illustrates a flowchart of the hand hygiene protocol enforcement system of
<figref idref="DRAWINGS">FIG. 1</figref> used to provide marketing based information sharing.</p>
</description-of-drawings>
<p id="p-0027" num="0026">The Figures depict preferred embodiments for purposes of illustration only. Alternative embodiments of the systems and methods illustrated herein may be employed without departing from the principles of the invention
described herein.</p>
<p id="p-0028" num="0027">Any quantitative dimensions shown in the drawing are to be understood as non-limiting illustrative examples. Unless otherwise indicated, the drawings are not to scale; if any aspect of the drawings is indicated as being
to scale, the illustrated scale is to be understood as non-limiting illustrative example.</p> <strong class="heading-4">DETAILED DESCRIPTION</strong>
<p id="p-0029" num="0028">Disclosed herein are hand hygiene protocol enforcement systems, devices, instructions (as stored on computer readable medium), and methods which improve compliance with a hand hygiene protocol. The disclosed hand hygiene
protocol enforcement directly detects compliance or noncompliance, and provides an immediate and active reminder upon detection of noncompliance, for example in the form of a flashing light and/or audible reminder.
Unlike static posted signage which is easily tuned out due to its static ubiquity, the flashing light and/or audible reminder is virtually certain to be noticed. Some embodiments of the disclosed hand hygiene protocol
enforcement also leverage group psychology by placing the flashing light in a prominent location so that passers-by are likely to notice it and thereby become aware of the noncompliance event; likewise, an audible
alarm can be made sufficiently loud to be heard by passers-by. In this way, it is not necessary to specifically identify noncompliant personnel (although identification of personnel as part of the hand hygiene protocol
enforcement is contemplated in some variant embodiments) Likewise, the disclosed hand hygiene protocol enforcement can optionally maintain compliance statistics which are anonymous as monitored individuals are not
identified (although, again, in some variant embodiments individuals may be identified and the statistics maintained for individuals). A further advantage of the disclosed hand hygiene protocol enforcement is that
it is easily retrofitted to existing hand sanitation stations, and can be added to existing or new hand sanitation stations as an after-market modification.</p>
<p id="p-0030" num="0029">With reference to
<figref idref="DRAWINGS">FIG. 1</figref>, a portion of a room is depicted, including a floor <strong>2</strong>, ceiling <strong>4</strong>, and walls <strong>6</strong>, <strong>8</strong>. The room could be a hospital corridor, a hospital
ward entranceway, a hospital ward exit hall, a bathroom, a restaurant kitchen, a laboratory hallway, or other such area or room typically used for sanitation or the like. An illustrative hand sanitation station
<strong>10</strong> disposed in the room includes two hand sanitation fluid dispensers: a dispenser <strong>12</strong> for providing liquid soap or an antiseptic, antibiotic, antibacterial, alcohol-based, or other
specialized hand sanitation fluid, and a water faucet <strong>14</strong>. The hand sanitation station <strong>10</strong> further includes a washbasin <strong>16</strong>. This is merely an illustrative example,
and the hand sanitation station may include numerous additional, fewer, and/or different components. As another example, the hand sanitation station could comprise a soap bar on a soap bar holder, and the illustrative
water faucet <strong>14</strong>. In this case, the water faucet is the sole sanitation fluid dispenser, and is used in conjunction with the soap bar. Another possible embodiment could include the dispenser <strong>12</strong> for providing an antiseptic, antibiotic, antibacterial, alcohol-based, or other specialized hand sanitation fluid, but not include the water faucet <strong>14</strong> and not include the washbasin <strong>16</strong>.
The hand sanitation fluid may be a liquid, foam (e.g., hospital foam dispenser), or the like. A sanitation fluid dispenser that dispenses a gaseous sanitation fluid is also contemplated. Furthermore, the hand sanitation
station may include a box of gloves or the like, a gown closet for use in a gown-and-glove protocol, or so forth (not shown).
<figref idref="DRAWINGS">FIG. 1</figref> illustrates a typical situation, in which the sanitation station <strong>10</strong> is situated next to or proximate to a door <strong>18</strong> or narrow passage. The hand hygiene protocol typically
calls for a person to use the hand sanitation station <strong>10</strong> prior to passing through the door <strong>18</strong>. For example, the door <strong>18</strong> could be the entrance to a patient ward
in a hospital, and the hand sanitation station <strong>10</strong> is located outside that door so that medical personnel sanitize their hands before entering the patient ward via door
<strong>18</strong>. Alternatively, the door <strong>18</strong> could be the exit from a patient ward, in which case the hand sanitation station <strong>10</strong> would be located inside the patient ward just
prior to reaching the exit door. As yet another example, the door <strong>18</strong> could be the door of a restaurant bathroom, and the hand sanitation station <strong>10</strong> would be the washbasin of the
bathroom. A further example, the room of
<figref idref="DRAWINGS">FIG. 1</figref> may comprise a door, a passage way, or portioned out area in a hospital ward or restaurant where each section of the room may be dedicated to a specific patient, specific food based task item, or
other specific person, event, or otherwise activity.</p>
<p id="p-0031" num="0030">With continuing reference to
<figref idref="DRAWINGS">FIG. 1</figref>, the illustrative hand sanitation station <strong>10</strong> is augmented by a hand hygiene compliance enforcement device as disclosed herein. The hand hygiene compliance enforcement device includes
at least one video camera <strong>20</strong> (and, in some embodiments, only a single video camera <strong>20</strong> as illustrated in
<figref idref="DRAWINGS">FIG. 1</figref>, or multiple video cameras (not shown), or, additionally or alternatively, non-camera based sensor(s)), at least one annunciator (in the illustrative example a noncompliance light <strong>22</strong>),
an electronic processor <strong>24</strong>, and a non-transitory storage medium <strong>26</strong> storing instructions <strong>30</strong>, <strong>32</strong>, <strong>34</strong> that are readable and executable
by the electronic processor <strong>24</strong> to process real time video acquired by the at least one video camera <strong>20</strong> in real time.</p>
<p id="p-0032" num="0031">The video camera <strong>20</strong> is arranged, positioned, or configured such that at least a portion of the hand sanitation station <strong>10</strong> coincides with a wash station zone <strong>40</strong> captured
by (i.e. imaged by) real time video acquired by the video camera <strong>20</strong>. The wash station zone <strong>40</strong> captured by the video camera <strong>20</strong> (which is only a portion of the full
field of view, i.e. FOV, of the video camera <strong>20</strong>) thus coincides with the spatial region within which the hand sanitation is expected to be performed by a person using the hand sanitation station
<strong>10</strong>. In the illustrative example, the wash station zone <strong>40</strong> coincides with the area of the dispenser tip of the hand sanitation fluid dispenser
<strong>12</strong>, the faucet <strong>14</strong>, and the area above the washbasin <strong>16</strong>.</p>
<p id="p-0033" num="0032">The electronic processor <strong>24</strong> and non-transitory storage medium <strong>26</strong> are shown diagrammatically in
<figref idref="DRAWINGS">FIG. 1</figref> by dotted or dashed lines. In physical implementations, the electronic processor <strong>24</strong> may be embodied as a notebook or desktop computer, or as a server computer accessible via the
Internet and/or another electronic network (e.g., a public or private local area network (LAN)), or may be embodied as a dedicated electronic component comprising a microprocessor or microcontroller mounted on a
printed circuit board (PCB) with ancillary electronics, or so forth. The non-transitory storage medium <strong>26</strong> may be embodied as a hard disk or other magnetic storage medium, a solid-state drive (SSD),
flash memory, or other electronic storage medium, an optical disk or other optical storage medium, various combinations thereof, and/or so forth. The non-transitory storage medium
<strong>26</strong> may be integrated with the computer or dedicated electronic component implementing the electronic processor <strong>24</strong>, e.g. embodied as an internal hard drive or SSD of the computer,
or as a read only memory (ROM) other electronic memory chip mounted on the PCB together with the microprocessor or microcontroller. Additionally or alternatively, the non-transitory storage medium
<strong>26</strong> may be separate from the computer or dedicated electronic component implementing the electronic processor <strong>24</strong>, e.g. embodied as an external hard drive or SSD connected with the
computer by a wired or wireless connection, or a network-based storage accessed by the server via the Internet and/or other electronic network. It will be appreciated that the components <strong>24</strong>,
<strong>26</strong> are shown diagrammatically in
<figref idref="DRAWINGS">FIG. 1</figref>, and do not typically occupy the location where they are shown. Rather, they may be located remotely, or may be integrated with the video camera <strong>20</strong> or into the camera mount, or so
forth. The electronic processor <strong>24</strong> is operatively connected with the video camera <strong>20</strong> via a wired or wireless connection in order to receive real-time video acquired by the video
camera <strong>20</strong>. By “real time” video, it is meant that the electronic processor <strong>24</strong> receives and processes the real-time video to identify whether a hand sanitation event occurs (that
is, whether a person uses the hand sanitation station <strong>10</strong> to sanitize their hands) and controls the annunciator <strong>22</strong> to annunciate a hand hygiene compliance status based on that determination
with sufficiently low time latency that the person perceives he or she is receiving immediate feedback, especially if the person walks past the hand sanitation station
<strong>10</strong> without using it. Typically, a time latency of three seconds or less is sufficient, although a time latency of two seconds or less is preferred, and a time latency of one second or less is even
more preferable. This high speed computing capacity is readily achieved, for example, if the electronic processor <strong>24</strong> is embodied as a typical commercially available microprocessor or microcontroller
such an ARM, Intel, or AMD microprocessor.</p>
<p id="p-0034" num="0033">As noted, the non-transitory storage medium <strong>26</strong> stores instructions <strong>30</strong>, <strong>32</strong>, <strong>34</strong> that are readable and executable by the electronic processor
<strong>24</strong> to process real time video acquired by the video camera <strong>20</strong> in real time. As diagrammatically indicated in
<figref idref="DRAWINGS">FIG. 1</figref>, in the illustrative embodiment the instructions include: occupancy detection instructions <strong>30</strong> readable and executable by the electronic processor <strong>24</strong> to analyze the
real time video to determine an occupancy time interval over which a sanitation zone imaged by the real time video is occupied; and sanitation detection instructions <strong>32</strong> readable and executable by
the electronic processor <strong>24</strong> to determine whether a hand sanitation event occurs based on images of the wash station zone <strong>40</strong> extracted from the real time video acquired during the
occupancy time period and to control the annunciator <strong>22</strong> to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs. Reporting instructions
<strong>34</strong> may also be provided, which are readable and executable by the electronic processor <strong>24</strong> to accumulate data statistics for a plurality of successive occupancy time periods on corresponding
determinations of whether a hand sanitation event occurs and generate a compliance report on the accumulated data statistics.</p>
<p id="p-0035" num="0034">The annunciator <strong>22</strong> typically includes the aforementioned noncompliance light <strong>22</strong>, which is controlled by execution of the sanitation detection instructions <strong>32</strong> to illuminate
to indicate detection of noncompliance with the hand hygiene protocol. Typically, the noncompliance light <strong>22</strong> is a red light, although other colors are contemplated herein and may be used. Optionally,
the annunciator may include a compliance light <strong>42</strong>, e.g. a green light, which is controlled by execution of the sanitation detection instructions <strong>32</strong> to illuminate or to indicate
detection of compliance with the hand hygiene protocol (e.g., detection of occurrence of a hand sanitation event). The illustrative lighting annunciators <strong>22</strong>,
<strong>42</strong> are shown as being placed above the door <strong>18</strong>, although they may be placed in some other prominent location where the person entering the door <strong>18</strong> is unlikely to
miss them. Additionally or alternatively, the annunciator may be implemented as an audio speaker <strong>44</strong> which is controlled by execution of the sanitation detection instructions <strong>32</strong> to output a verbal reminder to use the hand sanitation station <strong>10</strong> in response to detection of noncompliance with the hand hygiene protocol. Other annunciator devices are additionally or alternatively
contemplated, such as a display that displays a textual and/or graphical reminder to use the hand sanitation station <strong>10</strong> in response to detection of noncompliance with the hand hygiene protocol.
In lighting or display annunciators, illumination may be accomplished by a flashing illumination (especially in the case of the noncompliance annunciator light <strong>22</strong>), or, in the case of a display,
showing or displaying the textual and/or graphical reminder as a flashing text and/or flashing graphic.</p>
<p id="p-0036" num="0035">With reference now to
<figref idref="DRAWINGS">FIG. 2</figref>, some illustrative hand hygiene compliance enforcement methods suitably implemented by the hand hygiene compliance enforcement device of
<figref idref="DRAWINGS">FIG. 1</figref> are described. In illustrative
<figref idref="DRAWINGS">FIG. 2</figref>, method operations indicated by blocks are enclosed by dashed lines indicating operations performed by the occupancy detection instructions <strong>30</strong>, the sanitation detection instructions
<strong>32</strong>, and the reporting instructions <strong>34</strong>, respectively. However, it is to be appreciated that these are merely illustrative allocations, and that more generally the hand hygiene compliance
enforcement method may be encoded using different organizational arrangements. More generally, the non-transitory storage medium <strong>26</strong> stores instructions <strong>30</strong>, <strong>32</strong>,
<strong>34</strong> are readable and executable by the electronic processor <strong>24</strong> operatively connected with the at least one video camera <strong>20</strong> and at least one annunciator <strong>22</strong>,
<strong>42</strong>, <strong>44</strong>, and are executable by the electronic processor <strong>24</strong> to process real time video acquired by the at least one video camera <strong>20</strong> in real time
to perform the disclosed hand hygiene compliance enforcement methods.</p>
<p id="p-0037" num="0036">Additionally, or alternatively, with respect to
<figref idref="DRAWINGS">FIG. 1</figref>, one or both of electronic processor <strong>24</strong> and non-transitory storage medium <strong>26</strong>, may be dynamically shared, controlled, and/or utilized through a local collection of
devices. In such embodiments, electronic processor <strong>24</strong> and/or non-transitory storage medium <strong>26</strong>, as diagrammatically illustrated in
<figref idref="DRAWINGS">FIG. 1</figref> by dotted or dashed lines, may be connected within a local network and/or controlled in a network-based storage. For example, the physical components <strong>24</strong>, <strong>26</strong> may
be fully integrated into a device that physically holds the video camera <strong>20</strong>, hand sanitation station <strong>10</strong>, and/or other stated components in
<figref idref="DRAWINGS">FIG. 1</figref>. Such embodiments utilize the electronic processor <strong>24</strong> to connect with the video camera <strong>20</strong> in order to receive real-time video acquired by the video camera or some
type of “video like sensor.” This could receive and process video to identify whether a hand sanitation event occurs and control the annunciator <strong>22</strong> to annunciate a hand hygiene compliance status
among other embodiments. Additionally, or alternatively, the non-transitory storage medium <strong>26</strong> may be separate from the computer or dedicated electronic component implementing the electronic processor
<strong>24</strong>. As noted above, the non-transitory storage medium <strong>26</strong> stores instructions <strong>30</strong>, <strong>32</strong>, <strong>34</strong> that are readable and executable by the
electronic processor <strong>24</strong> to process real time video acquired by the video camera <strong>20</strong> in real time. In various embodiments herein, the non-transitory storage medium can be dynamically
changed, and by local or internet connected containers that serve as a system of structure to share, compute and hold sanitation zone occupancy detection instructions <strong>30</strong>, sanitation detection instructions
<strong>32</strong>, and reporting instructions <strong>34</strong>. Such node-based system could further pool and share resources from local electronic processors <strong>24</strong>, but host those resources via
the internet or other connection, this embodiment would further aid in computational ability for “real time” video and further add the ability for local inference. Such based node system would host and allocate
through the physical <strong>24</strong> electronic processor or the node processing power to execute the detection instructions <strong>30</strong>, sanitation detection instructions
<strong>32</strong>, reporting instructions <strong>34</strong> to, e.g., accumulate data statistics. In such embodiments, the processor <strong>24</strong>, non-transitory storage medium <strong>26</strong>, and/or
related instructions may comprise, or be part of, a cloud based platform, network, or system.</p>
<p id="p-0038" num="0037">In the illustrative example of
<figref idref="DRAWINGS">FIG. 2</figref>, in an operation <strong>50</strong> the hand hygiene compliance enforcement method remains in standby until motion is detected. For example, the video camera <strong>20</strong> may continuously
acquire video and a difference image generated for each successive frame by subtracting the immediately preceding frame (or some earlier frame, e.g. the second-most immediate frame) from the current frame. In the
absence of motion, this difference image should have no content; if motion is occurring then the difference image will have content corresponding to the change in image content caused by the motion. Motion is then
detected if the sum of pixel values over the area of the difference image is greater than some threshold value. This is merely an illustrative example, and other types of camera-based motion detection are contemplated.
As a variant embodiment, it is contemplated to provide a separate motion sensor (not shown) to perform the operation <strong>50</strong>, e.g. a laser beam that is broken to detect passage of a person.</p>
<p id="p-0039" num="0038">Upon detection of motion in the video, process flows to operation <strong>52</strong> where an image is acquired (or, alternatively, the most recent video frame processed in operation <strong>50</strong> is used). At
this stage, an image area <strong>54</strong> is considered, which includes the portion of the image depicting the sanitation station <strong>10</strong>, the door <strong>18</strong>, and closely adjacent area.
In an operation <strong>56</strong>, image analysis is performed on the image area <strong>54</strong> to confirm that the detected motion is due to a person being in the image area <strong>54</strong>. This analysis
may, for example, entail computing a difference image for the area <strong>54</strong> comparing the image from operation <strong>52</strong> with earlier frames for which no motion was detected in operation <strong>50</strong>,
or alternatively computing the difference image respective to a stored reference image of the image area <strong>54</strong>. Any suitable image analysis technique for detecting a person in an image can be employed,
e.g. connectivity analysis may be performed on the difference image to detect object size and aspect ratio which is compared with expected size and aspect ratios for typical human beings. If no person is detected,
flow returns to operation <strong>50</strong> to continue monitoring the video for motion detection.</p>
<p id="p-0040" num="0039">On the other hand, if the operation <strong>56</strong> confirms a person is in the image area <strong>54</strong>, then in an operation <strong>58</strong> it is determined whether a sanitation zone is occupied by
the person. The sanitation zone is an area within the area <strong>54</strong> which corresponds to the person being sufficiently proximate to the sanitation station <strong>10</strong> that the person would be
expected to be using the sanitation station <strong>10</strong> prior to passing through the door <strong>18</strong>. Preferably, the sanitation zone extends from the spatial point where a person might first be
close enough to use the sanitation station <strong>10</strong> to the spatial point where the person has fully passed the sanitation station <strong>10</strong> such that it becomes apparent that the person has
bypassed the sanitation station <strong>10</strong>. Various approaches can be used to determine the occupancy. For example, for successive frames of the real time video, an image of the sanitation zone can be extracted
from the frame and compared with a reference image of the sanitation zone to detect whether the extracted image of the sanitation zone indicates occupation of the sanitation zone. Additionally or alternatively,
for successive frames of the real time video, an image of the sanitation zone can be extracted from the frame. The images of the sanitation zone extracted from the successive frames of the real time video are then
compared to detect frame-to-frame changes that indicate occupation of the sanitation zone. Again, each of these are merely illustrative examples.</p>
<p id="p-0041" num="0040">With conditioned reference to
<figref idref="DRAWINGS">FIG. 2</figref> any portion of the sanitation detection instructions <strong>32</strong>, the reporting instructions <strong>34</strong>, and/or occupancy detection instructions <strong>30</strong> could be broken
out of the their respected “dotted lines”. These lines with the use of a local node of cloud computation can be broken out and separated if desired. Each logic block on
<figref idref="DRAWINGS">FIG. 2</figref> could be exchanged between CPU and/or GPU, and/or a cloud based computation back end to complete the overall cycle or implementation of the instructions described herein for each of the sanitation
detection instructions <strong>32</strong>, the reporting instructions <strong>34</strong>, and/or occupancy detection instructions <strong>30</strong>.</p>
<p id="p-0042" num="0041">Although not explicitly depicted in
<figref idref="DRAWINGS">FIG. 2</figref>, it is to be understood that this process may be performed for successive frames of the real-time video to determine an occupancy time interval over which a sanitation zone imaged by the real time
video is occupied. It is further noted that the successive frames are not necessarily immediately adjacent in the video stream. For example, if the video camera <strong>20</strong> is operating at a standard rate
of 30 fps (i.e. frames-per-second), the successive frames could, for example, be every fifth acquired frame (so that the successive frames actually analyzed would be at a rate of 6 fps). The choice of whether to
analyze every frame, or every second frame, or every fifth frame, or so forth is suitably made to balance processing speed versus temporal resolution. For example, processing every frame provides 30 fps→ 30 Hz temporal
resolution but requires processing speed to handle processing each frame in 1/30<sup>th </sup>of a second; whereas, processing every fifth frame provides coarser temporal resolution of 6 fps→ 6 Hz, but only requires
processing speed to handle processing each frame in ⅙<sup>th </sup>of a second.</p>
<p id="p-0043" num="0042">With continuing reference to
<figref idref="DRAWINGS">FIG. 2</figref>, upon detection at operation <strong>58</strong> of occupancy of the sanitation zone, execution of the sanitation detection instructions <strong>32</strong> by the electronic processor is triggered
in order to determine whether a hand sanitation event occurs based on images of the wash station zone <strong>40</strong> extracted from the real time video acquired during the occupancy time period and to control
the at least one annunciator <strong>22</strong>, <strong>42</strong>, <strong>44</strong> to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs. In
the illustrative example, the detection of whether a hand sanitation event occurs is performed using a Convolutional Neural Network (CNN), however it is to be understood that other artificial intelligence (AI) based
network(s) or algorithms, e.g., comprising a neural network structure, may be used. In various embodiments, CNN <strong>60</strong> may be pre-trained to generate a confidence value as to whether a hand sanitation
event is depicted in an image of the wash station zone using training images of the wash station zone that depict a hand sanitation event and training images of the wash station zone that do not depict a hand sanitation
event. More generally, other supervised, semisupervised, or unsupervised machine learning (ML) classifiers can be used for estimating whether a hand sanitation event is depicted in the image. For example, a machine
learning model, CNN, or other AI model as described herein, may be trained using a supervised or unsupervised machine learning program or algorithm. The machine learning program or algorithm may employ a neural
network, which may be a convolutional neural network, a deep learning neural network, or a combined learning module or program that learns in two or more features or feature datasets in a particular areas of interest.
The machine learning programs or algorithms may also include natural language processing, semantic analysis, automatic reasoning, regression analysis, support vector machine (SVM) analysis, decision tree analysis,
random forest analysis, K-Nearest neighbor analysis, naïve Bayes analysis, clustering, reinforcement learning, and/or other machine learning algorithms and/or techniques. Machine learning may involve identifying
and recognizing patterns in existing data (such as hand sanitation events as determined from pixel or sensor data as received from video cameras or non-camera-based sensors as described herein) in order to facilitate
making predictions for subsequent data (e.g., to predict whether a hand sanitation event has occurred).</p>
<p id="p-0044" num="0043">Machine learning model(s), such as those of a machine learning model, CNN, or other AI model as described herein, may be created and trained based upon example (e.g., “training data,”) inputs or data (which may be termed
“features” and “labels”) in order to make valid and reliable predictions for new inputs, such as testing level or production level data or inputs. In supervised machine learning, a machine learning program operating
on a server, computing device, or otherwise processor(s), may be provided with example inputs (e.g., “features”) and their associated, or observed, outputs (e.g., “labels”) in order for the machine learning program
or algorithm to determine or discover rules, relationships, or otherwise machine learning “models” that map such inputs (e.g., “features”) to the outputs (e.g., labels), for example, by determining and/or assigning
weights or other metrics to the model across its various feature categories. Such rules, relationships, or otherwise models may then be provided subsequent inputs in order for the model, executing on the server,
computing device, or otherwise processor(s), to predict, based on the discovered rules, relationships, or model, an expected output.</p>
<p id="p-0045" num="0044">In unsupervised machine learning, the server, computing device, or otherwise processor(s), may be required to find its own structure in unlabeled example inputs, where, for example multiple training iterations are executed
by the server, computing device, or otherwise processor(s) to train multiple generations of models until a satisfactory model, e.g., a model that provides sufficient prediction accuracy when given test level or
production level data or inputs, is generated. The disclosures herein may use one or both of such supervised or unsupervised machine learning techniques.</p>
<p id="p-0046" num="0045">With reference to
<figref idref="DRAWINGS">FIG. 2</figref>, to generate the training images, video (having pixel data) or sensor information (having sensor data) is taken or collected of various people passing through the sanitation zone who do, or do not,
use the sanitation station <strong>10</strong>. This approach for training the CNN <strong>60</strong> (or other ML classifier) requires collection of the training images or sensor data at the site where the sanitation
station <strong>10</strong> is deployed.</p>
<p id="p-0047" num="0046">Alternatively, the training may use training images or sensor data of a standard wash station zone that depict a hand sanitation event and training images of the standard wash station zone that do not depict a hand
sanitation event. To generate training images or sensor data, video is taken, or sensor data is collected, of various people passing through the sanitation zone of a standard sanitation station (preferably of the
same manufacture and model as the sanitation station <strong>10</strong> at which the compliance enforcement device is to be deployed) who do, or do not, use the standard sanitation station. This approach advantageously
allows for the pre-trained CNN (or other ML classifier) <strong>60</strong> to be shipped with the compliance enforcement device and eliminates the need to perform on-site training of the CNN.</p>
<p id="p-0048" num="0047">Rather than using the CNN <strong>60</strong>, another type of ML classifier could be used, such as another type of artificial neural network (ANN), or a support vector machine (SVM) classifier, or another architecture
such as VGG, ResNET, RCNN, MaskRCNN, MobileNET, Inception family or so forth. In another approach, the CNN <strong>60</strong> is trained to employ Human Pose estimation, in which the CNN <strong>60</strong> is
trained on key points that align to human body parts such as the right elbow, left knee, and so forth. Using a CNN for Human Pose estimation advantageously allows for deriving when a hand sanitation action would
occur. In tests, this approach was significantly more computationally heavy and less accurate, making real time inference difficult. Additionally or alternatively, a local node of cloud computation can be used,
which may allow the systems and/or methods of
<figref idref="DRAWINGS">FIG. 2</figref> to become more feasible. This Human Pose estimation could be further aggregated (e.g., via an aggregation method as described herein), or used in conjunction with other embodiments described herein,
to decide the gate and posture of the person. The ML classifier could also employ a semi- or unsupervised approach. A variable autoencoder or Generative Adversary Network (GAN) could be used to train two competing
networks to train towards creating different hand sanitation depictions. This could then be used as training for a model or inference side comparing the GAN's likelihood of possibility value to compare to currently
acquired images.</p>
<p id="p-0049" num="0048">With continuing reference to
<figref idref="DRAWINGS">FIG. 2</figref>, at an operation <strong>62</strong> for each frame of successive frames of the real time video acquired during the occupancy time period, the CNN is applied to the image of the wash station zone
extracted from the frame in order to generate a confidence value indicative of likelihood that the image of the wash station zone depicts a hand sanitation event. A determination is made of whether a hand sanitation
event occurs in the wash station zone based on the confidence values for the successive frames. This determination may be either (1) a hand sanitation event occurs <strong>64</strong>; or (2) a hand sanitation event
does not occur <strong>66</strong>. At an operation <strong>68</strong>, the annunciator <strong>22</strong>, <strong>42</strong>, <strong>44</strong> is controlled to annunciate a hand hygiene compliance status
based on the determination of whether a hand sanitation event occurs. This process continues until, at an operation <strong>70</strong>, it is determined (via continued execution of the occupancy detection instructions
<strong>30</strong>) that the person walks out of the sanitation zone, thereby ending the occupancy time interval. At an operation <strong>72</strong>, the final outcome (either a hand sanitation event occurred
over the occupancy time interval, or it did not) is determined. Optionally, this may trigger a further annunciation, e.g. if there has been no compliance then the noncompliance light
<strong>22</strong> may be set to a flashing illumination, or if the audio speaker <strong>44</strong> is provided at this point an audible reminder may be issued to use the sanitation station <strong>10</strong>.
The outcome operation <strong>70</strong> also triggers a reset <strong>74</strong> of the occupancy detection instructions <strong>30</strong> to reset the hand hygiene compliance enforcement device for the next
person.
</p>
<p id="p-0050" num="0049">In the illustrative example of
<figref idref="DRAWINGS">FIG. 2</figref>, the annunciator <strong>22</strong>, <strong>42</strong>, <strong>44</strong> is controlled at the operation <strong>68</strong> to annunciate the hand hygiene compliance status in an ongoing fashion
over the course of the person's progress through the sanitation zone. Hence, as the person enters the sanitation zone this is detected at operation <strong>58</strong>, and as initially the person will not have
used the sanitation station <strong>10</strong> the output of operation <strong>62</strong> is no sanitation <strong>66</strong> and the (e.g. red) noncompliance light <strong>22</strong> will be illuminated at
operation <strong>68</strong>. Optionally, this may also trigger an initial audible reminder via the audio speaker <strong>44</strong> (if provided) that the sanitation station <strong>10</strong> should be used
before entering the door <strong>18</strong>. If at some point the person uses the sanitation station <strong>10</strong> then this will be detected at operation <strong>62</strong> causing the output will to switch
to compliance <strong>64</strong> and the noncompliance light <strong>22</strong> will be turned off and the (e.g. green) compliance light <strong>42</strong> (if provided) is turned on. On the other hand, if the
person completes passage through the sanitation zone without using the sanitation station <strong>10</strong> then the outcome operation <strong>70</strong> optionally records the noncompliance and/or optionally
issues a final reminder, e.g. by causing the noncompliance light <strong>22</strong> to flash and/or issuing a final audible reminder via the loud speaker <strong>44</strong>. More generally, in this embodiment
the noncompliance light <strong>22</strong> is controlled to illuminate starting at the beginning of the occupancy time interval and ending at the earlier of (i) a stop time respective to the end of the occupancy
time interval or (ii) a time when it is determined that a hand sanitation event occurs. The stop time is typically set to be a second or so up to a few seconds after the end of the occupancy time interval so as
to remain illuminated until the person has, most likely, passed through the door <strong>18</strong>. This approach has certain advantages, such as providing an immediate reminder via illumination of the noncompliance
light <strong>22</strong> (and the optional audible reminder) that the person should use the sanitation station <strong>10</strong>.</p>
<p id="p-0051" num="0050">In a variant annunciation approach, the audible initial reminder via the loud speaker <strong>44</strong> (if provided) may initially be provided as the person enters the sanitation zone, but the noncompliance light
<strong>22</strong> is not initially activated. Thereafter, as the person passes through the sanitation zone no annunciation is provided (e.g., neither light <strong>22</strong>, <strong>42</strong> is lit, nor
is any audible annunciation output). If at some point the person uses the sanitation station <strong>10</strong> then this will be detected at operation <strong>62</strong> causing the (e.g. green) compliance light
<strong>42</strong> (if provided) is turned on. On the other hand, if the person completes passage through the sanitation zone without using the sanitation station <strong>10</strong> then the outcome operation
<strong>70</strong> illuminates the non-compliance light <strong>22</strong> (for the first time, in this embodiment) to remind the person of his or her noncompliance, and/or issues an audible reminder via the loud
speaker <strong>44</strong>. This approach provides a more delayed response, but has certain advantages. For example, in this embodiment the red light <strong>22</strong> only illuminates if the person passes completely
through the sanitation zone without using the sanitation station <strong>10</strong>. In other words, the red light <strong>22</strong> only illuminates if the person has failed to use the sanitation station <strong>10</strong>.
This embodiment advantageous leverages social pressure to encourage enforcement with the hand hygiene protocol, because if the red light <strong>22</strong> illuminates then any person in the area who sees it will
know the person approaching the door <strong>18</strong> has failed to use the sanitation station <strong>10</strong>.</p>
<p id="p-0052" num="0051">Setup of the hand hygiene compliance enforcement device may be done at some time after the sanitation station <strong>10</strong> is installed, and proceeds as follows. The video camera <strong>20</strong> and annunciator(s)
<strong>22</strong>, <strong>42</strong>, <strong>44</strong> are installed, and the electronic processor <strong>24</strong> and storage <strong>26</strong> are installed and operatively connected with the video
camera <strong>20</strong> and annunciator(s) <strong>22</strong>, <strong>42</strong>, <strong>44</strong>. These connections may be by way of physical wires or cables, or by wireless connections (e.g., Bluetooth™,
WiFi, et cetera). The software <strong>30</strong>, <strong>32</strong>, <strong>34</strong> can be installed, or, can come preinstalled on the storage medium <strong>26</strong>. Next, the area
<strong>54</strong> (see
<figref idref="DRAWINGS">FIG. 2</figref>), the sanitation zone, and the wash station zone <strong>40</strong> (see
<figref idref="DRAWINGS">FIG. 1</figref>) are defined. In one approach, this is done by taking test video frames and displaying them on a computer display, cellphone, or the like. An installer may need to adjust the positioning of the video
camera <strong>20</strong> to ensure it is fully capturing at least the sanitation zone (and hence the wash station zone <strong>40</strong> located inside the sanitation zone). When the video camera <strong>20</strong> is properly positioned, the installer acquires video and displays a frame on the display, and manually (e.g. using a mouse, touch-sensitive screen, or the like) defines boxes in the frame defining the wash station
zone and the larger sanitation zone (and the still larger image area <strong>54</strong>, unless this area <strong>54</strong> coincides with the sanitation zone). In some embodiments, additionally or alternatively,
a touch-sensitive screen, such as a touchscreen display, may be used to display hand sanitation messages. In such embodiments, a touchscreen display may be installed or positioned as an annunciator alone or with
other annunciator(s) described herein, including, for example any one or more of annunciator(s) <strong>22</strong>, <strong>42</strong>, <strong>44</strong>. That is, a touchscreen display may be used as a substitution
to, or addition to, the compliance lights <strong>22</strong> and/or <strong>42</strong>, or audio speaker <strong>44</strong>. In some embodiments, a touchscreen display may include a digital green and red lights
or similar animations or visual feedback. Additionally, or alternatively, a touchscreen display could hold animations and visuals combined with verbal or audio sound for prompting positive behavior (e.g., hand washing
compliance). Additionally, or alternatively, hand sanitations messages can include one or more positive messages instructing a user to wash his or hands and keep safe. Additionally, or alternatively, hand sanitations
messages may further comprise an animation, clock, or other depictive functional and/or non-functional graphics corresponding to a user's hand washing session or compliance. In various embodiments, a touchscreen
display may be configured to turn on when a person passes a sanitation zone in order to encourage the person to sanitize his or her hands. In some embodiments, the touchscreen display may allow the user to input
a command (e.g., a verbal or physical command) to indicate compliance. Such features may assist users that may be forgetful to use the hand hygiene compliance enforcement device (e.g., at critical locations, including
entry into a sensitive area), which can increase effective sanitation rates overall.</p>
<p id="p-0053" num="0052">In various embodiments, the annunciator(s) <strong>22</strong>, <strong>42</strong>, <strong>44</strong>, and/or a display or touchscreen display may be used for sanitation “information sharing.” Information sharing
comprises sharing information within a proximity to a hand hygiene compliance enforcement device, or portion thereof. The information may be shared, e.g., via a touchscreen display or display viewable by people)
in the vicinity of the hygiene compliance enforcement device, or portion thereof. Such public display would invoke “information sharing” to encourage individuals, who are not bound (e.g. legally bound) by compliance,
such as hospital visitors and others, to nonetheless engage in positive hand sanitation behaviors. For example, unlike, doctors, nurses, and hospital staff, or restaurant workers or lab workers, individuals not
required to comply with hand sanitation regulations may fail to comply with hand sanitation, without social pressure as caused by an “information sharing” annunciator, such as a display screen showing a given individual's
hand sanitation compliance. By centralizing, or making available, the dissemination of simple information at or near a hand hygiene compliance enforcement device, or portion thereof, information sharing on display
devices (or other annunciator(s) as described herein) may cause persons in public spaces that incorporate these device(s) to comply with sanitation regulations, thereby increasing hand sanitation compliance overall.
For example, in an embodiment, a touchscreen display may be used to share information about a given hospital's hand sanitation compliance, e.g., time, rating, and positive reports regarding hand sanitation that
the hospital would could then share with customers, patients, etc. By sharing such information on a display or screen, near a hand hygiene compliance enforcement device, or portion thereof, such information would
further encourage positive hand sanitation protocol(s). As a further example, a restaurant may use a screen to share information about sanitation compliance, including the restaurant's use of menus, QR codes, or
other touchless payment types that demonstrate restaurant compliance with certain disease protocols (e.g., COVID-19). As a further example, and in general, public spaces, such as museums, co-working spaces, or locations
where many people gather, could have installed an information sharing system, as described herein, to not only share compliance “guidelines,” but also to determine who is (and how many people are) are complying,
whether it be outside or inside of legal compliance. Such embodiments provide the advantage of increasing compliance because, as demonstrated in these examples, such embodiments may lead to reduced “sanitation leakage,”
especially in places where a lack of sanitation could occur because of a typical lack of tracking or monitoring. Accordingly, use of a hygiene compliance enforcement device, or portion thereof, in such locations
may cause increased sanitation, and related statistics, and better metrics overall.</p>
<p id="p-0054" num="0053">In various embodiments, information sharing data or information may be stored in statistics database <strong>80</strong>, or in other computer memorie(s) associated with electronic processor such as a non-transitory
storage medium <strong>26</strong>. The information sharing data or information may be reported as described herein, via reporting instructions <strong>34</strong> and/or compliance report generator <strong>82</strong>.
In further embodiments, a hand hygiene compliance enforcement device may be configured to control information sharing through an application (app), which may comprise one or more computing instructions stored in
non-transitory storage medium <strong>26</strong>, and configured for execution on electronic processor <strong>24</strong>. The application may be a native app, such as an APPLE iOS app or a GOOGLE ANDROID app.
The information sharing app could comprise an information sharing interface as implemented or displayed on a mobile device (e.g., smart phone or tablet) or could comprise a webpage of a website. The information
sharing interface could provide data or statistics for information sharing, as described herein, to a display screen, e.g., via reporting instructions <strong>34</strong> and/or compliance report generator <strong>82</strong>.
In this way, such information sharing data or statistics may be provided to a hygiene compliance enforcement device or, additionally or alternatively, to a group of devices (e.g., mobile devices) via one or more
corresponding information sharing interface(s). By using the information sharing interface(s), a customer or any person that wants to view or share information regarding hand sanitation compliance could access an
information sharing interface, e.g., by downloading an app, uploading an image, uploading a video, or sharing text content, to provide and/or monitor hand sanitation compliance of a particular zone or location,
even a location remote to the hygiene compliance enforcement device or the user.</p>
<p id="p-0055" num="0054">In some embodiments, information sharing and display, e.g., via information sharing interface(s) as described herein, may comprise displaying notifications, marketing, advertisements, or the like. In such embodiments,
“marketing verification inference analysis” may be provided as, or with, hand sanitation messages or other notifications as described herein. In particular, tracking marketing inferences in a physical marketing
environment is typically difficult for marketers. Through hand sanitation compliance, as described herein, however, tracking inference and physical presence of individuals could be implemented by use of hand hygiene
compliance enforcement devices as described herein. For example, display of advertisement(s) and related tracking as to one or more marketing inference(s) across one or more users may be stored on statistics database
<strong>80</strong>, or in other computer memorie(s) associated with electronic processor such as a non-transitory storage medium <strong>26</strong>. The display and/or tracking may be implemented though the hand
sanitation techniques as described herein. For example, the annunciators and related imaging analysis, such as provided via a display and/or touchscreen display, which may be in a proximity to a hand sanitation
zone and/or device as described herein, may be used for such tracking.</p>
<p id="p-0056" num="0055">For example,
<figref idref="DRAWINGS">FIG. 4</figref> illustrates a flowchart of the hand hygiene protocol enforcement system of
<figref idref="DRAWINGS">FIG. 1</figref> used to provide marketing based information sharing. In the example of
<figref idref="DRAWINGS">FIG. 4</figref>, at block <strong>202</strong> a hand hygiene protocol enforcement system may determine whether a person is in a field of view (FOV) of a camera or non-camera based sensor as described herein. If
no person is detected, at block <strong>204</strong>, the hand hygiene protocol enforcement system may standby until motion is detected. The hand hygiene protocol enforcement system may then continue to acquire
images (<strong>206</strong>) of the related environment, including the view of the sanitation device, doorways, etc. (<strong>208</strong>). If a person is detected, at block <strong>210</strong>, the hand hygiene
protocol enforcement system determines that a person is in the sanitation zone. The example of
<figref idref="DRAWINGS">FIG. 4</figref> comprises a “marketing zone,” which is an area near or around, and is typically larger than, the sanitation zone of the hand hygiene protocol enforcement system (e.g., the hand hygiene compliance
enforcement device) as described herein. The marketing zone defines an area from which individuals can view, e.g., a touchscreen or display screen, and potentially be shown or served an advertisement. In some embodiments,
the marketing zone may be a pass-by area that can serve or show advertisements in a vicinity of the hand hygiene protocol enforcement system (e.g., the hand hygiene compliance enforcement device). In instances (<strong>212</strong>)
where the hand hygiene protocol enforcement system determines that a person is not in the sanitation zone, but is in a marketing zone, a confirmation of a marketing inference (
<strong>214</strong>), e.g., presence of the person, can be determined and tracked by the hand hygiene protocol enforcement system. Alternatively, in instances (<strong>220</strong>) where a person is in the sanitation
zone, then an annunciator, such as a display (<strong>222</strong>), or green or red light (e.g., <strong>22</strong> and/or <strong>42</strong>, respectively) (<strong>224</strong>), may be used to confirm that
a person in a proximity to the hand hygiene protocol enforcement system during a time when a marketing advertisement was shown. In this way, for example, a screen or display may encourage hand sanitation, and at
the same time, may be used to provide or show advertisements. In such embodiments, a portion, e.g., small partition, of a screen or display may be used for advertising space. In other embodiments, utilizing multiple
screens or displays may be used, where one screen may show animations, green or red lights, or other hand sanitation related information and/or graphics, but where another screen may show adertisements. In such
embodiments, either way, occupation of either the marketing zone or sanitation zone would cause a confirmation of an individual being present for the display of an advertisement. Such information, as to one or more
marketing inference(s), may be tracked and stored on statistics database <strong>80</strong> (<strong>216</strong>), or in other computer memorie(s) associated with electronic processor such as a non-transitory
storage medium <strong>26</strong>. Tracking may allow advertisements to be rotated or changed over time (<strong>218</strong>) for specific users and/or specific locations.</p>
<p id="p-0057" num="0056">These advertisements can be controlled through a hand hygiene marketing platform, which may be implemented on the hand hygiene compliance enforcement device described herein. The platform can comprise a sanitation marketing
application that connects marketers through the hand hygiene compliance enforcement device. The hand hygiene marketing platform may be configured to control marking through a marketing application (app), which may
comprise one or more computing instructions stored in non-transitory storage medium <strong>26</strong>, and configured for execution on electronic processor <strong>24</strong>. The hand hygiene marketing platform
could comprise a marketing interface as implemented or displayed on a mobile device (e.g., smart phone or tablet) or could comprise a webpage of a website, or native app as described herein. The marketing interface
provides data or statistics for providing advertisement, as described herein, to a display screen, e.g., via reporting instructions <strong>34</strong> and/or compliance report generator
<strong>82</strong>. In this way, such advertisements may be provided to a hygiene compliance enforcement device or, additionally or alternatively, to a group of devices (e.g., mobile device) via one or more corresponding
display or screen interface(s).</p>
<p id="p-0058" num="0057">In various contemplated approaches, a CNN or other ML architecture can be used to define the wash station and sanitation zones, e.g. the CNN <strong>60</strong> could be another type of ANN, or an SVM classifier, or
could be trained to employ Human Pose estimation, or so forth. Finally, as previously noted, the CNN (or other ML classifier) <strong>60</strong> is pretrained to generate the confidence value using training images
of the wash station zone or a standard wash station zone that depict a hand sanitation event and training images of the wash station zone or the standard wash station zone that do not depict a hand sanitation event.
If training images of a standard wash station zone are used, then the pretrained CNN <strong>60</strong> comes preloaded with the sanitation detection software module <strong>32</strong>. On the other hand, if training
images of the wash station zone <strong>40</strong> of the sanitation station <strong>10</strong> are to be used in the training, then the installer collects the requisite training data. Preferably, the training
data are collected using a number of different people capturing the expected range of human variation (height, weight, skin tone, et cetera) commensurate with the full range of people expected to be using the sanitation
station <strong>10</strong>. In this latter case, the loaded software includes the CNN training software so that the CNN <strong>60</strong> can be trained on the just-collected training images. Finally, the installer
tests the compliance enforcement system using several test runs (again, preferably with people of different heights, weights, skin tones, et cetera) to verify accuracy of the compliance enforcement system.</p>
<p id="p-0059" num="0058">With continuing reference to
<figref idref="DRAWINGS">FIG. 2</figref>, optionally, the reporting instructions <strong>34</strong> are provided to generate statistical compliance data. To this end, a statistics database <strong>80</strong> collects the outcomes
<strong>72</strong>, and an operation <strong>82</strong> generates a compliance report based on inputs received from a user at the electronic processor <strong>24</strong>, e.g., the user may select via a keyboard,
mouse, or other user input device to generate a compliance report for the sanitation station <strong>10</strong> over some user-selected time interval, and optionally may further narrow the report to a certain work
shift or so forth. The hand hygiene compliance enforcement device does not identify the persons whose compliance is monitored—accordingly, the generated compliance report is anonymous. However, it is contemplated,
for some embodiments, to group the outcomes <strong>72</strong> in various ways to provide more targeted statistics. For example, the outcomes <strong>72</strong> can be grouped based on the ward or other locational
groupings. The outcomes <strong>72</strong> can also be grouped based on the time of event. For example, if the hospital runs three work shifts with defined work shift change times, then the statistics can be grouped
by work shift to assess compliance for each work shift. The outcomes <strong>72</strong> can additionally or alternatively be grouped by worker type, based on analysis of the image area <strong>54</strong>. For
example, different types of hospital workers (e.g. nurses, orderlies, doctors) can be distinguished based on types of uniform (or lack of a uniform). Alternatively, if hospital workers are required to wear radio
frequency identification (RFID) badges, then an RFID reader may be incorporated into the hand sanitation compliance monitoring system to identify classes of workers. In this way, compliance can be assessed for different
types of workers. As a further variant on this, in some cases visitors may not be required to perform hand sanitation before entering a ward. In this case, analysis of the image area <strong>54</strong> can be used
to identify visitors (e.g., based on lack of uniform, or by detecting a “Visitor” badge in the image area <strong>54</strong> or, if the “Visitor” badge includes an RFID tag, by incorporating an RFID tag reader
to identify the person as a visitor based on the tag reading) and discard outcomes for visitors. (This information may also optionally be fed to the notification operation <strong>68</strong> in order to avoid annunciating
a hand hygiene compliance status in the case of visitors). Conversely, if the hospital employs RFID badges for employees but not for visitors, then any person identified in the sanitation zone in operation <strong>58</strong> who does not also register on the RFID tag reader may be designated as a visitor and the corresponding outcome discarded.</p>
<p id="p-0060" num="0059">With continuing reference to
<figref idref="DRAWINGS">FIG. 2</figref>, in a variant embodiment the reporting instructions <strong>34</strong> further implement facial recognition <strong>84</strong>, or alternatively issue calls to a facial recognition module, whereby
video frames acquired over the course of the occupancy time period are processed by the facial recognition to compare against a facial database in an effort to identify the person. If the person is recognized, then
the outcome stored in the statistics database <strong>80</strong> is annotated with the person's identification. In this case, the compliance report generated at the operation <strong>82</strong> may be sorted by
name, so that persons with poor compliance can be individually identified. Optionally, in this variant embodiment it is also contemplated for the annunciator, if embodied at least in part as a loud speaker <strong>44</strong> or display, to identify the individual when annunciating the hand hygiene compliance status at the outcome <strong>70</strong>. In a variant approach, if the hospital employs an RFID badging system then an RFID
reader may be incorporated into the hand sanitation compliance monitoring system to identify individuals.</p>
<p id="p-0061" num="0060">With reference now to
<figref idref="DRAWINGS">FIG. 3</figref>, an illustrative embodiment is described of the operation <strong>62</strong> in which, for each frame of successive frames of the real time video acquired during the occupancy time period, the CNN
(or other ML classifier) <strong>60</strong> is applied to the image of the wash station zone extracted from the frame in order to generate a confidence value indicative of likelihood that the image of the wash
station zone depicts a hand sanitation event. A first frame <strong>100</strong> is received from the video camera <strong>20</strong>. In an operation <strong>102</strong>, the image of the wash station zone <strong>40</strong> is extracted from the frame <strong>100</strong>, and may optionally be further processed, for example by pixel resampling to modify the spatial resolution. For efficient processing, the extracted image of the wash
station zone is preferably relatively small. In one illustrative example, the video frame <strong>100</strong> is 1280×720 pixels in size (i.e., 1280 by 720 pixels), and the extracted image of the wash station zone
(after optional pixel resampling) is on the order of 72×72 pixels (i.e., 72 by 72 pixels) to 84×84 pixels (84 by 84 pixels) in size, although larger or smaller sizes are contemplated. In some embodiments, the image
of the wash station zone extracted in the operation <strong>102</strong> (including optional resampling) has pixel dimensions length by width, i.e., L×W (L by W) where L is 100 pixels or less and W is 100 pixels
or less. This has two advantages. First, the relatively small number of pixels provides for efficient processing, which facilitates real-time analysis of the video on a frame-by-frame basis. Second, and counterintuitively,
it is expected that pixel resampling to a coarser resolution having a smaller number of pixels (e.g. in the 100×100 pixel range or less) will improve signal-to-noise ratio (SNR) in assessing confidence that a hand
sanitation event has occurred. This is because although the resampling to a coarser resolution inherently entails loss of information, the operation of resampling to a coarser resolution also operates to smooth
the imaging data as higher spatial frequency components are suppressed or removed. This higher spatial frequency content is typically dominated by noise, rather than being useful signal content. When looking at
a model that determines whether a hand sanitation event occurs or does not in a video frame, it might be expected that utilizing a larger and higher resolution image for the model would lead to higher classification
accuracies. However, it is recognized herein that this is not the case. The reason a smaller sanitation zone is more accurate and computationally efficient for recognizing a hand sanitation event can be understood
as follows. When applying a CNN or other ML classifier to attempt to learn to detect an outcome (here a hand sanitation event), the classifier is trained to find patterns in the pixels that correlate with the outcome.
In other words, it attempts to group the pattern in the picture of pixels by the label that was given. The CNN or other ML classifier does not know what specific patterns it associates with the outcome, or whether
it is correct. Therefore, when the classifier is looking at finding patterns in the associated outcome or action it is desirable to give the classifier an image with clean patterns and low noise to the greatest
extent possible. In addition to accuracy, computing on a wash station zone with a smaller set of pixels (and hence less information) leads to greater computational efficiency. To achieve this goal the system leverages
two main steps (i) cropping the image to the wash station zone, and (ii) smoothing that preserves image gradients. These steps are preferably optimized to keep the optimal amount of pertinent information while cutting
incidental patterns. It should be noted that the cropping may be omitted if the computational capacity of the electronic processor <strong>24</strong> is sufficient to process the frames without cropping at sufficient
speed (e.g., within the frame rate of the video, i.e. each frame should be processed before the next frame is acquired and presented for processing). Additionally or alternatively, the smoothing may optionally be
omitted.
</p>
<p id="p-0062" num="0061">The region around a wash station is typically busy with nurses and doctors rushing from room to room and ward to ward. When looking at a full video frame (e.g., for 720 p video a frame has a height of 720 pixels and
a width of 1,280 pixels) there are many different patterns from the various moving persons. If the CNN <strong>60</strong> receive full frames (e.g. 720×1280 pixel frames of 720 p video) then detecting a difference
between a full frame capturing a hand sanitation event versus a full frame that does not capture a hand sanitation event, the CNN <strong>60</strong> will see many patterns and correlations that are not necessarily
relevant to the difference outcome. Incorrect correlation of a non-relevant pattern to a relevant outcome pattern can lead to bias and inaccuracies. By cropping the image processed by the CNN
<strong>60</strong> to the area around the hand sanitation station, the CNN is fed the essential information and patterns relevant to detecting a hand sanitation event. However, if taken too far this can also lead
to an inaccurate classifier. When the size of the crop is reduced, relevant patterns may be removed. The relevance of these removed patterns may not be apparent to the designer of the classifier, but would be useful
input for improving the accuracy of the classifier. By way of a specific example, only looking at the sanitation zone may cut out a window or other reflective surface that the designer might not recognize to be
relevant for detecting a hand sanitation event, but that might in fact improve the classifier accuracy. Hence, the size of the wash station zone <strong>40</strong> should preferably include all important relevant
patterns, while reducing or eliminating non-relevant patterns. In addition to accuracy, computing on less pixels increases overall system speed.</p>
<p id="p-0063" num="0062">In addition to design of the pixel dimensions of the wash station zone <strong>40</strong>, another way to improve classifier accuracy is to provide the classifier <strong>60</strong> with an image having clean patterns
and low noise to the extent possible. This can be achieved by smoothing without losing gradients containing relevant information in the image. Smoothing is a computer vision mechanism based on kernel mathematics
to reduce or eliminate high-frequency, outlier heavy aspects of an image. An approximating function should be used that captures important patterns in the data, while leaving out noise or other fine-scale structures
that might otherwise be incorrectly matched to patterns indicative of a hand sanitation event. The sample array values of the image are modified so the individual points are reduced and the points that are lower
than the adjacent points are increased leading to a smoother signal and less variation. Therefore, a sample series of data points are taken from the image, and a filter or kernel of various sizes and aggregation
methods is applied and repeatedly iterated through the image to output a smoothed image. By way of a non-limiting illustrative example, a linear filter may be used for smoothing, which determines the output pixels
by a simple weighted value from the surrounding pixels of various size. This is represented, for example, by the below Equation (1):</p>
<p id="p-0064" num="0063">
<maths id="MATH-US-00001" num="00001"> <math overflow="scroll"> <mtable> <mtr> <mtd> <mrow> <mrow> <mi>O</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <mi>r</mi> <mo>,</mo> <mi>g</mi> <mo>,</mo> <mi>b</mi> </mrow> <mo>)</mo> </mrow> </mrow> <mo>=</mo> <mrow> <munder> <mo>∑</mo> <mrow> <mi>r</mi> <mo>,</mo> <mi>g</mi> <mo>,</mo> <mi>b</mi> </mrow> </munder> <mo></mo> <mrow> <mrow> <mi>f</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <mrow> <mi>r</mi> <mo>+</mo> <mi>k</mi> </mrow> <mo>,</mo> <mrow> <mi>g</mi> <mo>+</mo> <mi>l</mi> </mrow> <mo>,</mo> <mrow> <mi>b</mi> <mo>+</mo> <mi>k</mi> </mrow> </mrow> <mo>)</mo> </mrow> </mrow> <mo></mo> <mrow> <mi>h</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <mi>k</mi> <mo>,</mo> <mi>l</mi> <mo>,</mo> <mi>h</mi> </mrow> <mo>)</mo> </mrow> </mrow> </mrow> </mrow> </mrow> </mtd> <mtd> <mrow> <mo>(</mo> <mn>1</mn> <mo>)</mo> </mrow> </mtd> </mtr> </mtable> </math> </maths>
</p>
<p id="p-0065" num="0064">As shown in Equation (1), the output pixels O(r,g,b) in red, green, and blue, respectively, is equal to the sum of the kernel h(k,l, h) multiplied by the weighted sum of the input pixels f(r+k, g+l, b+k), as expressed
in Equation (1). The kernel is, in this example, a matrix with a fixed size 15×15, which looks at the 15 pixels to consider in Equation (1). Therefore, the kernel is just a coefficient to the filter or the “window
size” and the filter is a sliding “window” that goes across portions of the image to output different sized and averaged array tensors or “smoother image.” In other words, this image transformation is applied to
each pixel in the image to achieve a different state. The goal of this operation is to smooth the image, which can reduce the noise. However, the smoothing can also smooth away the edges and important information
and patterns that are relevant for detecting a hand sanitation event. Hence, over-smoothing can lead to an inaccurate classifier and reduced real time accuracy. To address this, the smoothing operation may optionally
incorporate an Image Gradient to smooth away noise while retaining the relevant patterns. This problem is solved by looking at the rate of change in the pixels to determine important edges and features. By grouping
pixels that are close to one another in value and close to one another in spatial location and leveraging Laplacian, Robers or Prewitt operations these “edges” can be found. In other words, these operations are
designed to automatically limit the smoothing at “edges” in images by highly polarized pixel values. Therefore, the gradients build on the smoothing operation to be defined as Bilateral filtering, as expressed,
for example, by Equation (2):</p>
<p id="p-0066" num="0065">
<maths id="MATH-US-00002" num="00002"> <math overflow="scroll"> <mtable> <mtr> <mtd> <mrow> <mrow> <msup> <mi>I</mi> <mi>filtered</mi> </msup> <mo></mo> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> </mrow> <mo>=</mo> <mrow> <munder> <mo>∑</mo> <mrow> <mrow> <mi>x</mi> <mo></mo> <mstyle> <mspace width="0.3em" height="0.3ex"></mspace> </mstyle> <mo></mo> <mn>1</mn> </mrow> <mo>∈</mo> <mi>Ω</mi> </mrow> </munder> <mo></mo> <mrow> <mrow> <mi>I</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <mi>x</mi> <mo></mo> <mstyle> <mspace width="0.3em" height="0.3ex"></mspace> </mstyle> <mo></mo> <mn>1</mn> </mrow> <mo>)</mo> </mrow> </mrow> <mo></mo> <mrow> <mi>Fr</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <mo></mo> <mrow> <mrow> <mi>I</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <mi>x</mi> <mo></mo> <mstyle> <mspace width="0.3em" height="0.3ex"></mspace> </mstyle> <mo></mo> <mn>1</mn> </mrow> <mo>)</mo> </mrow> </mrow> <mo>-</mo> <mrow> <mi>I</mi> <mo></mo> <mrow> <mo>(</mo> <mi>x</mi> <mo>)</mo> </mrow> </mrow> </mrow> <mo></mo> </mrow> <mo>)</mo> </mrow> </mrow> <mo></mo> <mrow> <mi>Gs</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <mo></mo> <mrow> <mrow> <mi>x</mi> <mo></mo> <mstyle> <mspace width="0.3em" height="0.3ex"></mspace> </mstyle> <mo></mo> <mn>1</mn> </mrow> <mo>-</mo> <mi>x</mi> </mrow> <mo></mo> </mrow> <mo>)</mo> </mrow> </mrow> </mrow> </mrow> </mrow> </mtd> <mtd> <mrow> <mo>(</mo> <mn>2</mn> <mo>)</mo> </mrow> </mtd> </mtr> </mtable> </math> </maths>
</p>
<p id="p-0067" num="0066">As shown in Equation (2) above, I is the original input image, x is the coordinates of the current pixel, Ω is the “isolator window” centered in x, Fr is the range kernel for smoothing differences in intensities, and
Gs is the spatial kernel for smoothing difference in coordinates. So by utilizing the rate of change of the two different smoothing operations “un-important” pixels can be smoothed while bringing out “important”
pixels. This leads to a pixel window that is optimized for the most relevant patterns to show. This allows a greater chance for the classifier training to attach correct correlation to outcomes, while still allowing
for the unpredictable correlations to be present. Therefore these operations affect the training and system side positively. First off, training on higher accurate data and less physical information increases training
accuracy, iteration speed and overall computation efficiency. On the inference side cropping, smoothing and gradients allow for less pixels to be consumed which allows for real-time monitoring to be possible, while
increasing inference accuracy. The described bilateral filtering is merely an illustrative example, and other types of filtering could be employed to facilitate locating the sanitation zone, for example edge-detection
filtering to enhance the strongest edges in the image.</p>
<p id="p-0068" num="0067">The same down-sampling operations that encompass the smoothing and image gradients can be up-sampled. This means that for a blurry or smaller pixel window the image processing can attempt to make a sharper “better”
image. This is done by finding the inverse of the functions set forth above. Therefore, the sanitation zone could be up-sampled or down-sampled to be any pixel range, suitably chosen to optimize accuracy of the
classifier by avoiding losing too much relevant information or adding too much noise.</p>
<p id="p-0069" num="0068">As noted previously, the wash station zone <strong>40</strong> should coincide with the spatial region within which the hand sanitation is expected to be performed by a person using the hand sanitation station
<strong>10</strong>. During installation, the video camera <strong>20</strong> is positioned to capture the sanitation zone, and the installer delineates the area of a frame that depicts the wash station zone
<strong>40</strong>; accordingly, the extraction of the image of the wash station zone at operation <strong>102</strong> merely entails extracting that pre-delineated area of the frame. The image may optionally
entail further preprocessing to condition the image of the wash station zone for input to the CNN <strong>60</strong>. For example, in the illustrative embodiment at an operation <strong>103</strong> the extracted
image of the wash station zone is converted to RGB (i.e. red-green-blue) 3×3 channel arrays (tensor). More generally, the image of the wash station zone extracted from the frame <strong>100</strong> may be converted
to a tensor of channel arrays wherein each channel array represents a color channel of the image of the wash station zone. The color channels may be RGB color channels as in the illustrative example, or may be color
channels of another color space such as LAB, YCrCb, HSV, or so forth. In other embodiments, it is contemplated to replace (or augment) the RGB, YCrCb, HSV, or other color video feed with depth information. A depth
camera could be used or a regular camera could be used utilizing a depth map function. The depth of a pixel could be found using disparity=x−x=Bf/Z. Utilizing this, the depth of all pixels in an image can be derived.
However, it is expected that employing depth information would increase the computational load and may lead to an image with less distinctive sanitation attributes. Other types of imaging, such as thermal imaging
could also be used. In an operation <strong>104</strong>, the CNN <strong>60</strong> is applied to the extracted (and optionally preprocessed) image of the wash station zone to generate a confidence value <strong>106</strong> indicative of likelihood that the image of the wash station zone depicts a hand sanitation event. In this regard, it should be noted that the training images of the wash station zone <strong>40</strong> or of a
standard wash station zone used to (pre-)train the CNN (or other ML classifier) <strong>60</strong> should have been processed using the same extraction/preprocessing
<strong>102</strong>. For example, if the preprocessing <strong>102</strong> includes converting to RGB 3×3 channel arrays prior to input to the CNN <strong>60</strong> at operation <strong>104</strong>, then the
training images should also have been converted to RGB 3×3 channel arrays prior to use in training the CNN.</p>
<p id="p-0070" num="0069">At an operations <strong>107</strong>, <strong>108</strong>, the confidence <strong>106</strong> is assessed to determine whether the frame depicts an occurrence of a hand sanitation event. For example, if the confidence
<strong>106</strong> is higher than some threshold (e.g., an x-value), then it is determined that a hand sanitation event occurred <strong>64</strong>, and the process of
<figref idref="DRAWINGS">FIG. 3</figref> stops. In another approach (illustrated), a multiple-criteria decision making or analysis (MCDM or MCDA) <strong>107</strong> is employed to assess whether the confidence <strong>106</strong> to
determine whether the frame depicts an occurrence of a hand sanitation event. MCDM is a process of evaluating multiple conflicting criteria with different weights for decision making. In the instant case, the MCDM
<strong>107</strong> can be an aggregation method, which in one approach involves reading the confidence metrics from the model for a certain value and make a decision based on that. In other words, the binary decision
of whether an event occurred is made based on the events that occurred and their weights. Many aggregation-based MCDM approaches are known, including Geometric Median (GM), Non-Conventional Matching (NCM), The LogStructed
Merge-Tree (LSM-Tree), Singular Value Decomposition Method (SVDM), Ordered Weighted Averaging (OWA), Dynamic AHP (Analytic Hierarchy Process), ELETRE TRI, Technique for Order of Preference by Similarity to Ideal
Solution that is based on Bayesian vector networks, which is based on the distance of each alternative from the ideal solution (TOPSIS) with interval data, Fuzzy Analytic Hierarchy Process (FAHP) which uses pairwise
comparison metrics to than aggregate through fuzzy numbers, Fuzzy MCDM, which is based on trapezoidal fuzzy numbers through a TOOPSIS degree method, Multi-Attribute Utility (MAU), and variants of these. As an illustrative
example, an approach employing the OWA aggregation operator is described. This OWA gives the technique for directly aggregating “uncertain” information with “uncertain” weights. The uncertainty regards what action
is being committed (hand sanitation event or no hand sanitation event) and uncertainty weights indicate how confident the model is about the information. An OWA is defined, for example, as in Equation (3):</p>
<p id="p-0071" num="0070">
<maths id="MATH-US-00003" num="00003"> <math overflow="scroll"> <mtable> <mtr> <mtd> <mrow> <mrow> <mi>ϕ</mi> <mo></mo> <mrow> <mo>(</mo> <mi>a</mi> <mo>)</mo> </mrow> </mrow> <mo>=</mo> <mrow> <mrow> <mi>ϕ</mi> <mo></mo> <mrow> <mo>(</mo> <mrow> <msub> <mi>a</mi> <mn>1</mn> </msub> <mo>,</mo> <mi>…</mi> <mo></mo> <mstyle> <mspace width="0.8em" height="0.8ex"></mspace> </mstyle> <mo>,</mo> <msub> <mi>a</mi> <mi>n</mi> </msub> </mrow> <mo>)</mo> </mrow> </mrow> <mo>=</mo> <mrow> <munderover> <mo>∑</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </munderover> <mo></mo> <mrow> <msub> <mi>ω</mi> <mi>i</mi> </msub> <mo></mo> <msub> <mi>a</mi> <mrow> <mi>σ</mi> <mo></mo> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> </mrow> </msub> </mrow> </mrow> </mrow> </mrow> </mtd> <mtd> <mrow> <mo>(</mo> <mn>3</mn> <mo>)</mo> </mrow> </mtd> </mtr> </mtable> </math> </maths>
</p>
<p id="p-0072" num="0071">In the above Equation (3), ϕ is a mapping specified as ϕ:<sup>n</sup>→ which has weights defined as w=(ω<sub>1</sub>, . . . , ω<sub>n</sub>)<sup>T </sup>where the weights or confidence readings, ω<sub>1</sub>, . . .
, ω<sub>n</sub>, are between 0-1 and the sum of the weights is equal to one, i.e. w<sub>i</sub>∈[0,1] for all i=1, . . . ,n, and Σ<sub>i=1</sub><sup>n</sup>w<sub>i</sub>=1. Using Equation (3), an aggregation method
is formed that helps connect the linguistic outputs (here hand sanitation and no hand sanitation) with unsure values and weights. This is merely one illustrative aggregation method, and other simpler or more complex
aggregation methods can be utilized, or other approaches as already mentioned.</p>
<p id="p-0073" num="0072">On the other hand, if the frame is not sufficient to determine that a hand sanitation event occurred (i.e., if the confidence does not meet the threshold x-value), then in an operation <strong>110</strong> the result
is aggregated with confidence values for earlier frames. At an operation <strong>112</strong>, it is determined whether the person is still in the sanitation zone (that is, it is determined whether the occupancy
time interval is continuing or has ended). If the person is still in the sanitation zone then a next frame <strong>114</strong> of the video (which may be the immediately next frame in the video stream providing
a 30 Hz time resolution at 30 fps, or the fifth next frame providing a 6 Hz resolution at 30 fps, or so forth) is obtained and processed by the operations <strong>102</strong>, <strong>104</strong>,
<strong>108</strong>, <strong>110</strong> as described. This continues until the occupancy time interval ends, at which time the operation <strong>112</strong> will determine the person is no longer in the sanitation
zone. At this point, in an operation <strong>116</strong> the aggregation of confidences from the operation <strong>110</strong> is examined to determine whether it can be concluded that no hand sanitation event
occurred over the occupancy time interval. This can employ various statistical approaches. In one approach, the largest confidence from any of the processed frames is compared with a threshold y-value, and if it
is below this threshold y-value then it is determined that no hand sanitation event occurred <strong>66</strong>, and the process of
<figref idref="DRAWINGS">FIG. 3</figref> stops. On the other hand, if it is determined at the operation <strong>110</strong> that the largest confidence is above the threshold y-value (but below the threshold x-value since otherwise processing
would have stopped at the operation <strong>108</strong> due to a positive determination <strong>64</strong>), then the result is ambiguous and the resulting hand hygiene compliance status is deemed to be indeterminate.
This may in some embodiments be treated as a noncompliance so as to provide the final reminder to the person before he or she passes through the door <strong>18</strong>; or, the indeterminate result may be treated
differently from either a positive determination of compliance or positive determination of noncompliance, e.g. an indeterminate result may cause no annunciated output. In the optional statistics database <strong>80</strong>,
the indeterminate result is preferably recorded as indeterminate, so that the resulting compliance report identifies the fraction of times the compliance enforcement device was unable to make a positive determination.</p>
<p id="p-0074" num="0073">In the operations <strong>108</strong>, <strong>116</strong>, the respective thresholds x-value and y-value may be suitably optimized during the training of the CNN (or other ML classifier) <strong>60</strong>, e.g.
being an additional two parameters to be optimized in addition to the activation weights and/or other parameters of the CNN <strong>60</strong>.</p>
<p id="p-0075" num="0074">The illustrative embodiment of the operation <strong>62</strong> presented with reference to
<figref idref="DRAWINGS">FIG. 2</figref> is merely an illustrative example, and numerous variants are contemplated. Other types of artificial neural networks (ANNs) can be employed in place of the illustrative CNN <strong>60</strong>, or
even more generally other types of classifiers may be employed in place of the CNN <strong>60</strong>, such as a support vector machine (SVM) classifier. In general, a machine learning (ML) classifier (including
CNN, ANN, SVM, and other ML classifiers) may be used. Moreover, other types of analyses may be employed. For example,
<figref idref="DRAWINGS">FIG. 3</figref> illustrates an optional color classifier analysis operation <strong>118</strong>, which may be employed in addition to, or in place of, the CNN analysis <strong>104</strong>. In this case, it is
assumed that the hand sanitation fluid dispensed at the sanitation station <strong>10</strong> has a distinctive color. Hence, in the color classifier analysis operation <strong>118</strong>, it is determined whether
that color is (sufficiently) present in the image of the wash station zone, and a confidence is computed (or adjusted) on the basis of the color metric output by the color classifier analysis operation
<strong>118</strong>. Other types of visual cues detectable in the acquired video frame may be the basis for the classifier analysis, such as reflectivity, brightness (or, equivalently, dimness), texture metrics,
or so forth.</p>
<p id="p-0076" num="0075">The illustrative embodiments describe detection of a hand sanitation event via analysis of extracted images of the wash station. In the case of a “gown and glove” procedure such as is employed in a C-DIFF protocol,
it is additionally contemplated to employ similar analysis of an image of a gowning zone (e.g., proximate to a closet containing gowns to be put on during the gowning part of the C-DIFF protocol) to detect a gowning
event, so as to monitor compliance with both the gowning and gloving aspects of the C-DIFF or similar gown-and-glove procedure. In this case, the CNN or other ML classifier is trained using training videos of persons
gowning (positive examples) or failing to gown (negative examples) in a manner analogous to the training described for the illustrative CNN <strong>60</strong>. The electronic processor
<strong>24</strong> is used to determine whether a gowning event occurs based on images of a gowning zone extracted from the real time video, and the compliance status (variant of <strong>64</strong> or
<strong>66</strong> of
<figref idref="DRAWINGS">FIG. 2</figref>) is further based on the determination of whether a gowning event occurs. More specifically, a ML classifier is applied to the images of the gowning zone extracted from successive frames of the real
time video to generate per-frame gowning event confidence values, and the determination is made of whether a gowning event occurs based on the per-frame gowning event confidence values for the successive frames.
The ML classifier for generating the per-frame gowning event confidence values is pre-trained using training images of the gowning zone or a standard gowning zone that depict a gowning event and training images
of the gowning zone or the standard gowning zone that do not depict a gowning event.</p>
<p id="p-0077" num="0076">The illustrative examples employ a single video camera <strong>20</strong> as shown in
<figref idref="DRAWINGS">FIG. 1</figref>. However, it is contemplated to employ more than one video camera. For example, one video camera may be provided which is arranged to capture a wide FOV encompassing the area <strong>54</strong> and the sanitation zone; while, a second video camera is arranged with a narrower FOV to capture the smaller wash station zone <strong>40</strong>. This may allow for more accurate determination of whether a hand
sanitation event occurs due to more optimal placement of the second video camera respective to the wash station zone <strong>40</strong>. In another contemplated variant, a single video camera may be positioned
to capture the wash station zone <strong>40</strong>, but the occupancy time interval may be determined by non-camera-based sensor(s) such as motion sensors or depth sensors as previously discussed. In this case,
the occupancy detection instructions <strong>30</strong> are suitably replaced by instructions processing the output of the non-camera-based sensor(s) to determine the occupancy time interval.</p>
<p id="p-0078" num="0077">The preferred embodiments have been illustrated and described. Obviously, modifications and alterations will occur to others upon reading and understanding the preceding detailed description. It is intended that the
invention be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.</p>
<p id="p-0079" num="0078">Aspects of the Present Disclosure</p>
<p id="p-0080" num="0079">The following aspects of the disclosure are exemplary only and not intended to limit the scope of the disclosure.</p>
<p id="p-0081" num="0080">1. A hand hygiene compliance enforcement device comprising: at least one video camera; at least one annunciator; an electronic processor; and a non-transitory storage medium storing instructions that are readable and
executable by the electronic processor to process real time video acquired by the at least one video camera in real time, the instructions including: occupancy detection instructions readable and executable by the
electronic processor to analyze the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real time video is occupied; and sanitation detection instructions readable
and executable by the electronic processor to determine whether a hand sanitation event occurs based on images of a wash station zone extracted from the real time video acquired during the occupancy time period
and to control the at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.</p>
<p id="p-0082" num="0081">2. The hand hygiene compliance enforcement device of aspect 1, wherein the sanitation detection instructions are readable and executable by the electronic processor to determine whether a hand sanitation event occurs
by operations including: for each frame of successive frames of the real time video acquired during the occupancy time period, applying a machine learning (ML) classifier to the image of the wash station zone extracted
from the frame to generate a confidence value indicative of likelihood that the image of the wash station zone depicts a hand sanitation event; and determining whether a hand sanitation event occurs in the wash
station zone based on the confidence values for the successive frames, wherein the ML classifier is pre-trained to generate the confidence value using training images of the wash station zone or a standard wash
station zone that depict a hand sanitation event and training images of the wash station zone or the standard wash station zone that do not depict a hand sanitation event.</p>
<p id="p-0083" num="0082">3. The hand hygiene compliance enforcement device of aspect 2, wherein the applying of the ML classifier includes: converting the image of the wash station zone extracted from the frame to a tensor of channel arrays
wherein each channel array represents a color channel of the image of the wash station zone; and applying the ML classifier to the tensor of channel arrays.</p>
<p id="p-0084" num="0083">4. The hand hygiene compliance enforcement device of any one of aspects 1-3, wherein the images of the wash station zone extracted from the real time video acquired during the occupancy time period have pixel dimensions
L×W where L is 100 pixels or less and W is 100 pixels or less.</p>
<p id="p-0085" num="0084">5. The hand hygiene compliance enforcement device of aspect 4, wherein the extracting of the images of the wash station zone from the real time video acquired during the occupancy time period includes pixel resampling,
the pixel dimensions L×W being after the pixel resampling.</p>
<p id="p-0086" num="0085">6. The hand hygiene compliance enforcement device of any one of aspects 1-5, wherein the sanitation detection instructions are readable and executable by the electronic processor to determine whether a hand sanitation
event occurs in the wash station zone by operations including: detecting a color of a hand sanitation fluid in the images of the wash station zone extracted from the real time video acquired during the occupancy
time period.</p>
<p id="p-0087" num="0086">7. The hand hygiene compliance enforcement device of any one of aspects 1-6, wherein the occupancy detection instructions are readable and executable by the electronic processor to analyze the real time video to determine
the occupancy time interval by operations including at least one of: for successive frames of the real time video, extracting an image of the sanitation zone from the frame and comparing the extracted image of the
sanitation zone with a reference image of the sanitation zone to detect whether the extracted image of the sanitation zone indicates occupation of the sanitation zone; and/or for successive frames of the real time
video, extracting an image of the sanitation zone from the frame and comparing the images of the sanitation zone extracted from the successive frames of the real time video to detect frame-to-frame changes that
indicate occupation of the sanitation zone.</p>
<p id="p-0088" num="0087">8. The hand hygiene compliance enforcement device of any one of aspects 1-7, wherein the at least one annunciator comprises a noncompliance light and the sanitation detection instructions are readable and executable
by the electronic processor to annunciate the hand hygiene compliance status by at least one of: a) controlling the noncompliance light to illuminate starting at the beginning of the occupancy time interval and
ending at the earlier of (i) a stop time respective to the end of the occupancy time interval or (ii) a time when it is determined that a hand sanitation event occurs; b) controlling the noncompliance light to illuminate
starting at the end of the occupancy time interval if it is determined that no hand sanitation event occurred during the occupancy time interval.</p>
<p id="p-0089" num="0088">9. The hand hygiene compliance enforcement device of aspect 8, wherein the at least one annunciator further comprises a compliance light and the sanitation detection instructions are readable and executable by the electronic
processor to further annunciate the hand hygiene compliance status by at least one of: a) controlling the compliance light to illuminate starting at a time in the occupancy time interval when it is determined that
a hand sanitation event occurs; b)</p>
<p id="p-0090" num="0089">controlling the compliance light to illuminate starting at the end of the occupancy time interval if it is determined that a hand sanitation event occurred during the occupancy time interval.</p>
<p id="p-0091" num="0090">10. The hand hygiene compliance enforcement device of any one of aspects 8-9, wherein the noncompliance light is illuminated by a flashing illumination.</p>
<p id="p-0092" num="0091">11. The hand hygiene compliance enforcement device of any one of aspects 1-10, wherein the at least one annunciator includes at least one of: an at least one light and/or an audio speaker and/or a display.</p>
<p id="p-0093" num="0092">12. The hand hygiene compliance enforcement device of any one of aspects 1-11, wherein the instructions stored on the non-transitory storage medium that are readable and executable by the electronic processor to process
the real time video acquired by the at least one video camera do not include instructions readable and executable by the electronic processor to identify a person in the sanitation zone.</p>
<p id="p-0094" num="0093">13. The hand hygiene compliance enforcement device of aspect 12, wherein the instructions further include: reporting instructions readable and executable by the electronic processor to accumulate statistics for a plurality
of successive occupancy time periods on corresponding determinations of whether a hand sanitation event occurs and generate a compliance report on the accumulated statistics.</p>
<p id="p-0095" num="0094">14. The hand hygiene compliance enforcement device of any one of aspects 1-13, wherein the instructions further include: reporting instructions readable and executable by the electronic processor to identify a person
in the sanitation zone during the occupancy time period using a radio frequency identification (RFID) reader that detects RFID badges or facial recognition applied to the real time video; and accumulate statistics
for a plurality of successive occupancy time periods on corresponding determinations of whether a hand sanitation event occurs and generate a compliance report on the accumulated statistics including identification
of persons identified in the sanitation zone using the facial recognition during occupancy time periods for which the corresponding determination of whether a hand sanitation event occurs is that no hand sanitation
event occurred.</p>
<p id="p-0096" num="0095">15. The hand hygiene compliance enforcement device of any one of aspects 1-14, wherein the at least one video camera consists of a single video camera and the wash station zone is contained in the sanitation zone imaged
by the real time video.</p>
<p id="p-0097" num="0096">16. A hand sanitation station including: a hand sanitation station including at least a sanitation fluid dispenser; and a hand hygiene compliance enforcement device as set forth in any one of aspects 1-15, wherein the
at least one video camera is arranged such that at least a portion of the hand sanitation station coincides with the wash station zone in the real time video acquired by the at least one video camera.</p>
<p id="p-0098" num="0097">17. A non-transitory storage medium storing instructions that are readable and executable by an electronic processor operatively connected with at least one video camera and at least one annunciator to process real
time video acquired by the at least one video camera in real time to perform a hand hygiene compliance enforcement method comprising: analyzing the real time video to determine an occupancy time interval over which
a sanitation zone imaged by the real time video is occupied; determining whether a hand sanitation event occurs based on images of a wash station zone extracted from the real time video acquired during the occupancy
time period; and controlling the at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.
</p>
<p id="p-0099" num="0098">18. The non-transitory storage medium of aspect 17, wherein the determining of whether a hand sanitation event occurs comprises: applying a Convolutional Neural Network (CNN) to the images of the wash station zone extracted
from successive frames of the real time video acquired during the occupancy time period to generate per-frame confidence values; and determining whether a hand sanitation event occurs based on the per-frame confidence
values for the successive frames, wherein the CNN is pre-trained to generate confidence values indicating confidence that input images depict a hand sanitation event using training images of the wash station zone
or a standard wash station zone that depict a hand sanitation event and training images of the wash station zone or the standard wash station zone that do not depict a hand sanitation event.
</p>
<p id="p-0100" num="0099">19. The non-transitory storage medium of aspect 18, wherein the applying of the CNN includes: converting the image of the wash station zone extracted from the frame to a tensor of channel arrays wherein each channel
array represents a principal color channel of the image of the wash station zone; and applying the CNN to the tensor of channel arrays.</p>
<p id="p-0101" num="0100">20. The non-transitory storage medium of any one of aspects 17-19, wherein the images of the wash station zone extracted from the real time video acquired during the occupancy time period have pixel dimensions L×W where
L is 100 pixels or less and W is 100 pixels or less.</p>
<p id="p-0102" num="0101">21. The non-transitory storage medium of aspect 20, wherein the extracting of the images of the wash station zone from the real time video acquired during the occupancy time period includes pixel resampling, the pixel
dimensions L×W being after the pixel resampling.</p>
<p id="p-0103" num="0102">22. The non-transitory storage medium of any one of aspects 16-21, wherein the hand hygiene compliance enforcement method further comprises: accumulating statistics for a plurality of successive occupancy time periods
on corresponding determinations of whether a hand sanitation event occurs; and generating a compliance report on the accumulated statistics.</p>
<p id="p-0104" num="0103">23. A hand hygiene compliance enforcement method comprising: using at least one video camera, acquiring real time video of a field of view that includes a sanitation station having a wash station zone; using an electronic
processor, determining whether a hand sanitation event occurs based on images of the wash station zone extracted from the real time video; and using an annunciator, annunciating a compliance status based on the
determination of whether a hand sanitation event occurs.</p>
<p id="p-0105" num="0104">24. The hand hygiene compliance enforcement method of aspect 23, wherein the determining of whether a hand sanitation event occurs comprises: applying a machine learning (ML) classifier to the images of the wash station
zone extracted from successive frames of the real time video to generate per-frame confidence values; and determining whether a hand sanitation event occurs based on the per-frame confidence values for the successive
frames, wherein the ML classifier is pre-trained using training images of the wash station zone or a standard wash station zone that depict a hand sanitation event and training images of the wash station zone or
the standard wash station zone that do not depict a hand sanitation event.</p>
<p id="p-0106" num="0105">25. The hand hygiene compliance enforcement method of aspect 24, wherein the applying of the ML classifier includes: converting the image of the wash station zone extracted from the frame to a tensor of channel arrays
wherein each channel array represents a principal color channel of the image of the wash station zone; and applying the ML classifier to the tensor of channel arrays.</p>
<p id="p-0107" num="0106">26. The hand hygiene compliance enforcement method of any one of aspects 23-25 further comprising: analyzing the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real
time video is occupied; wherein the determining of whether a hand sanitation event occurs is based on images of the wash station zone extracted from only the real time video acquired during the occupancy time interval.</p>
<p id="p-0108" num="0107">27. The hand hygiene compliance enforcement method of any one of aspects 23-26 further comprising: using the electronic processor, determining whether a gowning event occurs based on images of a gowning zone extracted
from the real time video, wherein the compliance status is further based on the determination of whether a gowning event occurs.</p>
<p id="p-0109" num="0108">28. The hand hygiene compliance enforcement method of aspect 27, wherein the determining of whether a gowning event occurs comprises: applying a machine learning (ML) classifier to the images of the gowning zone extracted
from successive frames of the real time video to generate per-frame gowning event confidence values; and determining whether a gowning event occurs based on the per-frame gowning event confidence values for the
successive frames, wherein the ML classifier for generating the per-frame gowning event confidence values is pre-trained using training images of the gowning zone or a standard gowning zone that depict a gowning
event and training images of the gowning zone or the standard gowning zone that do not depict a gowning event.</p>
<p id="p-0110" num="0109">Additional Aspects of the Present Disclosure</p>
<p id="p-0111" num="0110">The following additional aspects of the disclosure are exemplary only and not intended to limit the scope of the disclosure.</p>
<p id="p-0112" num="0111">1. A hand hygiene compliance enforcement device comprising: at least one video camera; at least one annunciator; an electronic processor; and a non-transitory storage medium storing instructions that are readable and
executable by the electronic processor, wherein the instructions, when executed by the electronic processor, cause the electronic processor to: process real time video acquired by the at least one video camera in
real time, analyze, by execution of occupancy detection instructions, the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real time video is occupied, determine,
by execution of sanitation detection instructions, whether a hand sanitation event occurs based on images of a wash station zone as extracted from the real time video acquired during the occupancy time interval,
and control the at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.
</p>
<p id="p-0113" num="0112">2. The hand hygiene compliance enforcement device of aspect 1, wherein the electronic processor is configured to execute the sanitation detection instructions to cause the electronic processor to: apply, for each respective
frame of successive frames of the real time video acquired during the occupancy time interval, a machine learning (ML) classifier to one or more images of the wash station zone as extracted from the respective frame
to generate a confidence value indicative of likelihood that the one or more images of the wash station zone depicts a hand sanitation event, and determine whether a hand sanitation event occurs in the wash station
zone based on the confidence values for the successive frames, wherein the ML classifier is pre-trained to generate the confidence value using (1) training images of the wash station zone or a standard wash station
zone that depicts a hand sanitation event, and (2) training images of the wash station zone or the standard wash station zone that do not depict a hand sanitation event.
</p>
<p id="p-0114" num="0113">3. The hand hygiene compliance enforcement device of aspect 2, wherein application of the ML classifier comprises: converting an image of the wash station zone as extracted from a frame of the successive frames to a
tensor of channel arrays, wherein each channel array represents a color channel of the image of the wash station zone; and applying the ML classifier to the tensor of channel arrays.</p>
<p id="p-0115" num="0114">4. The hand hygiene compliance enforcement device as in any one of aspects 1-3, wherein the images of the wash station zone, as extracted from the real time video acquired during the occupancy time interval, have pixel
dimensions length (L) by width (W) where L is 100 pixels or less and W is 100 pixels or less, wherein extraction of the images of the wash station zone from the real time video acquired during the occupancy time
interval comprises pixel resampling, the pixel dimensions L by W being determined after the pixel resampling.</p>
<p id="p-0116" num="0115">5. The hand hygiene compliance enforcement device as in any one of aspects 1-4, wherein the electronic processor is configured to execute the sanitation detection instructions to cause the electronic processor to: determine
whether a hand sanitation event occurs in the wash station zone by detecting a color of a hand sanitation fluid in the images of the wash station zone as extracted from the real time video acquired during the occupancy
time interval.</p>
<p id="p-0117" num="0116">6. The hand hygiene compliance enforcement device as in any one of aspects 1-5, wherein the electronic processor is configured to execute the occupancy detection instructions to cause the electronic processor to perform
at least one of: a) extracting, for one or more successive frames of the real time video, an extracted image of the sanitation zone from a frame of the one or more successive frames, and comparing the extracted
image of the sanitation zone with a reference image of the sanitation zone to detect whether the extracted image of the sanitation zone indicates occupation of the sanitation zone; or b) extracting, for two or more
successive frames of the real time video, an extracted image of the sanitation zone from a respective frame of the two or more successive frames, and comparing a first image of the sanitation zone as extracted from
a first respective frame of the two or more successive frames with a second image of the sanitation zone as extracted from a second respective frame of the two or more successive frames to detect frame-to-frame
changes that indicate occupation of the sanitation zone.</p>
<p id="p-0118" num="0117">7. The hand hygiene compliance enforcement device as in any one of aspects 1-6, wherein the at least one annunciator comprises a noncompliance light, and wherein the electronic processor is configured to execute the
sanitation detection instructions to cause the electronic processor to perform at least one of: a) controlling the noncompliance light to illuminate starting at a beginning of the occupancy time interval and ending
at the earlier of (i) a stop time respective to an end of the occupancy time interval, or (ii) a time determined when a hand sanitation event occurs; or b) controlling the noncompliance light to illuminate starting
at an end of the occupancy time interval upon a determination that no hand sanitation event occurred during the occupancy time interval.</p>
<p id="p-0119" num="0118">8. The hand hygiene compliance enforcement device of aspect 7, wherein the at least one annunciator further comprises a compliance light, and wherein the electronic processor is configured to execute the sanitation
detection instructions to cause the electronic processor to perform at least one of: a) controlling the compliance light to illuminate starting at a time in the occupancy time interval upon a determination that
a hand sanitation event occurs; b) controlling the compliance light to illuminate starting at an end of the occupancy time interval upon a determination that a hand sanitation event occurred during the occupancy
time interval.</p>
<p id="p-0120" num="0119">9. The hand hygiene compliance enforcement device of aspect 7, wherein the noncompliance light is illuminated by a flashing illumination.</p>
<p id="p-0121" num="0120">10. The hand hygiene compliance enforcement device as in any one of aspects 1-9, wherein the at least one annunciator comprises at least one of: a light; an audio speaker; a display; or, a touchscreen display.</p>
<p id="p-0122" num="0121">11. The hand hygiene compliance enforcement device as in any one of aspects 1-10, wherein the instructions are configured to lack or filter identification of a person in the sanitation zone.</p>
<p id="p-0123" num="0122">12. The hand hygiene compliance enforcement device of aspect 11, wherein the instructions comprise reporting instructions readable and executable by the electronic processor, the reporting instructions that when executed
by the electronic processor, cause the electronic processor to: accumulate data statistics for a plurality of successive occupancy time intervals on corresponding determinations of whether a hand sanitation event
occurs and generate a compliance report on the data statistics.</p>
<p id="p-0124" num="0123">13. The hand hygiene compliance enforcement device as in any one of aspects 1-12, wherein the instructions comprise reporting instructions readable and executable by the electronic processor, the reporting instructions
that when executed by the electronic processor, cause the electronic processor to: identify a person in the sanitation zone during the occupancy time interval using (i) a radio frequency identification (RFID) reader
that detects RFID badges, or (ii) facial recognition applied to the real time video; accumulate data statistics for a plurality of successive occupancy time intervals on corresponding determinations of whether a
hand sanitation event occurs; and generate a compliance report based on the data statistics including identification of one or more persons identified in the sanitation zone during occupancy time intervals for which
the corresponding determination of whether a hand sanitation event occurs is that no hand sanitation event occurred.</p>
<p id="p-0125" num="0124">14. The hand hygiene compliance enforcement device as in any one of aspects 1-13, wherein the at least one video camera consists of a single video camera and the wash station zone is contained in the sanitation zone
imaged by the real time video.</p>
<p id="p-0126" num="0125">15. The hand hygiene compliance enforcement device as in any one of aspects 1-14 further comprising: a hand sanitation station including at least a sanitation fluid dispenser, wherein the at least one video camera is
positioned such that at least a portion of the hand sanitation station coincides with the wash station zone in the real time video acquired by the at least one video camera.</p>
<p id="p-0127" num="0126">16. A non-transitory storage medium storing instructions for performing a hand hygiene compliance enforcement, instructions being readable and executable by an electronic processor, wherein the instructions, when executed
by the electronic processor, cause the electronic processor to: receive real time video acquired by at least one video camera in real time; analyze the real time video to determine an occupancy time interval over
which a sanitation zone imaged by the real time video is occupied; determine whether a hand sanitation event occurs based on images of a wash station zone as extracted from the real time video acquired during the
occupancy time interval; and control at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.
</p>
<p id="p-0128" num="0127">17. The non-transitory storage medium of aspect 16, wherein determination of whether the hand sanitation event occurs comprises: applying a convolutional neural network (CNN) to the images of the wash station zone as
extracted from successive frames of the real time video acquired during the occupancy time interval to generate per-frame confidence values; and determining whether the hand sanitation event occurs based on the
per-frame confidence values, wherein the CNN is pre-trained to generate confidence values indicating confidence that input images depict a hand sanitation event using training images of the wash station zone or
a standard wash station zone that depict a hand sanitation event and training images of the wash station zone or the standard wash station zone that do not depict a hand sanitation event.
</p>
<p id="p-0129" num="0128">18. The non-transitory storage medium of aspect 17 wherein application of the CNN comprises: converting the image of the wash station zone as extracted from a frame of the successive frames to a tensor of channel arrays
wherein each channel array represents a color channel of the image of the wash station zone; and applying the CNN to the tensor of channel arrays.</p>
<p id="p-0130" num="0129">19. The non-transitory storage medium as in any one of aspects 1-17, wherein the instructions, when executed by the electronic processor, further cause the electronic processor to: accumulate data statistics for a plurality
of successive occupancy time intervals on corresponding determinations of whether a hand sanitation event occurs; and generate a compliance report based on the data statistics.</p>
<p id="p-0131" num="0130">20. A hand hygiene compliance enforcement method comprising: acquiring, via a at least one video camera, real time video of a field of view that includes a sanitation station having a wash station zone; determining,
via an electronic processor, whether a hand sanitation event occurs based on images of the wash station zone as extracted from the real time video; and annunciating, via an annunciator, a compliance status based
on the determination of whether a hand sanitation event occurs.</p>
<p id="p-0132" num="0131">21. The hand hygiene compliance enforcement method of aspect 20, wherein determination of whether a hand sanitation event occurs comprises: applying a machine learning (ML) classifier to the images of the wash station
zone as extracted from successive frames of the real time video to generate per-frame confidence values; and determining whether a hand sanitation event occurs based on the per-frame confidence values for the successive
frames, wherein the ML classifier is pre-trained using training images of the wash station zone or a standard wash station zone that depict a hand sanitation event and training images of the wash station zone or
the standard wash station zone that do not depict a hand sanitation event.</p>
<p id="p-0133" num="0132">22. The hand hygiene compliance enforcement method of aspect 21, wherein application of the ML classifier comprises: converting an image of the wash station zone as extracted from the frame of the successive frames
to a tensor of channel arrays, wherein each channel array represents a color channel of the image of the wash station zone; and applying the ML classifier to the tensor of channel arrays.</p>
<p id="p-0134" num="0133">23. The hand hygiene compliance enforcement method as in any one of aspects 1-22 further comprising analyzing the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real
time video is occupied, wherein the determination of whether a hand sanitation event occurs is based on images of the wash station zone as extracted from only the real time video acquired during the occupancy time
interval.
</p>
<p id="p-0135" num="0134">24. The hand hygiene compliance enforcement method as in any one of aspects 1-23 further comprising determining, with the electronic processor, whether a gowning event occurs based on images of a gowning zone as extracted
from the real time video, wherein the compliance status is further based on the determination of whether a gowning event occurs.</p>
<p id="p-0136" num="0135">25. The hand hygiene compliance enforcement method of aspect 24, wherein determination of whether the gowning event occurs comprises: applying a machine learning (ML) classifier to the images of the gowning zone as
extracted from successive frames of the real time video to generate per-frame gowning event confidence values; and determining whether the gowning event occurs based on the per-frame gowning event confidence values
for the successive frames, wherein the ML classifier for generating the per-frame gowning event confidence values is pre-trained using training images of the gowning zone or a standard gowning zone that depict a
gowning event and training images of the gowning zone or the standard gowning zone that do not depict a gowning event.</p>
<p id="p-0137" num="0136">26. A hand hygiene compliance enforcement device comprising: at least one non-camera-based sensor; at least one annunciator; an electronic processor; and a non-transitory storage medium storing instructions that are
readable and executable by the electronic processor, wherein the instructions, when executed by the electronic processor, cause the electronic processor to: process real time data output of the non-camera-based
sensor in real time, analyze, by execution of occupancy detection instructions, the data output to determine an occupancy time interval over which a sanitation zone determined by the data output is occupied, determine,
by execution of sanitation detection instructions, whether a hand sanitation event occurs based on the data output corresponding to a wash station zone, the data output acquired during the occupancy time interval,
and control the at least one annunciator to annunciate a hand hygiene compliance status based on the hand sanitation event.</p>
<p id="p-0138" num="0137">27. The hand hygiene compliance enforcement device of aspect 26, wherein the electronic processor is configured to execute the sanitation detection instructions to cause the electronic processor to: apply, for each
series of data output of the non-camera-based sensor as acquired during the occupancy time interval, a machine learning (ML) classifier to sensor data corresponding to the wash station zone, to generate a confidence
value indicative of likelihood that the sensor data of the wash station zone corresponds a hand sanitation event, and determine whether a hand sanitation event occurs in the wash station zone based on the confidence
values for the successive frames, wherein the ML classifier is pre-trained to generate the confidence value using training sensor data of the wash station zone or a standard wash station zone that defines a hand
sanitation event and sensor data of the wash station zone or the standard wash station zone that do not correspond to a hand sanitation event.
</p>
<p id="p-0139" num="0138">28. The hand hygiene compliance enforcement device of aspect 27, wherein application of the ML classifier comprises: converting sensor data of the wash station zone to a tensor of channel arrays, wherein each channel
array emulates a color channel of an image of the wash station zone; and applying the ML classifier to the tensor of channel arrays.</p>
<p id="p-0140" num="0139">29. The hand hygiene compliance enforcement device of aspect 1 further comprising an information sharing application comprising an information sharing interface, the information sharing interface configured to display
sanitation or hygiene compliance data or information.</p>
<p id="p-0141" num="0140">30. The hand hygiene compliance enforcement device of aspect 1 further comprising a hand hygiene marketing platform comprising a marketing interface, the marketing interface configured to display advertisements to users
in the sanitation zone or a marketing zone.</p>
<p id="p-0142" num="0141">Additional Considerations</p>
<p id="p-0143" num="0142">Although the disclosure herein sets forth a detailed description of numerous different embodiments, it should be understood that the legal scope of the description is defined by the words of the claims set forth at
the end of this patent and equivalents. The detailed description is to be construed as exemplary only and does not describe every possible embodiment since describing every possible embodiment would be impractical.
Numerous alternative embodiments may be implemented, using either current technology or technology developed after the filing date of this patent, which would still fall within the scope of the claims.</p>
<p id="p-0144" num="0143">The following additional considerations apply to the foregoing discussion. Throughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although
individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations
be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality
presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.</p>
<p id="p-0145" num="0144">Additionally, certain embodiments are described herein as including logic or a number of routines, subroutines, applications, or instructions. These may constitute either software (e.g., code embodied on a machine-readable
medium or in a transmission signal) or hardware. In hardware, the routines, etc., are tangible units capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments,
one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software
(e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.</p>
<p id="p-0146" num="0145">In various embodiments, a hardware module may be implemented mechanically or electronically. For example, a hardware module may comprise dedicated circuitry or logic that is permanently configured (e.g., as a special-purpose
processor, such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) to perform certain operations. A hardware module may also comprise programmable logic or circuitry
(e.g., as encompassed within a general-purpose processor or other programmable processor) that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement
a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.</p>
<p id="p-0147" num="0146">Accordingly, the term “hardware module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g.,
programmed) to operate in a certain manner or to perform certain operations described herein. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware
modules need not be configured or instantiated at any one instance in time. For example, where the hardware modules comprise a general-purpose processor configured using software, the general-purpose processor may
be configured as respective different hardware modules at different times. Software may accordingly configure a processor, for example, to constitute a particular hardware module at one instance of time and to constitute
a different hardware module at a different instance of time.</p>
<p id="p-0148" num="0147">Hardware modules may provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple of such
hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the hardware modules. In embodiments in which multiple
hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures
to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further
hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and may operate on a
resource (e.g., a collection of information).</p>
<p id="p-0149" num="0148">The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the
relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred
to herein may, in some example embodiments, comprise processor-implemented modules.</p>
<p id="p-0150" num="0149">Similarly, the methods or routines described herein may be at least partially processor-implemented. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented
hardware modules. The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example
embodiments, the processor or processors may be located in a single location, while in other embodiments the processors may be distributed across a number of locations.</p>
<p id="p-0151" num="0150">The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the
one or more processors or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other embodiments, the one or
more processors or processor-implemented modules may be distributed across a number of geographic locations.</p>
<p id="p-0152" num="0151">This detailed description is to be construed as exemplary only and does not describe every possible embodiment, as describing every possible embodiment would be impractical, if not impossible. A person of ordinary skill
in the art may implement numerous alternate embodiments, using either current technology or technology developed after the filing date of this application.</p>
<p id="p-0153" num="0152">Those of ordinary skill in the art will recognize that a wide variety of modifications, alterations, and combinations can be made with respect to the above described embodiments without departing from the scope of the
invention, and that such modifications, alterations, and combinations are to be viewed as being within the ambit of the inventive concept.</p>
<p id="p-0154" num="0153">The patent claims at the end of this patent application are not intended to be construed under 35 U.S.C. § 112(f) unless traditional means-plus-function language is expressly recited, such as “means for” or “step for”
language being explicitly recited in the claim(s). The systems and methods described herein are directed to an improvement to computer functionality, and improve the functioning of conventional computers.
</p>
</div>
</div>
<div id="claims" class="wrapper">
<div class="title-wrapper">
<h2 class="heading-2">Claims</h2>
</div>
<div class="wrap">
<div class="claim">
<p>1. A hand hygiene compliance enforcement device comprising:</p>
<dl>
<dd>at least one video camera;</dd>
</dl>
<dl>
<dd>at least one annunciator;</dd>
</dl>
<dl>
<dd>an electronic processor; and</dd>
</dl>
<dl>
<dd>a non-transitory storage medium storing instructions that are readable and executable by the electronic processor, wherein the instructions, when executed by the electronic processor, cause the electronic processor
to:
</dd>
</dl>
<dl>
<dd>process real time video acquired by the at least one video camera in real time,</dd>
</dl>
<dl>
<dd>analyze, by execution of occupancy detection instructions, the real time video to determine an occupancy time interval over which a sanitation zone imaged by the real time video is occupied,</dd>
</dl>
<dl>
<dd>determine, by execution of sanitation detection instructions, whether a hand sanitation event occurs based on images of a wash station zone as extracted from the real time video acquired during the occupancy
time interval,</dd>
</dl>
<dl>
<dd>convert an image of the wash station zone as extracted from a frame of one or more frames to a tensor of channel arrays wherein each channel array represents a color channel of the image of the wash station
zone,
</dd>
</dl>
<dl>
<dd>apply a convolutional neural network (CNN) to the tensor of channel arrays, and</dd>
</dl>
<dl>
<dd>control the at least one annunciator to annunciate a hand hygiene compliance status based on the determination of whether a hand sanitation event occurs.</dd>
</dl>
</div>
<div class="claim">
<p>2. The hand hygiene compliance enforcement device of claim 1, wherein the electronic processor is configured to execute the sanitation detection instructions to cause the electronic processor to:</p>
<dl>
<dd>apply, for each respective frame of successive frames of the real time video acquired during the occupancy time interval, a machine learning (ML) classifier to one or more images of the wash station zone as
extracted from the respective frame to generate a confidence value indicative of likelihood that the one or more images of the wash station zone depicts a hand sanitation event, and</dd>
</dl>
<dl>
<dd>determine whether a hand sanitation event occurs in the wash station zone based on the confidence values for the successive frames,</dd>
</dl>
<dl>
<dd>wherein the ML classifier is pre-trained to generate the confidence value using (1) training images of the wash station zone or a standard wash station zone that depicts a hand sanitation event, and (2) training
images of the wash station zone or the standard wash station zone that do not depict a hand sanitation event.</dd>
</dl>
</div>
<div class="claim">
<p>3. The hand hygiene compliance enforcement device of claim 2, wherein application of the ML classifier comprises:</p>
<dl>
<dd>converting an image of the wash station zone as extracted from a frame of the successive frames to a tensor of channel arrays, wherein each channel array represents a color channel of the image of the wash station
zone; and</dd>
</dl>
<dl>
<dd>applying the ML classifier to the tensor of channel arrays.</dd>
</dl>
</div>
<div class="claim">
<p>4. The hand hygiene compliance enforcement device of claim 1,</p>
<dl>
<dd>wherein the images of the wash station zone, as extracted from the real time video acquired during the occupancy time interval, have pixel dimensions length (L) by width (W) where L is 100 pixels or less and
W is 100 pixels or less, and</dd>
</dl>
<dl>
<dd>wherein extraction of the images of the wash station zone from the real time video acquired during the occupancy time interval comprises pixel resampling, the pixel dimensions L by W being determined after the
pixel resampling.</dd>
</dl>
</div>
<div class="claim">
<p>5. The hand hygiene compliance enforcement device of claim 1, wherein the electronic processor is configured to execute the sanitation detection instructions to cause the electronic processor to:</p>
<dl>
<dd>determine whether a hand sanitation event occurs in the wash station zone by detecting a color of a hand sanitation fluid in the images of the wash station zone as extracted from the real time video acquired
during the occupancy time interval.</dd>
</dl>
</div>
<div class="claim">
<p>6. The hand hygiene compliance enforcement device of claim 1, wherein the electronic processor is configured to execute the occupancy detection instructions to cause the electronic processor to perform at least
one of:</p>
<dl>
<dd>a) extracting, for one or more successive frames of the real time video, an extracted image of the sanitation zone from a frame of the one or more successive frames, and comparing the extracted image of the
sanitation zone with a reference image of the sanitation zone to detect whether the extracted image of the sanitation zone indicates occupation of the sanitation zone; or</dd>
</dl>
<dl>
<dd>b) extracting, for two or more successive frames of the real time video, an extracted image of the sanitation zone from a respective frame of the two or more successive frames, and comparing a first image of
the sanitation zone as extracted from a first respective frame of the two or more successive frames with a second image of the sanitation zone as extracted from a second respective frame of the two or more
successive frames to detect frame-to-frame changes that indicate occupation of the sanitation zone.</dd>
</dl>
</div>
<div class="claim">
<p>7. The hand hygiene compliance enforcement device of claim 1,</p>
<dl>
<dd>wherein the at least one annunciator comprises a noncompliance light, and</dd>
</dl>
<dl>
<dd>wherein the electronic processor is configured to execute the sanitation detection instructions to cause the electronic processor to perform at least one of:</dd>
</dl>
<dl>
<dd>a) controlling the noncompliance light to illuminate starting at a beginning of the occupancy time interval and ending at the earlier of (i) a stop time respective to an end of the occupancy time interval, or
(ii) a time determined when a hand sanitation event occurs; or</dd>
</dl>
<dl>
<dd>b) controlling the noncompliance light to illuminate starting at an end of the occupancy time interval upon a determination that no hand sanitation event occurred during the occupancy time interval.</dd>
</dl>
</div>
<div class="claim">
<p>8. The hand hygiene compliance enforcement device of claim 7,</p>
<dl>
<dd>wherein the at least one annunciator further comprises a compliance light, and</dd>
</dl>
<dl>
<dd>wherein the electronic processor is configured to execute the sanitation detection instructions to cause the electronic processor to perform at least one of:</dd>
</dl>
<dl>
<dd>a) controlling the compliance light to illuminate starting at a time in the occupancy time interval upon a determination that a hand sanitation event occurs;</dd>
</dl>
<dl>
<dd>b) controlling the compliance light to illuminate starting at an end of the occupancy time interval upon a determination that a hand sanitation event occurred during the occupancy time interval.</dd>
</dl>
</div>
<div class="claim">
<p>9. The hand hygiene compliance enforcement device of claim 7, wherein the noncompliance light is illuminated by a flashing illumination.</p>
</div>
<div class="claim">
<p>10. The hand hygiene compliance enforcement device of claim 1, wherein the at least one annunciator comprises at least one of:</p>
<dl>
<dd>a light;</dd>
</dl>
<dl>
<dd>an audio speaker;</dd>
</dl>
<dl>
<dd>a display; or</dd>
</dl>
<dl>
<dd>a touchscreen display.</dd>
</dl>
</div>
<div class="claim">
<p>11. The hand hygiene compliance enforcement device of claim 1, wherein the instructions are configured to lack or filter identification of a person in the sanitation zone.</p>
</div>
<div class="claim">