A temporal computer-vision study that turns two-hand landmark sequences into six gesture labels. Its evidence is numerical rather than photographic: 180 sequences, a 30×126 frame representation and a three-layer LSTM.
MediaPipe
LSTM
Role
Academic Software Developer & AI Researcher
Context
Üsküdar University · Computer-vision research
Platform
Temporal computer-vision study
Stack
MediaPipe · LSTM
What it is
Hand Gesture Recognition classifies movement across time rather than treating each camera frame as an independent image. MediaPipe extracts numerical hand landmarks, and the model reads a 30-frame window before producing one of six labels: one, two, three, four, five or ok.
The retained dataset contains landmark arrays only, not raw training images or video.
How it works
Each detected hand contributes 21 x/y/z landmarks. Two fixed 63-value hand slots create 126 features per frame, and 30 frames form one sequence. The dataset contains 30 sequences for each of six classes: 180 sequences in total.
The model shape is (30,126) → LSTM 64 → LSTM 128 → LSTM 64 → Dense 64 → Dense 32 → Softmax 6, with 203,558 trainable parameters. Training used Adam and categorical cross-entropy on an unseeded, unstratified 171/9 split.
The final recorded nine-sequence holdout classified 9/9 sequences correctly. With nine held-out sequences and an unstratified split, that result does not establish general accuracy.
From landmarks to a temporal label
Six diagrams trace the pipeline, landmark skeleton, temporal sequence, balanced dataset, model architecture and recorded training history.
Landmark-sequence dataset by gesture class
Class
Sequences
Frames / sequence
Values / frame
Frame arrays
one
30
30
126
900
two
30
30
126
900
three
30
30
126
900
four
30
30
126
900
five
30
30
126
900
ok
30
30
126
900
Total
180
126
5400
View all 543 recorded training epochs
Recorded training loss and categorical accuracy; no validation series exists
Epoch
Training loss
Training categorical accuracy
1
1.1578009129
0.5438596606
2
1.0313410759
0.6081871390
3
0.8568369746
0.7426900864
4
0.6842311621
0.7836257219
5
0.6161634326
0.7836257219
6
0.6161122322
0.7485380173
7
0.5541213751
0.7836257219
8
0.4896924198
0.8713450432
9
0.3764736354
0.8654970527
10
0.3231026232
0.8596491218
11
0.2707445920
0.8888888955
12
0.2810843289
0.9064327478
13
0.2748120725
0.8830409646
14
0.2583157718
0.9122806787
15
0.2185157388
0.9122806787
16
0.2147673070
0.9122806787
17
0.3400277793
0.8596491218
18
0.2102598846
0.9298245907
19
0.1820447892
0.9239766002
20
0.2035875767
0.9298245907
21
0.2060296386
0.9181286693
22
0.2191816121
0.9122806787
23
0.3154517412
0.9064327478
24
0.2856536806
0.9122806787
25
0.1995094717
0.9473684430
26
0.1923433989
0.9473684430
27
0.1630701572
0.9532163739
28
0.1524677873
0.9356725216
29
0.1074356735
0.9766082168
30
0.0846264958
0.9707602262
31
0.0845843330
0.9707602262
32
0.0679690987
0.9766082168
33
0.0678139552
0.9824561477
34
0.0765199140
0.9707602262
35
0.0583755150
0.9824561477
36
0.0661458969
0.9707602262
37
0.0826285705
0.9649122953
38
0.0765512586
0.9707602262
39
0.1010208204
0.9532163739
40
0.2512041032
0.9239766002
41
0.9410870671
0.7309941649
42
0.5852725506
0.7836257219
43
0.4715543687
0.9064327478
44
0.4037683606
0.9064327478
45
0.3163694739
0.9532163739
46
0.2838202119
0.9239766002
47
0.2278554887
0.9590643048
48
0.2011085004
0.9649122953
49
0.1725596935
0.9590643048
50
0.1417733431
0.9649122953
51
0.1385067403
0.9590643048
52
0.1137113422
0.9707602262
53
0.0969036743
0.9766082168
54
0.0953205749
0.9707602262
55
0.0855420083
0.9766082168
56
0.0789767727
0.9590643048
57
0.0621409677
0.9824561477
58
0.0627583489
0.9824561477
59
0.0552054010
0.9824561477
60
0.0514369085
0.9824561477
61
0.0485312194
0.9824561477
62
0.0468996689
0.9824561477
63
0.0472824760
0.9824561477
64
0.0442447215
0.9824561477
65
0.0518371128
0.9766082168
66
0.1682147533
0.9649122953
67
0.0777952373
0.9766082168
68
0.0644988641
0.9824561477
69
0.0597442202
0.9824561477
70
0.0567674041
0.9824561477
71
0.0522771738
0.9824561477
72
0.0505024604
0.9824561477
73
0.0481830835
0.9824561477
74
0.0479780957
0.9824561477
75
0.0437816195
0.9824561477
76
0.0428902842
0.9824561477
77
0.0424159802
0.9824561477
78
0.0485070981
0.9707602262
79
0.0468586497
0.9824561477
80
0.0414346270
0.9824561477
81
0.0435113683
0.9707602262
82
0.0620787442
0.9649122953
83
0.0396533348
0.9824561477
84
0.6332854629
0.8713450432
85
0.6613140106
0.7894737124
86
0.4670028985
0.8304093480
87
0.2939976156
0.9005848169
88
0.2375048399
0.9239766002
89
0.1905562431
0.9239766002
90
0.2108155638
0.9181286693
91
0.1662392467
0.9298245907
92
0.1563734710
0.9590643048
93
0.1585954130
0.9356725216
94
0.1113915667
0.9707602262
95
0.1041415036
0.9707602262
96
0.0854264945
0.9766082168
97
0.0783174336
0.9707602262
98
0.0695845485
0.9766082168
99
0.0623288974
0.9824561477
100
0.0594459437
0.9766082168
101
0.0557903275
0.9824561477
102
0.0544462353
0.9824561477
103
0.0670498013
0.9766082168
104
0.0762896091
0.9649122953
105
0.0572458506
0.9824561477
106
0.0800400078
0.9649122953
107
0.0673298836
0.9707602262
108
0.0579447597
0.9824561477
109
0.0545185953
0.9766082168
110
0.0499280728
0.9824561477
111
0.0447078459
0.9824561477
112
0.0412017740
0.9824561477
113
0.0413728803
0.9824561477
114
0.0406385548
0.9824561477
115
0.0407022797
0.9824561477
116
0.0393029526
0.9824561477
117
0.0392706990
0.9824561477
118
0.0397994816
0.9766082168
119
0.0400946513
0.9824561477
120
0.0425490886
0.9824561477
121
0.0404326245
0.9824561477
122
0.0388051867
0.9824561477
123
0.0409050286
0.9649122953
124
0.0470327847
0.9707602262
125
0.0384638906
0.9824561477
126
0.0402793437
0.9824561477
127
0.0391100682
0.9824561477
128
0.0383144580
0.9824561477
129
0.0378894433
0.9824561477
130
0.0384798534
0.9824561477
131
0.0379611440
0.9824561477
132
0.0378930680
0.9824561477
133
0.0374797061
0.9824561477
134
0.0381480753
0.9824561477
135
0.0370692164
0.9824561477
136
0.0385469906
0.9824561477
137
0.0381922275
0.9824561477
138
0.0383257940
0.9824561477
139
0.0374774188
0.9824561477
140
0.0367885344
0.9824561477
141
0.0371616520
0.9824561477
142
0.0368369408
0.9824561477
143
0.0371929742
0.9824561477
144
0.0382863842
0.9824561477
145
0.0381635353
0.9824561477
146
0.0362503640
0.9824561477
147
0.0371155106
0.9824561477
148
0.0399011746
0.9824561477
149
0.0374046341
0.9824561477
150
0.0365602076
0.9824561477
151
0.0371937156
0.9824561477
152
0.0369740017
0.9824561477
153
0.0369820893
0.9824561477
154
0.0371140465
0.9824561477
155
0.0370325260
0.9824561477
156
0.0368992575
0.9824561477
157
0.0370526053
0.9824561477
158
0.0369984806
0.9824561477
159
0.0385989882
0.9824561477
160
0.0377466641
0.9824561477
161
0.0364422947
0.9824561477
162
0.0369879752
0.9824561477
163
0.0366889983
0.9824561477
164
0.0371001400
0.9824561477
165
0.0372471921
0.9824561477
166
0.0363066159
0.9824561477
167
0.0370234177
0.9824561477
168
0.0391105860
0.9824561477
169
0.0383314304
0.9824561477
170
0.0364814550
0.9824561477
171
0.0366182216
0.9824561477
172
0.0363418907
0.9824561477
173
0.0365712233
0.9824561477
174
0.0367952250
0.9824561477
175
0.0368481837
0.9824561477
176
0.0374038219
0.9824561477
177
0.0368510187
0.9824561477
178
0.0367996693
0.9824561477
179
0.0359689407
0.9824561477
180
0.0364526995
0.9824561477
181
0.0371203572
0.9824561477
182
0.0373599194
0.9824561477
183
0.0362827480
0.9824561477
184
0.0373911485
0.9824561477
185
0.0367436409
0.9824561477
186
0.0364847034
0.9824561477
187
0.0362130366
0.9824561477
188
0.0365806036
0.9824561477
189
0.0374372676
0.9824561477
190
0.0361092985
0.9824561477
191
0.0383385867
0.9824561477
192
0.0371739157
0.9824561477
193
0.0362518542
0.9824561477
194
0.0362024829
0.9824561477
195
0.0364537500
0.9824561477
196
0.0364911221
0.9824561477
197
0.0362763852
0.9824561477
198
0.0366517007
0.9824561477
199
0.0360454917
0.9824561477
200
0.0378368050
0.9766082168
201
0.0359797962
0.9824561477
202
0.0367180631
0.9824561477
203
0.0382663608
0.9824561477
204
0.0362911783
0.9824561477
205
0.0427905619
0.9707602262
206
0.0358047001
0.9824561477
207
0.0402464531
0.9649122953
208
0.0377740487
0.9824561477
209
0.0362097397
0.9824561477
210
0.0377335399
0.9766082168
211
0.0368281566
0.9766082168
212
0.0365250446
0.9824561477
213
0.0363042913
0.9824561477
214
0.0361936502
0.9824561477
215
0.0363266692
0.9824561477
216
0.0363913812
0.9824561477
217
0.0362923592
0.9824561477
218
0.0367265679
0.9824561477
219
0.0367503799
0.9824561477
220
0.0369520038
0.9824561477
221
0.0365664661
0.9824561477
222
0.0361721031
0.9824561477
223
0.0359965935
0.9824561477
224
0.0380469263
0.9824561477
225
0.0361770540
0.9824561477
226
0.0377342440
0.9824561477
227
0.0359824635
0.9824561477
228
0.0370657332
0.9824561477
229
0.0376189165
0.9824561477
230
0.0374432877
0.9824561477
231
0.0365192778
0.9824561477
232
0.0366483182
0.9824561477
233
0.0367321819
0.9824561477
234
0.0375153422
0.9824561477
235
0.0363449678
0.9824561477
236
0.0363556892
0.9824561477
237
0.0362888426
0.9824561477
238
0.0361300446
0.9824561477
239
0.0362388194
0.9824561477
240
0.0362070389
0.9824561477
241
0.0362585001
0.9824561477
242
0.0363137200
0.9824561477
243
0.0358173214
0.9824561477
244
0.0435241237
0.9649122953
245
0.0447322614
0.9824561477
246
0.0358291231
0.9824561477
247
0.0370618626
0.9766082168
248
0.0374054387
0.9766082168
249
0.0370923579
0.9766082168
250
0.0374854133
0.9824561477
251
0.0364145190
0.9824561477
252
0.0373244807
0.9824561477
253
0.0371678099
0.9824561477
254
0.0368589386
0.9824561477
255
0.0371101461
0.9824561477
256
0.0360004418
0.9824561477
257
0.0359600075
0.9824561477
258
0.0359774381
0.9824561477
259
0.0359989591
0.9824561477
260
0.0363394395
0.9824561477
261
0.0361437276
0.9824561477
262
0.0367171355
0.9824561477
263
0.0364045613
0.9824561477
264
0.0364263617
0.9824561477
265
0.0364555977
0.9824561477
266
0.0362888016
0.9824561477
267
0.0361939408
0.9824561477
268
0.0362255797
0.9824561477
269
0.0359721109
0.9824561477
270
0.0364179872
0.9824561477
271
0.0361903384
0.9824561477
272
0.0361168496
0.9824561477
273
0.0359805264
0.9824561477
274
0.0361222513
0.9824561477
275
0.0360535346
0.9824561477
276
0.0361658074
0.9824561477
277
0.0359649435
0.9824561477
278
0.0360480323
0.9824561477
279
0.0363731235
0.9824561477
280
0.0360605568
0.9824561477
281
0.0363988653
0.9824561477
282
0.0359231867
0.9824561477
283
0.0361375958
0.9824561477
284
0.0363416784
0.9824561477
285
0.0359754451
0.9824561477
286
0.0359524414
0.9824561477
287
0.0360958725
0.9824561477
288
0.0364686362
0.9824561477
289
0.0358196795
0.9824561477
290
0.0360262953
0.9824561477
291
0.0361660868
0.9824561477
292
0.0360067002
0.9824561477
293
0.0364272073
0.9824561477
294
0.0360623673
0.9824561477
295
0.0362018570
0.9824561477
296
0.0362351909
0.9824561477
297
0.0359334014
0.9824561477
298
0.0359414183
0.9824561477
299
0.0358596146
0.9824561477
300
0.0359899625
0.9824561477
301
0.0362326466
0.9824561477
302
0.0372249223
0.9824561477
303
0.0361805968
0.9824561477
304
0.0364938341
0.9824561477
305
0.0362034738
0.9824561477
306
0.0361369550
0.9824561477
307
0.0360628255
0.9824561477
308
0.0361192115
0.9824561477
309
0.0359122567
0.9824561477
310
0.0360486694
0.9824561477
311
0.0359763876
0.9824561477
312
0.0358466767
0.9824561477
313
0.0358377658
0.9824561477
314
0.0358671956
0.9824561477
315
0.0357321836
0.9824561477
316
0.0359350219
0.9824561477
317
0.0360730812
0.9824561477
318
0.0361330248
0.9824561477
319
0.0358980820
0.9824561477
320
0.0362412632
0.9824561477
321
0.0360143557
0.9824561477
322
0.0361372791
0.9824561477
323
0.0356284752
0.9824561477
324
0.0363727286
0.9824561477
325
0.0363800488
0.9824561477
326
0.0363331810
0.9824561477
327
0.0362125710
0.9824561477
328
0.0359371491
0.9824561477
329
0.0362351649
0.9824561477
330
0.0358572230
0.9824561477
331
0.0361029878
0.9824561477
332
0.0359888449
0.9824561477
333
0.0358666852
0.9824561477
334
0.0365638807
0.9824561477
335
0.0363855585
0.9824561477
336
0.0360660739
0.9824561477
337
0.0361332260
0.9824561477
338
0.0363593884
0.9824561477
339
0.0363889448
0.9824561477
340
0.0359831341
0.9824561477
341
0.0358334519
0.9824561477
342
0.0358616635
0.9824561477
343
0.0361575335
0.9824561477
344
0.0362299383
0.9824561477
345
0.0365812182
0.9824561477
346
0.0363328680
0.9824561477
347
0.0361300595
0.9824561477
348
0.0359482989
0.9824561477
349
0.0360421166
0.9824561477
350
0.0361504070
0.9824561477
351
0.0359258428
0.9824561477
352
0.0361432470
0.9824561477
353
0.0358467810
0.9824561477
354
0.0363101922
0.9824561477
355
0.0359484069
0.9824561477
356
0.0366898812
0.9824561477
357
0.0360810757
0.9824561477
358
0.0365141779
0.9824561477
359
0.0358092226
0.9824561477
360
0.0365246534
0.9824561477
361
0.0364644639
0.9824561477
362
0.0370634869
0.9824561477
363
0.0365214832
0.9766082168
364
0.0366283059
0.9766082168
365
0.0364349447
0.9824561477
366
0.0364565253
0.9824561477
367
0.0362715237
0.9824561477
368
0.0359365158
0.9824561477
369
0.0361673757
0.9824561477
370
0.0358693823
0.9824561477
371
0.0358925164
0.9824561477
372
0.0364202745
0.9824561477
373
0.0365264975
0.9824561477
374
2.8760762215
0.8245614171
375
8.9292459488
0.3157894611
376
2.2707521915
0.2397660762
377
1.9005860090
0.1754385978
378
1.7572470903
0.2280701697
379
1.7046931982
0.2807017565
380
1.6344677210
0.3450292349
381
1.6387418509
0.3274853826
382
1.6178569794
0.3216374218
383
1.5759234428
0.3859649003
384
1.5198593140
0.4327485263
385
1.4279619455
0.3859649003
386
1.3400405645
0.4853801131
387
1.3643544912
0.4035087824
388
1.2689682245
0.5087719560
389
1.3814005852
0.4853801131
390
1.2116527557
0.4619883001
391
1.2484536171
0.5146198869
392
1.1076824665
0.5263158083
393
1.0389728546
0.5380116701
394
1.0570843220
0.5497075915
395
1.0434678793
0.5672514439
396
0.9035646319
0.5906432867
397
0.8552541733
0.6023392081
398
0.8659574986
0.5964912176
399
0.7990543246
0.6140350699
400
0.7304780483
0.6666666865
401
0.6644337177
0.7076023221
402
0.6198091507
0.6842105389
403
0.5962339044
0.7426900864
404
0.5420210958
0.7602339387
405
0.5206702352
0.7777777910
406
1.3507410288
0.5087719560
407
1.6422460079
0.2865497172
408
1.3241603374
0.4502924085
409
1.1083149910
0.6432748437
410
0.9992468953
0.6257309914
411
0.8836395741
0.6198830605
412
0.7818229795
0.6491228342
413
0.7257116437
0.7017543912
414
0.6375381947
0.7485380173
415
0.5678042769
0.7660818696
416
0.5127984285
0.8011695743
417
0.4751322269
0.7777777910
418
0.4133342803
0.8479532003
419
0.3562930226
0.8713450432
420
0.3192921579
0.8947368264
421
0.3691354692
0.8713450432
422
0.2948046327
0.8947368264
423
0.2826605737
0.9064327478
424
0.2911058068
0.9005848169
425
0.2562049031
0.9064327478
426
0.2182441801
0.9298245907
427
0.3062621355
0.8771929741
428
0.4564759135
0.8245614171
429
0.4321752489
0.7953216434
430
0.3488669991
0.8654970527
431
0.3374247551
0.8538011909
432
0.2492301911
0.9356725216
433
0.2211891562
0.9415204525
434
0.2172000259
0.9064327478
435
0.1869375259
0.9590643048
436
0.1562567651
0.9473684430
437
0.2887307703
0.9415204525
438
0.2108276039
0.9239766002
439
0.2128351629
0.9298245907
440
0.2501989007
0.8947368264
441
0.2585971057
0.9005848169
442
0.2206017971
0.9122806787
443
0.2229290307
0.9064327478
444
0.1330333799
0.9415204525
445
0.1317730099
0.9415204525
446
0.1210611835
0.9649122953
447
0.1067393422
0.9590643048
448
0.4022831023
0.9181286693
449
1.4179722071
0.5555555820
450
1.4231789112
0.4561403394
451
0.7788818479
0.6257309914
452
0.7895883322
0.7134503126
453
0.6300913692
0.7485380173
454
0.5853778124
0.7777777910
455
0.5166907907
0.8479532003
456
0.4636623859
0.8421052694
457
0.4148782194
0.8538011909
458
0.3744796515
0.8771929741
459
0.3448122144
0.8888888955
460
0.3005862534
0.9239766002
461
0.2766854763
0.9181286693
462
0.2488416731
0.9356725216
463
0.2258714885
0.9181286693
464
0.2115034759
0.9239766002
465
0.2024492919
0.9298245907
466
0.1845954359
0.9181286693
467
0.4039756358
0.8654970527
468
15.6660728455
0.5555555820
469
4.3343586922
0.4152046740
470
2.1533570290
0.3742690086
471
1.3684335947
0.3918128610
472
1.3066668510
0.3976608217
473
1.1771554947
0.4970760345
474
1.0907211304
0.5672514439
475
1.0130087137
0.6081871390
476
0.9545162916
0.6257309914
477
0.8882846832
0.6725146174
478
0.8269982338
0.7017543912
479
0.7850760221
0.7017543912
480
0.7031983733
0.7251461744
481
0.6816830039
0.6959064603
482
0.6385164857
0.7309941649
483
0.5735319257
0.7251461744
484
0.5039868355
0.7836257219
485
0.4826014042
0.7777777910
486
0.4105020165
0.7660818696
487
0.3799361885
0.8362573385
488
0.4325477779
0.7777777910
489
0.3693962097
0.8362573385
490
0.4027125537
0.8421052694
491
0.5500428677
0.8128654957
492
0.4162323177
0.8421052694
493
0.3780180812
0.8771929741
494
0.3225773275
0.8947368264
495
0.2893761098
0.8713450432
496
0.2466968000
0.9122806787
497
0.2267953008
0.9239766002
498
0.1790778339
0.9181286693
499
0.1790047586
0.9181286693
500
0.1532746255
0.9356725216
501
0.1318845302
0.9473684430
502
0.1558524221
0.9298245907
503
0.3580333591
0.8947368264
504
0.7112588882
0.7894737124
505
0.4584672749
0.7719298005
506
0.3823319674
0.8538011909
507
0.2852408886
0.9005848169
508
0.2305891514
0.9415204525
509
0.2089510411
0.9356725216
510
0.1760285646
0.9532163739
511
0.1701777726
0.9473684430
512
0.1507224441
0.9473684430
513
0.1215157732
0.9649122953
514
0.1186873019
0.9532163739
515
0.1158343852
0.9473684430
516
0.1219747216
0.9590643048
517
0.1082247123
0.9532163739
518
0.1567566693
0.9181286693
519
0.1477563232
0.9649122953
520
0.0995481089
0.9532163739
521
0.0920622796
0.9649122953
522
0.0878862664
0.9532163739
523
0.0720091686
0.9824561477
524
0.0778620020
0.9649122953
525
0.0688425899
0.9590643048
526
0.0874751955
0.9590643048
527
0.1042684019
0.9532163739
528
0.1020535156
0.9590643048
529
0.1160022244
0.9473684430
530
0.1217020974
0.9532163739
531
0.1160039753
0.9473684430
532
0.0740852281
0.9532163739
533
0.0833951384
0.9649122953
534
0.0747850463
0.9649122953
535
0.0711269528
0.9707602262
536
0.0698723868
0.9707602262
537
0.0662771389
0.9649122953
538
0.0648476109
0.9707602262
539
0.0459457524
0.9824561477
540
0.0437977389
0.9824561477
541
0.0463541113
0.9766082168
542
0.0427462384
0.9824561477
543
0.0440863967
0.9824561477
Figures
Camera frames become landmarks, a 30-frame buffer and one temporal gesture label.
Each detected hand is represented by 21 numerical x/y/z landmarks.
Two fixed hand slots produce 126 values per frame across a 30-frame sequence.
Six classes contribute 30 landmark sequences each, for 180 sequences in total.
Three LSTM layers and two dense layers lead to a six-way softmax output.
The 543-epoch training history records loss and training accuracy; no validation series exists.
The key lesson
The decisive choice was the representation: a gesture is a sequence of changing landmarks, not one still frame.
