SensorPositionComparison - Full Pipeline#

Further Links:

A challenge to test the performance of a full pipeline on various laboratory tests.

Comparisons are performed by comparing the mean spatial and temporal gait parameters of the entire gait test. Reference gait parameters are provided using a marker based motion capture system in combination with hand labeled stride borders. The entire validation is run as a 5-fold cross-validation to allow the algorithms to optimize parameters on an independent train set.

General Information#

Dataset

The Sensor Position Comparison 2019 dataset [1] (usage example, download) contains 4x10m, 2x20m and a long walk (5 min) gait tests of 14 participants. The 4x10m and 2x20m tests are performed at 3 different speeds (slow, normal, fast). The dataset uses sensors at 6 different positions of each foot. However, for this challenge only the instep sensors are used.

Sensor System

The instep sensors are NilsPod IMU sensors (204.8 Hz). The two sensors are synchronized with each other and the motion capture system with sub-sample accuracy.

Reference System

A marker-based motion capture system by Qualisys (Opus 700+ Qualisys, 28 cameras, 20x30 m capture volume) at 100 Hz is used to track the foot position using 4 markers per foot (at the calcaneus (CAL), at the tip of the shoe (TOE), and on top of the first and the fifth metatarsal (MET1 and MET5). For this challenge the marker at the calcaneus is used to calculate stride length.

Implementation Recommendations#

A pipeline should only use the raw data of each datapoint and not further annotations provided by the dataset. The strides of the Mocap system include ALL strides (including turns). This means these strides should also be included in the algorithms output to get comparable results. At the beginning and end of the gait test, the participants was supposed to be standing still. However, some participants move slightly during this period. Hence, it is recommended to validate the static period at the beginning, if this required for the algorithm and not just assume it is there.

Notes#

The way the results are presented at the moment, each gait test is considered one datapoint. However, due to the vastly different length of the tests, this might not lead to a full fair comparison. We recommend digging deeper into the results, in case the average results for an algorithm are not as expected.

References#

[1]

Küderle, Arne, Nils Roth, Jovana Zlatanovic, Markus Zrenner, Bjoern Eskofier, and Felix Kluge. “The Placement of Foot-Mounted IMU Sensors Does Affect the Accuracy of Spatial Parameters during Regular Walking.” PLOS ONE 17, no. 6 (June 9, 2022): e0269567. https://doi.org/10.1371/journal.pone.0269567.