Dominik Prossel

Combining Kalman Filters and Neural Networks for Stride Trajectory Estimation

Master's Thesis

Combining Kalman Filters and Neural Networks for Stride Trajectory Estimation

Advisors

Arne Küderle (M.Sc.), Prof. Dr. B. Eskofier

Duration:

01 / 2021 – 07 / 2021

 

Abstract

Walking is one of the basic means of locomotion for humans and is affected by different diseases
like multiple sclerosis, Parkinson’s, cardiopathy, and stroke. By analyzing gait, useful medical
information about the progression of such diseases in a human can be gathered [1]. Current
methods to perform gait analysis include video-based motion capture, pressure sensor carpets,
and inertial measurement units (IMUs) [1].
Compared to the other two techniques, methods using IMUs are generally cheaper and more
versatile in their application, as they do not need a sophisticated environment setup. They can be
attached to the foot or other body parts and typically measure three dimensional acceleration and
angular rate through MEMS-based accelerometers and gyroscopes. On the other hand they require
more difficult postprocessing of the raw sensor data, as the measurements have to be numerically
integrated to reconstruct the sensor trajectory [2]. From this trajectory spatial-temporal gait
parameters like stride length are then calculated.
Straight numeric integration of the values will lead to increasingly inaccurate results for longer
integration sequences. This can be attributed to noise and bias in the raw sensor data, which
results in a drift in the integrated data. However, there exist various approaches to account for
noise and drift.
One of them is to assume that the foot is almost stationary during stance phases between strides
and introduce zero velocity updates (ZUPTs) between strides [3]. These ZUPTs can for example
be used to segment the integration period into different strides and integrate every stride on its
own or be used in a Kalman filter as virtual velocity measurements and reset the velocity to zero
between strides [4, 5].
Even with those corrections applied, some strides will result in implausible or simply wrong
trajectories. These can often be tracked down to short periods of flawed orientation estimation,
which can not be corrected over the duration of one stride, but only during the next ZUPT.
To further reduce the number of outliers, it might be advantageous to incorporate more virtual
measurements modelling human gait into the filters.
One way to inject such auxiliary information into the integration process is to use a trainable
model (e.g. neuronal network) that can implicitly learn the constraints of stride trajectories.
Neural networks and especially LSTMs have been successfully used in different ways to calculate
stride trajectories from IMU data. One straight forward application are end to end networks that
take the raw IMU data and output trajectories [6]. They have also been extended to predict the
future trajectory and gait phases [7]. Other methods replace only part of the integration scheme
by a neural network, for example only the attitude estimation [8].
These approaches all have in common that the substituting neural network acts as a black box,
which does not know about the underlying navigation equations describing IMU integration. This
leads to complex networks, that are very reliant on training data and can be hard to interpret.
This work will evaluate the feasibility of using a neural network as a virtual sensor in traditional
Kalman filter integration. The numerical integration will be performed by a Kalman filter using
the known navigation equations, while the neural network can inject learned information about
the constraints of stride trajectories into it. Ideally this should result in a transparent system, that
optimally uses the physical knowledge about IMU integration as well as biomechanical knowledge
about gait, learned by the network.
This goal will be approached in two steps: The first one is to develop and train a neural network
that can accurately predict future trajectory samples from a given trajectory and generalizes well
over various gait speeds and subjects.
The second one is to investigate, if such a neural network model can be used as a virtual sensor
in an Error-State-Extended-Kalman filter for trajectory reconstruction to make it more robust to
outliers and further reduce the integration error.
The models and algorithms will initially be developed and evaluated on a data set with about
10000 strides of 14 healthy subjects, where motion capture data is available as ground truth.

 

References:

[1] Muro-de-la-Herran, Alvaro et al.: Gait Analysis Methods: An Overview of Wearable and
Non-Wearable Systems, Highlighting Clinical Applications. Sensors, 14.2, 2014.
[2] Seel, Thomas et al.: Inertial Sensors – Applications and Challenges in a Nutshell. Sensors,
20.21, 2020.
[3] Hannink, Julius et al.: Benchmarking Foot Trajectory Estimation Methods for Mobile Gait
Analysis. Sensors, 17.9, 2017.
[4] Colomar, Simon et al.: Smoothing for ZUPT-aided INSs. IEEE International Conference
on Indoor Positioning and Indoor Navigation (IPIN), 2012.
[5] Tunca, Can et al.: Inertial Sensor-Based Robust Gait Analysis in Non-Hospital Settings for
Neurological Disorders. Sensors, 17.4, 2017.
[6] do Monte Lima, Silva et al.: End-to-End Learning Framework for IMU-Based 6-DOF Odometry.
Sensors, 19.17, 2019.
[7] Su, Binbin et al.: Gait Trajectory and Gait Phase Prediction Based on an LSTM Network.
Sensors, 20.24, 2020.
[8] Weber, Daniel et al.: Neural Networks Versus Conventional Filters for Inertial-Sensor-based
Attitude Estimation. IEEE 23rd International Conference on Information Fusion (FUSION),
2020.