## Combining Kalman Filters and Neural Networks for Stride Trajectory Estimation

Arne Küderle (M.Sc.), Prof. Dr. B. Eskofier

**Duration:**

01 / 2021 – 07 / 2021

**Abstract**

Walking is one of the basic means of locomotion for humans and is affected by different diseases

like multiple sclerosis, Parkinson’s, cardiopathy, and stroke. By analyzing gait, useful medical

information about the progression of such diseases in a human can be gathered [1]. Current

methods to perform gait analysis include video-based motion capture, pressure sensor carpets,

and inertial measurement units (IMUs) [1].

Compared to the other two techniques, methods using IMUs are generally cheaper and more

versatile in their application, as they do not need a sophisticated environment setup. They can be

attached to the foot or other body parts and typically measure three dimensional acceleration and

angular rate through MEMS-based accelerometers and gyroscopes. On the other hand they require

more difficult postprocessing of the raw sensor data, as the measurements have to be numerically

integrated to reconstruct the sensor trajectory [2]. From this trajectory spatial-temporal gait

parameters like stride length are then calculated.

Straight numeric integration of the values will lead to increasingly inaccurate results for longer

integration sequences. This can be attributed to noise and bias in the raw sensor data, which

results in a drift in the integrated data. However, there exist various approaches to account for

noise and drift.

One of them is to assume that the foot is almost stationary during stance phases between strides

and introduce zero velocity updates (ZUPTs) between strides [3]. These ZUPTs can for example

be used to segment the integration period into different strides and integrate every stride on its

own or be used in a Kalman filter as virtual velocity measurements and reset the velocity to zero

between strides [4, 5].

Even with those corrections applied, some strides will result in implausible or simply wrong

trajectories. These can often be tracked down to short periods of flawed orientation estimation,

which can not be corrected over the duration of one stride, but only during the next ZUPT.

To further reduce the number of outliers, it might be advantageous to incorporate more virtual

measurements modelling human gait into the filters.

One way to inject such auxiliary information into the integration process is to use a trainable

model (e.g. neuronal network) that can implicitly learn the constraints of stride trajectories.

Neural networks and especially LSTMs have been successfully used in different ways to calculate

stride trajectories from IMU data. One straight forward application are end to end networks that

take the raw IMU data and output trajectories [6]. They have also been extended to predict the

future trajectory and gait phases [7]. Other methods replace only part of the integration scheme

by a neural network, for example only the attitude estimation [8].

These approaches all have in common that the substituting neural network acts as a black box,

which does not know about the underlying navigation equations describing IMU integration. This

leads to complex networks, that are very reliant on training data and can be hard to interpret.

This work will evaluate the feasibility of using a neural network as a virtual sensor in traditional

Kalman filter integration. The numerical integration will be performed by a Kalman filter using

the known navigation equations, while the neural network can inject learned information about

the constraints of stride trajectories into it. Ideally this should result in a transparent system, that

optimally uses the physical knowledge about IMU integration as well as biomechanical knowledge

about gait, learned by the network.

This goal will be approached in two steps: The first one is to develop and train a neural network

that can accurately predict future trajectory samples from a given trajectory and generalizes well

over various gait speeds and subjects.

The second one is to investigate, if such a neural network model can be used as a virtual sensor

in an Error-State-Extended-Kalman filter for trajectory reconstruction to make it more robust to

outliers and further reduce the integration error.

The models and algorithms will initially be developed and evaluated on a data set with about

10000 strides of 14 healthy subjects, where motion capture data is available as ground truth.

**References:**

[1] Muro-de-la-Herran, Alvaro et al.: Gait Analysis Methods: An Overview of Wearable and

Non-Wearable Systems, Highlighting Clinical Applications. Sensors, 14.2, 2014.

[2] Seel, Thomas et al.: Inertial Sensors – Applications and Challenges in a Nutshell. Sensors,

20.21, 2020.

[3] Hannink, Julius et al.: Benchmarking Foot Trajectory Estimation Methods for Mobile Gait

Analysis. Sensors, 17.9, 2017.

[4] Colomar, Simon et al.: Smoothing for ZUPT-aided INSs. IEEE International Conference

on Indoor Positioning and Indoor Navigation (IPIN), 2012.

[5] Tunca, Can et al.: Inertial Sensor-Based Robust Gait Analysis in Non-Hospital Settings for

Neurological Disorders. Sensors, 17.4, 2017.

[6] do Monte Lima, Silva et al.: End-to-End Learning Framework for IMU-Based 6-DOF Odometry.

Sensors, 19.17, 2019.

[7] Su, Binbin et al.: Gait Trajectory and Gait Phase Prediction Based on an LSTM Network.

Sensors, 20.24, 2020.

[8] Weber, Daniel et al.: Neural Networks Versus Conventional Filters for Inertial-Sensor-based

Attitude Estimation. IEEE 23rd International Conference on Information Fusion (FUSION),

2020.