Lite-DIO Is Actually What You Need for Efficient Inertial Localization

In this work, we propose a simple and effective framework (i.e., Lite-DIO), marking the first attempt to accelerate deep inertial odometry with knowledge distillation. In Lite-DIO, we first independently construct the Transformer-based teacher model and a lightweight student network. Then, adaptive transferring knowledge is enabled between the teacher model and the student network in a duallevel contrastive distillation manner. With such design, the distilled knowledge comes from not only the teacher model's predictions but also the latent high-order collaborative semantics preserved in embeddings. Extensive experiments conducted on three real-world datasets demonstrate that the proposed Lite-DIO significantly reduces model size and inference time compared to existing popular alternatives, while the compressed model still maintains competitive localization accuracy.