Author Identifier (ORCID)
Abstract
Deploying Machine Learning (ML) applications on resource-constrained mobile devices remains challenging due to limited computational resources and poor platform compatibility. While Mobile Edge Computing (MEC) offers an offloading-based inference paradigm using GPU servers, existing approaches can be divided into non-transparent and transparent methods, with the former necessitating source-code modifications. Non-transparent offloading achieves high performance but requires intrusive code changes, limiting compatibility with diverse applications. Transparent offloading, in contrast, offers wide compatibility but introduces two challenges: performance degradation under MEC's low and fluctuating bandwidth, where per-operator remote procedure calls (RPCs) inflate end-to-end latency and energy consumption; and black-box logic reconstruction, which requires recovering the operator sequence solely from low-level runtime logs without hints from upper-layer frameworks. To address these challenges, we propose RRTO, the first high-performance transparent offloading system tailored for MEC inference that exploits the static operator sequence in ML models to eliminate repetitive RPCs. RRTO applies record-and-replay to coalesce reactive per-operator RPCs into a proactive one-shot RPC per inference, and introduces a two-stage Operator Sequence Search to reconstruct the operator sequence accurately from raw logs. Evaluation demonstrates that RRTO reduces per-inference latency and energy consumption by up to 98% relative to state-of-the-art transparent methods while achieving performance comparable to non-transparent approaches, without requiring any source-code modification.
Keywords
computation offloading, distributed system and network, mobile edge computing, model inference
Document Type
Journal Article
Date of Publication
1-1-2026
E-ISSN
15580660
ISSN
15361233
Publication Title
IEEE Transactions on Mobile Computing
Publisher
IEEE
School
School of Engineering
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Sun, Z., Guan, X., Lin, Z., Qing, Y., Song, H., Fang, Z., Chen, Z., Liu, F., Cui, H., Ni, W., & Luo, J. (2026). RRTO: A high-performance transparent offloading system for model inference in mobile edge computing. IEEE Transactions on Mobile Computing. Advance online publication. https://doi.org/10.1109/TMC.2026.3712271