Deep Reinforcement Learning-Aided Strategies for Big Data Offloading in Vehicular Networks

We consider vehicular networking scenarios where existing vehicle-to-vehicle (V2V) links can be leveraged for an effective uploading of large-size data to the network. In particular, we consider a group of vehicles where one vehicle can be designated as the \textit{leader} and other \textit{follower} vehicles can offload their data to the leader vehicle or directly upload it to the base station (or a combination of the two). In our proposed framework, the leader vehicle is responsible for receiving the data from other vehicles and processing it in order to remove the redundancy (deduplication) before uploading it to the base station. We present a mathematical framework of the considered network and formulate two separate optimization problems for minimizing (i) total time and (ii) total energy consumption by vehicles for uploading their data to the base station. We employ deep reinforcement learning (DRL) tools to obtain solutions in a dynamic vehicular network where network parameters (e.g., vehicle locations and channel coefficients) vary over time. Our results demonstrate that the application of DRL is highly beneficial, and data offloading with deduplication can significantly reduce the time and energy consumption. Furthermore, we present comprehensive numerical results to validate our findings and compare them with alternative approaches to show the benefits of the proposed DRL methods.

💡 Research Summary

The paper addresses the challenge of offloading massive vehicular data to the network by exploiting both vehicle‑to‑vehicle (V2V) and vehicle‑to‑infrastructure (V2I) links in a coordinated manner. A cluster of nearby vehicles is formed, and one vehicle is elected as the “leader” while the remaining vehicles act as “followers.” Each follower partitions its large data object into a set of smaller chunks; each chunk can be transmitted within a single time slot. Followers may either upload a chunk directly to the base station (BS) over a V2I link or first forward it to the leader over a V2V link. The leader aggregates received chunks, performs deduplication to eliminate redundant content (which is common because vehicles traveling together capture similar images or videos), and finally uploads the unique data to the BS.

Two optimization problems are formulated: (i) minimization of total transmission time, and (ii) minimization of total energy consumption of all vehicles. Decision variables include per‑slot transmission powers for V2V and V2I links, the fraction of each chunk offloaded via V2V versus V2I, and the selection of the leader. Constraints enforce power limits, latency budgets, and the requirement that every chunk be completely transmitted. The resulting problems are non‑convex mixed‑integer programs that are difficult to solve in real time because vehicle positions and channel coefficients change continuously.

To cope with this dynamic environment, the authors propose deep reinforcement learning (DRL) solutions. They design both centralized and decentralized frameworks. In the centralized case a global controller observes the full state (vehicle locations, channel gains, remaining data sizes, battery levels) and selects actions for all vehicles jointly. In the decentralized case each vehicle observes only its local state and decides independently. Three DRL algorithms are implemented and compared: Deep Q‑Network (DQN), Double DQN, and Proximal Policy Optimization (PPO). The state vector contains the current channel rates (R_v2v, R_v2i), power budgets, and redundancy ratios (β). Actions consist of (a) choosing the offloading path for each chunk (direct V2I or V2V→leader) and (b) allocating transmission power for the chosen path. The reward function is directly tied to the objective—negative total latency for the time‑minimization problem or negative total energy for the energy‑minimization problem—thereby guiding the agent toward the desired performance metric.

Simulation experiments are conducted in an urban scenario with varying numbers of vehicles (5–20), different chunk sizes (0.5–2 MB), and redundancy levels (β ranging from 0.3 to 0.8). Baselines include (1) pure V2I uploading (all vehicles send directly to the BS), (2) a static hybrid scheme where a fixed proportion of data uses V2V, and (3) a static optimization solution that does not adapt to time‑varying channels. Results show that DRL‑based policies achieve up to 30 % reduction in total transmission time and more than 25 % reduction in total energy consumption compared with the baselines. The benefit grows with higher redundancy because deduplication at the leader eliminates a larger fraction of duplicated data, reducing the amount that finally reaches the BS by up to 40 %. Centralized DRL attains the best performance, while decentralized DRL incurs only a modest 5–7 % performance loss but offers lower signaling overhead and better scalability.

The paper’s contributions are fourfold: (1) a novel chunk‑based offloading framework that integrates V2V forwarding, leader‑based deduplication, and V2I uploading; (2) rigorous mathematical models for time‑ and energy‑optimal offloading under realistic vehicular constraints; (3) the design and comparative evaluation of centralized and decentralized DRL algorithms tailored to the dynamic vehicular environment; and (4) extensive simulation evidence that DRL combined with deduplication markedly improves network efficiency. The authors suggest future extensions such as multi‑leader selection, joint edge‑computing cooperation, security and privacy mechanisms for the deduplication process, and real‑world vehicular test‑bed validation.