ICP-Based Pallet Tracking for Unloading on Inclined Surfaces by Autonomous Forklifts

This paper proposes a control method for autonomous forklifts to unload pallets on inclined surfaces, enabling the fork to be withdrawn without dragging the pallets. The proposed method applies the Iterative Closest Point (ICP) algorithm to point clouds measured from the upper region of the pallet and thereby tracks the relative position and attitude angle difference between the pallet and the fork during the unloading operation in real-time. According to the tracking result, the fork is aligned parallel to the target surface. After the fork is aligned, it is possible to complete the unloading process by withdrawing the fork along the tilt, preventing any dragging of the pallet. The effectiveness of the proposed method is verified through dynamic simulations and experiments using a real forklift that replicate unloading operations onto the inclined bed of a truck.

💡 Research Summary

The paper addresses a practical challenge in modern logistics: unloading pallets onto non‑horizontal surfaces such as a truck bed that may be tilted due to load‑induced suspension sag or intentional drainage slopes. Conventional autonomous forklift solutions assume a flat target plane; they keep the fork horizontal while lowering the pallet and then pull the fork straight out. When the target surface is inclined, this approach either drags the pallet or causes the fork to collide with the pallet, leading to damage or operational delays.

To overcome this, the authors propose a real‑time perception‑control loop that continuously estimates the relative pose (height and tilt) between the pallet and the fork using the Iterative Closest Point (ICP) algorithm applied to point clouds captured by an RGB‑D camera mounted on the mast of the forklift. The workflow is as follows:

1. Sensing – An Azure Kinect (640 × 576, 75° × 65° FoV) mounted on the inner mast captures a dense point cloud of the pallet’s upper region. A bounding box (BB) that roughly matches the pallet dimensions isolates the region of interest. The cloud is down‑sampled to about 7 000 points by random sampling to keep processing time constant.

2. ICP‑based tracking – At each control cycle (≈5 Hz) the previously stored “source” cloud (taken at the start of the fork descent) is translated in the camera frame by the known fork descent distance, thereby reducing the initial misalignment and accelerating convergence. The translated source cloud and the newly measured cloud are fed to a GPU‑accelerated ICP implementation (cuPCL). The resulting transformation matrix C_T_icp(t) is multiplied by the known static transform from the camera to the forklift base, yielding the relative rotation Δo_R_pallet(t) and translation Δo_height(t).

3. Pose extraction – From Δo_R_pallet(t) the tilt angle difference is computed as
   Δtilt(t) = atan2(e_x·Δo_R·e_z, e_z·Δo_R·e_z)
   where e_x and e_z are unit vectors along the forklift’s lateral and forward axes. The height difference is simply Δheight(t) = e_z·Δo_height(t).

4. Tilt control – A simple proportional controller adjusts the hydraulic tilt actuator of the fork until |Δtilt| falls below a threshold (≈0.25°). Because the hydraulic system is relatively slow and exhibits one‑way force limits, the controller updates at 50 Hz via UDP to a MicroAutoBox II (MAB) that directly commands the actuators.

5. Load detection and withdrawal – A limit switch mounted at the fork base detects when the pallet has completely left the fork. Once the switch goes OFF and the tilt is aligned, the fork is commanded to withdraw along a diagonal trajectory that matches the estimated surface inclination. The withdrawal distance is derived from wheel odometry; a proportional height‑error controller ensures the fork stays on the intended line, preventing inadvertent contact with the pallet during retraction.

The authors validate the approach in two ways. First, they implement a high‑fidelity simulation in Choreonoid, modeling the hydraulic cylinders with PD controllers, one‑way force limits, and a first‑order lag to emulate real actuation dynamics. Four test cases are examined: (1) a box on an inclined bed with the proposed controller, (2) the same scenario without tilt control, (3) a cage containing cabbage on an inclined bed with the controller, and (4) the same without control. In the controlled cases the fork’s tilt converges to the target angle (≈‑4° for the box, ≈ 2° for the cage) and the withdrawal completes without dragging the load. In the uncontrolled cases the fork remains in contact, the limit switch never releases, or the cage is dragged after withdrawal begins.

Second, a physical experiment uses a Toyota AGF‑Rinova 8AFBR15 forklift equipped with the same sensor‑actuator stack. The perception‑control pipeline runs on an NVIDIA Jetson Xavier, while the MAB handles low‑level hydraulic commands. The system operates synchronously with the RGB‑D stream at 5 Hz and sends state updates at 50 Hz. Experimental results mirror the simulation: with the controller the fork aligns to within 0.2° of the target inclination, the limit switch reliably signals load release, and the pallet is retracted cleanly. Without the controller the fork either stalls (limit switch stays high) or drags the pallet during withdrawal.

Key contributions include: (i) a real‑time ICP‑based method for estimating pallet pose on short, constant‑duration cycles, (ii) a simple yet effective tilt‑control algorithm that compensates for hydraulic latency and one‑way force constraints, and (iii) an integrated hardware‑software framework that bridges simulation and real‑world testing via ROS nodes and common messaging.

The paper demonstrates that accurate, low‑latency 3‑D perception combined with straightforward geometric control can enable autonomous forklifts to handle inclined unloading tasks safely, expanding the applicability of warehouse automation to semi‑outdoor and variable‑inclination environments. Limitations noted by the authors involve sensitivity to point‑cloud quality under poor lighting or dust, and the need for further validation in multi‑pallet, multi‑forklift scenarios. Future work may explore sensor fusion (e.g., LiDAR + RGB‑D), adaptive ICP parameter tuning, and coordinated fleet control to scale the solution for high‑throughput logistics operations.