Evaluating Feature Dependent Noise in Preference-based Reinforcement Learning

February 09, 2026

Reading time: 2 minute

...

📝 Original Info

Title: Evaluating Feature Dependent Noise in Preference-based Reinforcement Learning
ArXiv ID: 2601.01904
Date: 2026-01-05
Authors: Yuxuan Li, Harshith Reddy Kethireddy, Srijita Das

📝 Abstract

Learning from Preferences in Reinforcement Learning (PbRL) has gained attention recently, as it serves as a natural fit for complicated tasks where the reward function is not easily available. However, preferences often come with uncertainty and noise if they are not from perfect teachers. Much prior literature aimed to detect noise, but with limited types of noise and most being uniformly distributed with no connection to observations. In this work, we formalize the notion of targeted feature-dependent noise and propose several variants like trajectory feature noise, trajectory similarity noise, uncertainty-aware noise, and Language Model noise. We evaluate feature-dependent noise, where noise is correlated with certain features in complex continuous control tasks from DMControl and Meta-world. Our experiments show that in some feature-dependent noise settings, the state-of-the-art noise-robust PbRL method's learning performance is significantly deteriorated, while PbRL method with no explicit denoising can surprisingly outperform noise-robust PbRL in majority settings. We also find language model's noise exhibits similar characteristics to featuredependent noise, thereby simulating realistic humans and call for further study in learning with feature-dependent noise robustly.

📄 Full Content

...(본문 내용이 길어 생략되었습니다. 사이트에서 전문을 확인해 주세요.)

Evaluating Feature Dependent Noise in Preference-based Reinforcement Learning

📝 Original Info

📝 Abstract

📄 Full Content

Table of Contents

Table of Contents

📝 Original Info

📝 Abstract

📄 Full Content

Start searching

No results found