探索全球前沿学术脉络

On the stability of rarefaction for stochastic viscous conservation law

arXiv:2606.24167v1 Announce Type: new Abstract: We study the asymptotic stability of rarefaction waves for one-dimensional stochastic viscous conservation laws driven by nonlinear conservative noise. In a critical scaling where stochastic energy injection and viscous dissipation compete at comparable magnitudes, standard kinetic and viscosity frameworks encounter obstructions due to regularity gaps and non-integrable profiles. To address this, we introduce a stochastic area inequality controlling accumulated energy fluctuations, a local $L^1$ contraction principle via stochastic Kru\v{z}kov doubling-of-variables that yields pathwise uniqueness without global integrability, and a modified Galerkin scheme preserving the $H^2$ energy structure. Assuming local $H^2$ regularity, we prove almost sure algebraic convergence to the rarefaction wave. For sufficiently small initial perturbations, we establish global well-posedness and sharp decay estimates in expectation. The smallness condition identifies a regime where viscous dissipation dominates stochastic injection, reflecting a structural stability threshold rather than a technical artifact. Our approach extends the analytical framework for conservative SPDEs with rough fluxes.

Bias Fitting to Mitigate Length Bias of Reward Model in RLHF

arXiv:2505.12843v2 Announce Type: replace-cross Abstract: Reinforcement Learning from Human Feedback (RLHF) relies on reward models to align large language models with human preferences. However, RLHF often suffers from reward hacking, wherein policy learning exploits flaws in the trained reward model to maximize reward scores without genuinely aligning with human preferences. A significant example of such reward hacking is length bias, where reward models usually favor longer responses irrespective of actual response quality. Previous works on tackling length bias have notable limitations, these approaches either mitigate bias without characterizing the bias form, or simply assume a linear length-reward relation. To accurately model the intricate nature of length bias and facilitate more effective bias mitigation, we propose FiMi-RM (Bias Fitting to Mitigate Length Bias of Reward Model), a framework that autonomously learns and corrects underlying bias patterns. Our approach consists of three stages: First, we warm up by training a standard reward model which inherently contains length bias. Next, we deploy a lightweight fitting model to capture the non-linear relation between length and reward. Finally, we incorporate this learned relation into the reward model, effectively decoupling length from reward while preserving preference modeling capabilities. Experimental results demonstrate that FiMi-RM achieves a more balanced length-reward distribution. Furthermore, when applied to alignment algorithms such as Direct Preference Optimization (DPO) and Best-of-N (BoN), our debiased reward model improves length-controlled win rate and reduces verbosity without compromising its performance.