Failure of stability of a maximal operator bound for perturbed Nevo-Thangavelu means

Let $G$ be a two-step nilpotent Lie group, identified via the exponential map with the Lie-algebra $\mathfrak g=\mathfrak g_1\oplus\mathfrak g_2$, where $[\mathfrak g,\mathfrak g]\subset \mathfrak g_2$. We consider maximal functions associated to spheres in a $d$-dimensional linear subspace $H$, dilated by the automorphic dilations. $L^p$ boundedness results for the case where $H=\mathfrak g_1$ are well understood. Here we consider the case of a tilted hyperplane $H\neq \mathfrak g_1$ which is not invariant under the automorphic dilations. In the case of Métivier groups it is known that the $L^p$-boundedness results are stable under a small linear tilt. We show that this is generally not the case for other two-step groups, and provide new necessary conditions for $L^p$ boundedness. We prove these results in a more general setting with tilted versions of submanifolds of $\mathfrak g_1$.

💡 Research Summary

The paper investigates the $L^{p}$ boundedness of maximal operators associated with spherical averages on two‑step nilpotent Lie groups $G$, when the averaging spheres are taken in a “tilted’’ $d$‑dimensional linear subspace $H\subset\mathfrak g$ that is not invariant under the natural automorphic dilations $\delta_{t}(X,U)=(tX,t^{2}U)$.

Background.
For a two‑step group $G$ with Lie algebra $\mathfrak g=\mathfrak g_{1}\oplus\mathfrak g_{2}$ (the centre $\mathfrak g_{2}$), the classical Nevo–Thangavelu maximal operator $M_{0}$ corresponds to averaging over the unit sphere in $\mathfrak g_{1}$ (i.e. $H=\mathfrak g_{1}$, $\Lambda=0$). Earlier work (Müller–Seeger, Narayanan–Thangavelu, etc.) established that for $d\ge3$ the operator is bounded on $L^{p}(G)$ precisely when $p>d/(d-1)$. In the special class of Métivier groups (those for which every non‑zero $\vartheta\in\mathfrak g_{2}^{}$ yields an isomorphism $J_{\vartheta}:\mathfrak g_{1}\to\mathfrak g_{1}^{}$) this $L^{p}$ bound persists under a small linear perturbation $\Lambda:\mathfrak g_{1}\to\mathfrak g_{2}$; i.e. the maximal operator $M_{\Lambda}$ remains bounded for the same range of $p$ provided $|\Lambda|$ is sufficiently small. This phenomenon was called “stability’’ of the bound.

Goal.
The authors ask whether such stability holds for arbitrary two‑step groups, not just Métivier ones. They show that, in general, it does not. Moreover, they derive a new necessary condition for $L^{p}$ boundedness that depends on a linear algebraic quantity $r$, the dimension of a certain subspace $V_{\Lambda,\vartheta}$ defined below.

Setup.
Let $H$ be a $d$‑dimensional plane transversal to $\mathfrak g_{2}$, written as $H={(X,\Lambda X):X\in\mathfrak g_{1}}$ with a linear map $\Lambda:\mathfrak g_{1}\to\mathfrak g_{2}$. Choose a $k$‑dimensional $C^{1}$ submanifold $\Sigma\subset\mathfrak g_{1}$ (later $k$ will be $d$ or $d-1$) and a smooth non‑negative cutoff $\chi\in C_{c}^{\infty}(\mathfrak g_{1})$. Define a surface measure $\sigma$ on $\Sigma$ and the weighted measure $d\mu=\chi,d\sigma$. For $t>0$, set $\mu_{t}$ by dilating both coordinates: $\langle f,\mu_{t}\rangle=\int_{\Sigma}f(tX,t^{2}\Lambda X),\chi(X),d\sigma(X)$. The associated maximal operator is
\