Is Your LLM Overcharging You? Tokenization, Transparency, and Incentives

State-of-the-art large language models require specialized hardware and substantial energy to operate. As a consequence, cloud-based services that provide access to large language models have become very popular. In these services, the price users pay for an output provided by a model depends on the number of tokens the model uses to generate it: they pay a fixed price per token. In this work, we show that this pricing mechanism creates a financial incentive for providers to strategize and misreport the (number of) tokens a model used to generate an output, and users cannot prove, or even know, whether a provider is overcharging them. However, we also show that, if an unfaithful provider is obliged to be transparent about the generative process used by the model, misreporting optimally without raising suspicion is hard. Nevertheless, as a proof-of-concept, we develop an efficient heuristic algorithm that allows providers to significantly overcharge users without raising suspicion. Crucially, we demonstrate that the cost of running the algorithm is lower than the additional revenue from overcharging users, highlighting the vulnerability of users under the current pay-per-token pricing mechanism. Further, we show that, to eliminate the financial incentive to strategize, a pricing mechanism must price tokens linearly on their character count. While this makes a provider’s profit margin vary across tokens, we introduce a simple prescription under which the provider who adopts such an incentive-compatible pricing mechanism can maintain the average profit margin they had under the pay-per-token pricing mechanism. Along the way, to illustrate and complement our theoretical results, we conduct experiments with several large language models from the $\texttt{Llama}$, $\texttt{Gemma}$ and $\texttt{Ministral}$ families, and input prompts from the LMSYS Chatbot Arena platform.

💡 Research Summary

The paper investigates the economic incentives embedded in the prevailing pay‑per‑token pricing model used by cloud‑based large language model (LLM) services. In the typical workflow, a user submits a prompt, the provider runs an LLM on proprietary hardware, and the model generates a sequence of tokens. The user is billed a fixed price per token, even though tokenization of a given string is not unique: the same output text can be represented by many different token sequences. This asymmetry creates a classic principal‑agent (moral‑hazard) problem where the provider (agent) can misreport the number of tokens to increase revenue, while the user (principal) lacks any means to verify the true token count.

The authors formalize the interaction as a principal‑agent contract: the provider observes the full generative process, including intermediate token choices, while the user only sees the reported token sequence. They define additive pricing mechanisms, focusing on the standard pay‑per‑token scheme, and model the provider’s utility as revenue minus the energy cost of generation and any additional cost of a reporting policy.

Key theoretical contributions:

1. Under pay‑per‑token pricing, the provider’s utility is directly proportional to the length of the reported token sequence, giving a clear monetary incentive to over‑report tokens.
2. If providers are forced to disclose the next‑token probability distribution at each generation step (a transparency requirement), finding the longest plausible tokenization becomes computationally hard (no polynomial‑time algorithm is known). This suggests that perfect auditing is infeasible.
3. Nevertheless, the authors design an efficient heuristic—Algorithm 2 (Heuristic‑Expand)—that searches for alternative tokenizations with high model probability. The algorithm expands high‑probability token branches, checks overall log‑probability against a threshold, and selects a longer tokenization when feasible. Empirically, the heuristic runs in sub‑second time, adds negligible GPU cost, yet yields additional revenue that far exceeds this cost across a variety of LLMs (Llama, Gemma, Ministral) and prompts from the LMSYS Chatbot Arena benchmark.

To eliminate the incentive to misreport, the paper proposes a pay‑per‑character pricing mechanism. Because token length is linearly related to character count, pricing each character equally makes the provider’s revenue independent of how the string is tokenized. This is the unique incentive‑compatible additive pricing rule. However, character‑based pricing leads to variable profit margins across different tokens. The authors therefore introduce a “average‑margin‑preserving” prescription: adjust the per‑character price so that the provider’s overall average profit margin remains unchanged when transitioning from pay‑per‑token to pay‑per‑character, while still removing any gain from token‑count manipulation.

Experimental validation shows that the heuristic can increase reported token counts by 30‑70 % on average, with the extra computational cost representing less than 5 % of total generation cost. When the pay‑per‑character scheme is applied, total revenue stays roughly constant, but the opportunity for over‑charging disappears, demonstrating the practical viability of the proposed pricing reform.

In summary, the paper makes five major points: (1) the current token‑based pricing creates a quantifiable incentive for providers to over‑charge; (2) transparency alone does not fully curb this behavior due to computational hardness; (3) low‑cost heuristics can still exploit the incentive profitably; (4) a character‑based pricing rule is the only incentive‑compatible alternative; and (5) providers can adopt this new rule without sacrificing average profit margins by following the authors’ simple adjustment formula. The work bridges technical tokenization details with economic contract theory, offering both theoretical insight and concrete policy recommendations for fairer LLM‑as‑a‑service markets.