Unified Algebraic Absorption of Finite-Blocklength Penalties via Generalized Logarithmic Mapping
In finite-blocklength information theory, evaluating the fundamental limits of channel coding typically relies on normal approximations and Edgeworth expansions, which introduce additive polynomial corrections for skewness and higher-order moments. T…
Authors: Hiroki Suyari
1 Unified Algebraic Absorption of Finite-Blocklength Penalties via Generalized Logarithmic Mapping Hiroki Suyari, Member , IEEE, Abstract In finite-blocklength information theory , ev aluating the fundamental limits of channel coding typically relies on normal approximations and Edgew orth expansions, which introduce additive polynomial corrections for skewness and higher-order moments. This paper proposes an alternativ e approach: rather than appending external error terms to a Gaussian baseline, we absorb these finite-length penalties using a generalized q -algebraic framew ork. By introducing a dynamic scaling law 1 − q n = αn − 1 for the tuning parameter , we prove that the q -generalized information density corresponds to macroscopic higher -order fluctuations. Specifically , by setting this scaling constant to α = T / (3 V 2 ) (where V is the varentrop y and T is the third central moment), our framew ork recov ers the third-order coding limit, absorbing the O (1) non-Gaussian penalty without relying on Hermite polynomials. Furthermore, we demonstrate that the k -th degree term of our algebraic expansion matches the O ( n 1 − k/ 2 ) asymptotic order of the ( k + 1) -th moment Edgeworth correction. This approach unifies classical probabilistic approximations within a single algebraic structure, establishing a mathematical connection between finite-blocklength analysis and generalized logarithmic mappings. Index T erms Finite-blocklength regime, channel capacity , normal approximation, Edgeworth expansion, generalized logarithmic mapping, algebraic absorption. I . I N T R O D U C T I O N S HANNON’S channel capacity [1] dictates the fundamental limit of reliable communication as the blocklength n approaches infinity . Howe ver , in modern communication systems governed by latenc y constraints, such as ultra-reliable lo w-latency communication (URLLC), the assumption of infinite blocklength is no longer viable. Consequently , characterizing the limits of channel coding in the finite-blocklength regime has emerged as a central problem in contemporary information theory . The foundational framew ork for the second-order asymptotic analysis in this regime was established by Polyanskiy , Poor, and V erd ´ u [2], who demonstrated that the maximum achiev able coding rate can be approximated by the normal approximation, and independently developed from the perspective of information spectrum methods by Hayashi [3]. This second-order formulation refines the asymptotic Shannon limit by introducing a penalty term characterized by the channel dispersion, or varentropy . While the normal approximation captures the dominant finite-length behavior , it assumes a Gaussian distribution for the information density . Howev er, the information density of discrete memoryless channels exhibits asymmetric tail behaviors, particularly in the short-blocklength regime. T o correct this discrepancy , classical probability theory employs the Edgew orth expansion (and its in verse, the Cornish-Fisher expansion) [4], [5] to incorporate higher-order moments. By introducing the third central moment (skewness), the third-order asymptotic expansion provides an O (1) constant correction to the coding rate. This traditional approach requires appending combinations of orthogonal Hermite polynomials to a Gaussian baseline to ev aluate finite-length penalties. As one pursues higher-order precision (e.g., kurtosis), this additi ve methodology leads to a combinatorial explosion. In this paper , we propose an alternati ve to the additiv e polynomial approach. Rather than treating the non-Gaussian tail behaviors as external error penalties that must be corrected, we in vestigate whether the information measure itself can be algebraically generalized to absorb these finite-length fluctuations. T o address this, we introduce a unified framework based on the generalized algebraic structure originating from non-extensiv e statistical mechanics [6], specifically utilizing the q -generalized logarithmic mapping [7]. Originally studied in generalized algebra as a one-parameter deformation of the natural logarithm, we apply this structural transformation to the centralized information density . W e prov e that by scaling the tuning parameter as 1 − q n = αn − 1 , the intrinsic algebraic structure of the q -logarithm generates the polynomial forms required by finite-blocklength analysis. The main contributions of this paper are summarized as follows: • W e define the centralized q -generalized information density , utilizing the moment-generating function to conserve the macroscopic Shannon limit. • W e prove that the dynamic scaling law 1 − q n = αn − 1 is the necessary and sufficient condition to renormalize the O ( n ) quadratic fluctuations down to an O (1) scale. H. Suyari is with Graduate School of Informatics, Chiba Univ ersity , 1-33, Y ayoi-cho, Inage-ku, Chiba 263-8522, Japan (e-mail: suyari@ieee.org, suyari@faculty .chiba-u.jp, suyarilab@gmail.com). 2 • W e demonstrate that by tuning the fundamental constant to α = T 3 V 2 , where V is the v arentropy and T is the third central moment of the information density , the proposed algebraic framework absorbs the third-order ske wness penalty , coinciding with the classical Edgeworth expansion. • W e sho w a mathematical correspondence: the k -th degree term of the q -algebraic expansion yields the univ ersal asymptotic order O ( n 1 − k/ 2 ) , aligning with the ( k + 1) -th moment Edgew orth correction. By embedding the finite-length variations directly into the algebraic structure of the information measure, our frame work circumvents the combinatorial complexity of classical probability theory , offering a unified perspectiv e on finite-blocklength limits. The remainder of this paper is organized as follows. Section II revie ws the classical finite-blocklength asymptotics and the Edgeworth expansion. Section III formally introduces the q -algebraic framew ork and derives the dynamic scaling law . Section IV provides the mathematical proof of the exact absorption of the third-order skewness penalty . Section V presents numerical validations of the generalized limits. Section VI discusses the universal resonance of higher-order asymptotic orders, and Section VII concludes the paper . I I . P R E L I M I NA R I E S : F I N I T E - B L O C K L E N G T H I N F O R M A T I O N T H E O RY W e briefly revie w the standard tools and the current limitations of finite-blocklength information theory . In particular , we focus on the probabilistic behavior of information content and the con ventional asymptotic expansions used to ev aluate coding limits. A. Information Density and V ar entropy Consider a discrete memoryless information source producing a sequence of independent and identically distributed (i.i.d.) random variables X n = ( X 1 , X 2 , . . . , X n ) drawn from a known probability distribution P ( X ) . In the finite-blocklength regime, the fundamental random variable characterizing the system is the information density . Throughout this paper, ln denotes the natural logarithm, and all information measures are expressed in nats. It is defined as S n := − ln P ( X n ) = n X i =1 − ln P ( X i ) . (1) In classical asymptotic theory ( n → ∞ ), the Law of Large Numbers dictates that 1 n S n con verges to its expectation, the Shannon entropy H 1 = E [ − ln P ( X )] . Howe ver , for a finite n , the fluctuation of S n around its mean plays a critical role. This fluctuation is governed by the variance of the self-information, termed varentr opy , defined as V := V ar[ − ln P ( X )] = E h ( − ln P ( X ) − H 1 ) 2 i . (2) B. Second-Order Asymptotics W ork by Polyanskiy , Poor , and V erd ´ u [2] demonstrated that incorporating v arentropy is essential for ev aluating the fun- damental limits of coding at finite blocklengths. For instance, in almost-lossless source coding, the minimum required code length L ⋆ ( n, ϵ ) to achiev e a target error probability ϵ is giv en by the second-order asymptotic expansion: L ⋆ ( n, ϵ ) = nH 1 + √ nV Q − 1 ( ϵ ) + O (1) , (3) where Q − 1 ( · ) is the inv erse of the standard Gaussian complementary cumulativ e distribution function. The term √ nV Q − 1 ( ϵ ) represents the backoff (or penalty) required to absorb the finite-length fluctuations under a normal approximation guaranteed by the Central Limit Theorem. C. Limitations of the Normal Appr oximation While (3) provides an approximation, its reliance on the normal approximation limits its accuracy when the blocklength n is short or when the underlying distribution P ( X ) is skewed. For an asymmetric distribution (e.g., a binary asymmetric source with extreme probabilities p ≪ 1 − p ), the distribution of the information density S n exhibits ske wness and kurtosis. In such cases, the error induced by the normal approximation becomes large. T o correct this discrepancy , classical probability theory employs the Edgew orth expansion (and its in verse, the Cornish- Fisher expansion) [8], [4], [9], [5] to incorporate higher-order moments. By introducing the third central moment (ske wness), the third-order asymptotic expansion provides an O (1) constant correction to the coding rate: L ⋆ ( n, ϵ ) = nH 1 + √ nV Q − 1 ( ϵ ) + B ( T , ϵ ) + O 1 √ n , (4) 3 where B ( T , ϵ ) is an O (1) correction term that explicitly depends on the third moment T and the target error ϵ . If higher precision is required, further nonlinear correction terms inv olving the fourth moment (kurtosis) must be analytically deri ved and appended. This necessity to derive and append correction terms highlights a challenge within the conv entional normal approximation paradigm. This motiv ates a unified mathematical framework that absorbs these higher-order fluctuations without requiring step-by-step analytical expansions. I I I . T H E q - A L G E B R A I C F R A M E W O R K F O R F I N I T E - B L O C K L E N G T H W e first formally introduce the mathematical operators of the q -algebraic framew ork. The mathematical foundation of the generalized algebraic framew ork relies on the q -logarithm [10], [11], defined for x > 0 and q = 1 as: ln q x := x 1 − q − 1 1 − q . (5) It is straightforward to verify via L ’H ˆ opital’ s rule that this generalized function recov ers the standard natural logarithm in the limit, lim q → 1 ln q x = ln x . Correspondingly , the generalized entropy (Tsallis entropy) for a probability distribution P is defined as the q -expectation of the q -information: H q ( P ) := X x P ( x ) ln q 1 P ( x ) = 1 − P x P ( x ) q q − 1 . (6) Remark 1 (Notation on Entr opy and Information Density): In the statistical mechanics literature (e.g., [6]), the generalized macroscopic entropy is con ventionally denoted by S q . T o align with the standard conv entions of information theory and prevent ambiguity , this paper consistently uses H q (and H 1 ) to denote the macroscopic entropy as an expectation, while reserving S q (and S n ) to denote the microscopic information density as a random variable. T o understand how this q -algebraic framework relates to finite-blocklength information theory , we examine the local behavior of these functions near the Shannon limit ( q = 1 ). The T aylor expansion of the q -logarithm around q = 1 yields: ln q x = ln x + 1 − q 2 (ln x ) 2 + O (1 − q ) 2 . (7) The implication of this algebraic expansion becomes clear when applied to the macroscopic entropy . Let us recall the definition of varentropy , which represents the variance of the information density: V ( P ) := X x P ( x ) ln 1 P ( x ) 2 − ( H 1 ( P )) 2 . (8) By utilizing this variance, the generalized entropy mathematically decomposes from the q -logarithmic expansion as follows: H q ( P ) = H 1 ( P ) + 1 − q 2 V ( P ) + ( H 1 ( P )) 2 + O (1 − q ) 2 . (9) This expansion constitutes a mathematical connection between non-extensi ve statistical mechanics and second-order information theory . A. Blocklength Extension and q -Information Density Extending (9) to a sequence of blocklength n yields further insights. Consider an i.i.d. sequence X n = ( X 1 , . . . , X n ) with joint distribution P ( X n ) = Q n i =1 P ( X i ) . For this block, both Shannon entropy and varentropy are additive: H 1 ( P ( n ) ) = nH 1 and V ( P ( n ) ) = nV . Applying (9) to the block X n giv es the macroscopic q -entropy: H q ( P ( n ) ) = nH 1 + 1 − q 2 nV + n 2 H 2 1 + O (1 − q ) 2 . (10) The term n 2 H 2 appears in this expansion. As the blocklength n increases, this O ( n 2 ) term represents a macroscopic fluctuation. Furthermore, modern finite-blocklength theorems are governed by the tail probabilities of the random variable itself. Thus, we define the q -information density as a random variable: S q ( X n ) := ln q 1 P ( X n ) . (11) Using (7), this random variable expands locally around q = 1 as: S q ( X n ) = S n + 1 − q 2 S 2 n + O (1 − q ) 2 , (12) where S n = − ln P ( X n ) is the classical information density defined in (1). Since S n scales as O ( n ) by the Law of Large Numbers, its square S 2 n scales as O ( n 2 ) , mirroring the macroscopic behavior seen in (10). 4 B. Dynamic Scaling of the q -P arameter Con ventional finite-blocklength analysis relies on normal approximations and Edge worth expansions to ev aluate the tail bounds of S n . Rather than treating the generalized parameter q as a fixed universal constant, we introduce a tuning parameter , q n , that scales with the blocklength n . T o determine the required scaling law , we apply the q -logarithmic transformation to the centered fluctuation of the information density . Let W n = S n − nH 1 denote this macroscopic fluctuation, which scales as O ( n 1 / 2 ) by the Central Limit Theorem. Note that W n has zero mean ( E [ W n ] = 0 ) and variance E [ W 2 n ] = nV . Mathematically , we introduce the transformation governed by the q -logarithm: ln q ( x ) = x 1 − q − 1 1 − q . If one were to directly apply this mapping to the exponential of the fluctuation, the resulting measure would be ln q n (exp( W n )) = exp((1 − q n ) W n ) − 1 1 − q n . Howe ver , because W n is a random variable, the non-linear nature of this exponential mapping would shift the expected value of the information density away from the fundamental Shannon limit nH 1 . T o conserve this deterministic macroscopic mean, the algebraic transformation must be centralized by subtracting its expectation, which corresponds to the moment-generating function (MGF) of the fluctuation. Definition 2 (Centralized q -Generalized Information Density): T o conserve the fundamental limit E [ S q n ( X n )] = nH 1 , the centralized q -generalized information density is defined as: S q n ( X n ) = nH 1 + exp((1 − q n ) W n ) − E [exp((1 − q n ) W n )] 1 − q n . (13) W ith this definition, we can determine the scaling behavior required to ev aluate the finite-blocklength limits. Pr oposition 3 (Dynamic Scaling Law for Skewness Absorption): T o generate the finite-length O (1) skewness correction directly from the algebraic structure of the generalized information density , the tuning parameter (1 − q n ) must scale as: 1 − q n = O ( n − 1 ) . (14) Pr oof: W e deriv e the polynomial behavior by expanding both the exponential and its expectation via T aylor series with respect to (1 − q n ) : exp((1 − q n ) W n ) = 1 + (1 − q n ) W n + (1 − q n ) 2 2 W 2 n + (1 − q n ) 3 6 W 3 n + . . . (15) and its expectation (noting that E [ W n ] = 0 and E [ W 2 n ] = nV ): E [exp((1 − q n ) W n )] = 1 + (1 − q n ) 2 2 nV + (1 − q n ) 3 6 E [ W 3 n ] + . . . (16) Substituting these expansions back into (13) and dividing by (1 − q n ) , the constant term ( 1 ) cancels out, yielding the centralized formulation: S q n ( X n ) = nH 1 + W n + 1 − q n 2 ( W 2 n − nV ) + (1 − q n ) 2 6 ( W 3 n − E [ W 3 n ]) + . . . (17) This probabilistic centralization yields the quadratic term ( W 2 n − nV ) . T o analyze its asymptotic contribution under the Gaussian baseline, we substitute the normalized fluctuation W n = √ nV Z (where Z ∼ N (0 , 1) ). This term then maps to nV ( Z 2 − 1) , which generates the second Hermite polynomial H e 2 ( Z ) = Z 2 − 1 . Since W n = O ( n 1 / 2 ) , the quadratic term scales as W 2 n = O ( n ) . In con ventional finite-blocklength formulas, howe ver , the ske wness correction (deriv ed via the Berry- Esseen or Edgeworth expansion using this Hermite polynomial) scales as an O (1) constant, because the O ( n − 1 / 2 ) ske wness cancels with the O ( n 1 / 2 ) standard deviation. T o renormalize the O ( n ) fluctuation down to O (1) within the quadratic term 1 − q n 2 O ( n ) , the parameter (1 − q n ) must be tuned as O ( n − 1 ) . This completes the proof. Remark 4 (Absorption of Kurtosis): Under this scaling la w 1 − q n = O ( n − 1 ) , the subsequent cubic term (1 − q n ) 2 6 ( W 3 n − E [ W 3 n ]) in our algebraic expansion scales as O ( n − 2 ) × O ( n 3 / 2 ) = O ( n − 1 / 2 ) . This matches the asymptotic order of the fourth-moment (kurtosis) correction in the classical Edgew orth expansion, demonstrating that the generalized framework absorbs higher-order probabilistic penalties via its algebraic structure. I V . M A T H E M ATI C A L P R O O F O F T H E S E C O N D - O R D E R C O D I N G R AT E In this section, we prov e that the proposed q -algebraic framew ork, endowed with the dynamic scaling law 1 − q n = O ( n − 1 ) , recov ers the higher-order coding rates established by the conv entional normal and Edgeworth approximations. A. The Renormalization of Higher-Or der Moments Unlike con ventional additiv e corrections, our framework incorporates higher-order fluctuations into the algebraic properties of the measure. By adopting S q n ( X n ) as the fundamental measure, we renormalize the macroscopic fluctuations directly into the operational structure of the information measure. 5 B. Information Spectrum under q -Algebra Definition 5 ( q -Generalized Information Spectrum Limit): Extending the classical information spectrum method [12], [2], we define the q -generalized information spectrum limit L q as the threshold satisfying the tail probability under the generalized measure: P ( S q n ( X n ) ≤ L q ) ≥ 1 − ϵ. (18) T o isolate the algebraic contrib ution of the q -deformation, we e valuate this limit by mapping the macroscopic fluctuation W n to its Gaussian equiv alent √ nV Z (where Z ∼ N (0 , 1) ). Under this normal baseline, the q -generalized random variable is expressed as: ˜ S q n ( Z ) = nH 1 + √ nV Z + 1 − q n 2 nV ( Z 2 − 1) + O ( n − 1 / 2 ) . (19) C. Matching the Edgeworth Expansion T o confirm that our framew ork recovers the discrete finite-blocklength results, we examine the boundary condition. Theor em 6 (Algebr aic Absorption of Ske wness): By ev aluating the q -generalized limit under the standard normal baseline and setting the scaling constant to: α = T 3 V 2 (20) where V is the varentrop y and T is the third central moment (skewness), the resulting boundary condition coincides with the exact third-order Edgeworth expansion of the original information density . Pr oof: Applying the dynamic scaling 1 − q n = αn − 1 to the normal-approximated random variable ˜ S q n ( Z ) , the limit L q satisfying P ( ˜ S q n ( Z ) ≤ L q ) = 1 − ϵ is deriv ed by ev aluating the quantile at Z ϵ = Q − 1 ( ϵ ) : L q = nH 1 + √ nV Z ϵ + αn − 1 2 nV ( Z 2 ϵ − 1) = nH 1 + √ nV Z ϵ + αV 2 ( Z 2 ϵ − 1) . (21) In classical probability theory , the third-order fluctuation of the original information density S n is deriv ed via the Cornish-Fisher (Edgew orth) expansion, which introduces a skewness penalty: L edge = nH 1 + √ nV Z ϵ + T 6 V ( Z 2 ϵ − 1) . (22) Equating the q -algebraic residual term L q to the analytical Cornish-Fisher expansion L edge to enforce structural matching yields: αV 2 = T 6 V = ⇒ α = T 3 V 2 . (23) Thus, by tuning the algebraic deformation via α , the q -measure ev aluated under the Gaussian baseline absorbs the skewness penalty of the true distribution. This confirms that the q -parameter acts as a structural control parameter . By choosing α appropriately , the q -generalized information density S q n becomes a sufficient statistic for the finite-length coding limit. V . N U M E R I C A L V A L I DA T I O N F O R T H E S E C O N D - O R D E R A S Y M P T OT I C S In this section, we numerically validate the q -algebraic framework within the second-order regime to demonstrate how the dynamic tuning parameter q n captures the finite-blocklength penalty . A. Simulation Setup and the Canonical Problem T o evaluate our theoretical bound, we consider a discrete memoryless coding scenario: a binary memoryless source (Bernoulli source). Let the source alphabet be X = { 0 , 1 } with a probability distribution P ( X = 1) = p . For the numerical ev aluation, we set the source parameter to p = 0 . 11 (an asymmetric source) and the target error probability to ϵ = 0 . 01 . The exact fundamental limit L ⋆ ( n, ϵ ) for a finite blocklength n can be computed strictly from the cumulative distribution function (CDF) of the binomial distribution, without relying on any asymptotic approximations. W e compare the following three fundamental quantities: • Shannon Limit (Asymptotic): The first-order theoretical baseline, nH 1 . • Normal Appr oximation (V erd ´ u): The standard second-order formulation incorporating the varentropy penalty , L norm = nH 1 + √ nV Q − 1 ( ϵ ) . • Pr oposed q -Algebraic Bound: The limit deriv ed from our generalized information spectrum L q , dynamically incorporating the skewness correction via the tuning parameter 1 − q n = αn − 1 with α = T 3 V 2 . 6 25 50 75 100 125 150 175 200 B l o c k l e n g t h n 0.5 0.6 0.7 0.8 0.9 Coding R ate (nats/symbol) C o m p a r i s o n o f F i n i t e - B l o c k l e n g t h B o u n d s ( p = 0 . 1 1 , = 0 . 0 1 ) Exact F inite-L ength Limit Nor mal Appr o x. (2nd Or der) Edgeworth Exp. (3r d Or der) P r o p o s e d q - A l g e b r a i c B o u n d 20 25 30 35 40 45 50 0.60 0.65 0.70 0.75 Fig. 1. Unified comparison of the exact finite-blocklength limit, normal approximation, Edgeworth expansion, and the proposed q -algebraic bound. The inset highlights the short-blocklength regime ( n ∈ [20 , 50] ), demonstrating the algebraic absorption of the skewness penalty . B. Discussion: The Second-Order Discr epancy The normal approximation improv es over the asymptotic Shannon limit. Ho wever , as illustrated by the blue dashed line in Fig. 1, in the short blocklength regime (e.g., n < 100 ), the normal approximation exhibits a deviation from the exact finite-length step behavior . This discrepancy arises because the normal approximation truncates the third-order skewness and higher-order moments. T o compensate for this gap without relying on additiv e error terms, we examine the higher-order generalizations discussed in the next section. V I . H I G H E R - O R D E R A S Y M P T O T I C S A N D T H E R E S O NA N C E O F E R R O R P E N A LT I E S In the previous sections, we established that the fundamental dynamic scaling 1 − q n = αn − 1 coupled with α = T 3 V 2 absorbs the third-order O (1) skewness penalty . This framew ork can be extended to arbitrary higher-order regimes. A. The Generalized O ( n 1 − k/ 2 ) Resonance Theor em 7 (Algebraic Resonance of Asymptotic Or ders): Let the centered fluctuation be W n = O ( n 1 / 2 ) and the generalized parameter scale as 1 − q n = O ( n − 1 ) . The k -th degree term of the q -logarithmic algebraic expansion matches the asymptotic order of the ( k + 1) -th moment correction in the classical Edgew orth expansion, yielding the univ ersal order O ( n 1 − k/ 2 ) . Pr oof: The q -deformation of the centered fluctuation W n expands generally as: S q n ( X n ) = nH 1 + W n + ∞ X k =2 (1 − q n ) k − 1 k ! ( W k n − E [ W k n ]) . (24) By substituting the standard growth rate of the fluctuation W n = O ( n 1 / 2 ) and the dynamic scaling limit 1 − q n = O ( n − 1 ) into the k -th degree term, the asymptotic order ev aluates to: O ( n − 1 ) k − 1 × O ( n 1 / 2 ) k = O n − k +1 × O n k/ 2 = O n 1 − k/ 2 . (25) 7 Evaluating (25) for successiv e values of k shows a structural correspondence with the classical finite-blocklength limits: • k = 1 = ⇒ O ( n 1 / 2 ) : Recovers the normal varentrop y penalty . • k = 2 = ⇒ O (1) : Generates the third-order ske wness correction, fully absorbed via α = T 3 V 2 . • k = 3 = ⇒ O ( n − 1 / 2 ) : Matches the asymptotic order of the fourth-order kurtosis penalty . • k = m = ⇒ O ( n 1 − m/ 2 ) : Aligns with the corresponding ( m + 1) -th moment correction order for any m ≥ 1 . B. Beyond the Third Or der The generalized order relation O ( n 1 − k/ 2 ) provides evidence that the dynamic parameter q n acts as an algebraic generator for finite-length corrections. The single scaling rule 1 − q n = O ( n − 1 ) establishes a relationship that offsets the macroscopic behavior of higher-order moments without the combinatorial complexity of accumulating orthogonal polynomials. While this paper has proven the absorption of the fluctuations up to the third order (ske wness) by matching the algebraic coefficients, the emergence of the O ( n − 1 / 2 ) scale for k = 3 suggests that the generalized framew ork encapsulates the kurtosis and beyond. Elucidating a comprehensive structural mapping between the classical Hermite polynomial basis and the q - logarithmic T aylor coefficients for k ≥ 3 remains an open problem. Resolving this will complete a mathematical connection between traditional probability theory and our generalized algebraic framew ork. C. Numerical V alidation of the Skewness Absorption T o visually substantiate the algebraic absorption of the third-order skewness penalty (as prov en in Theorem 6), we direct our attention back to the unified plot in Fig. 1, specifically focusing on the third-order limits and the magnified inset. As shown in the inset of Fig. 1, while the second-order normal approximation does not capture the step behavior , the third-order Edgeworth expansion (green dash-dotted line) corrects this de viation via an external additiv e polynomial. Our proposed q -algebraic bound (red solid line) coincides with the Edgew orth curve, overlapping across the finite-length regime. This numerical match supports our theoretical assertion: the non-Gaussian tail behavior of the information density does not require external polynomial corrections. Instead, by appropriately scaling the dynamic parameter 1 − q n = O ( n − 1 ) , the generalized information measure structurally transforms to absorb the finite-length penalty . V I I . C O N C L U S I O N In this paper, we presented a generalized framework in finite-blocklength information theory by transitioning from additiv e probabilistic corrections to a unified algebraic absorption via generalized logarithmic mappings. Classical higher-order asymp- totics rely on appending Hermite polynomials to a Gaussian baseline, which leads to a combinatorial explosion. T o ov ercome this, we introduced the dynamically scaled q -generalized information density . By treating the non-Gaussian tail behaviors as intrinsic algebraic properties of the information measure rather than external error penalties, we renormalized the macroscopic fluctuations. Our results demonstrate that by applying the dynamic scaling law 1 − q n = O ( n − 1 ) to the centered fluctuation W n , the algebraic structure of the q -logarithm corresponds to the higher-order moments. Specifically , by identifying the tuning parameter as α = T 3 V 2 , the proposed framework matches the discrete finite-blocklength limits up to the third order , generating the O (1) ske wness correction. Furthermore, we proved that the k -th degree term of our generalized expansion yields the asymptotic order O ( n 1 − k/ 2 ) , aligning with the ( k + 1) -th moment Edgew orth correction. The explicit coefficient mapping for the fourth order and beyond remains for future research. This algebraic approach encompasses classical probabilistic approximations, establishing a unified framew ork for finite-blocklength information theory . R E F E R E N C E S [1] C. E. Shannon, “ A mathematical theory of communication, ” The Bell System T echnical Journal , vol. 27, no. 3, pp. 379–423, 1948. [2] Y . Polyanskiy , H. V . Poor, and S. V erd ´ u, “Channel coding rate in the finite blocklength regime, ” IEEE T ransactions on Information Theory , vol. 56, no. 5, pp. 2307–2359, 2010. [3] M. Hayashi, “Information spectrum approach to second-order coding rate in channel coding, ” IEEE T ransactions on Information Theory , vol. 55, no. 11, pp. 4947–4966, Nov 2009. [4] W . Feller , An Introduction to Probability Theory and Its Applications, V ol. 2 , 2nd ed. Ne w Y ork: Wile y , 1971. [5] V . Y . F . T an, “ Asymptotic estimates in information theory with non-vanishing error probabilities, ” F oundations and Tr ends in Communications and Information Theory , vol. 11, no. 1-2, pp. 1–184, 2014. [6] C. Tsallis, “Possible generalization of boltzmann-gibbs statistics, ” Journal of Statistical Physics , vol. 52, pp. 479–487, 1988. [7] ——, Introduction to Nonextensive Statistical Mechanics: Appr oaching a Complex W orld . Springer , New Y ork, 2009. [8] V . Strassen, “ Asymptotische Absch ¨ atzungen in Shannons Informationstheorie, ” in T ransactions of the Third Prague Conference on Information Theory , Statistical Decision Functions, Random Processes . Prague: Czechoslovak Academy of Sciences, 1962, pp. 689–723. [9] M. T omamichel and V . Y . F . T an, “ A tight upper bound for the third-order asymptotics for most discrete memoryless channels, ” IEEE T ransactions on Information Theory , vol. 59, no. 11, pp. 7041–7051, 2013. [10] H. Suyari, “Mathematical structures derived from the q -multinomial coefficient in tsallis statistics, ” Physica A , vol. 368, pp. 63–82, 2006. [11] H. Suyari, H. Matsuzoe, and A. M. Scarfone, “ Advantages of q -logarithm representation over q -exponential representation from the sense of scale and shift on nonlinear systems, ” The Eur opean Physical Journal Special T opics , vol. 229, pp. 773–785, 2020. [12] T . S. Han and S. V erd ´ u, “ Approximation theory of output statistics, ” IEEE T ransactions on Information Theory , vol. 39, no. 3, pp. 752–772, 1993.
Original Paper
Loading high-quality paper...
Comments & Academic Discussion
Loading comments...
Leave a Comment