key: cord-0057400-1r0mtw1n
authors: Bakiev, T. N.; Nakashidze, D. V.; Savchenko, A. M.
title: Certain Relations in Statistical Physics Based on Rényi Entropy
date: 2021-03-14
journal: Moscow Univ
DOI: 10.3103/s002713492006003x
sha: 4e51f3edc0aadcf64f895d3a5f20397f40c4a0f0
doc_id: 57400
cord_uid: 1r0mtw1n

The statistical theory based on the parametric family of Rényi entropy functionals is a generalization of Gibbs statistics. Depending on the value of the involved parameter, the corresponding Rényi distribution can take both an exponential form and a power-law form, which is typical for a wide range of statistical models. In this paper, we prove the energy equipartition theorem in the case of Rényi statistics, which makes it possible to solve the problem of obtaining the average energy for a large number of classical statistical models. The proposed approach for calculating the average energy is compared with the procedure for directly calculating this quantity for a system described by the simplest power-low Hamiltonian. New relations are presented that simplify the calculations in the considered theory. A special case of the Rényi distribution, which represents a generalization of a power-low distribution and thus allows us to approximate some empirical data more precisely, has been studied.

The entropy functional proposed by the Hungarian mathematician Alfred R´enyi [1, 2] makes it possible to generalize the famous Boltzmann-Gibbs-Shannon entropy [3] , which is widely used in equilibrium statistical physics. The procedure for maximizing the R´enyi entropy functional under certain conditions leads to the R´enyi distribution [4] . It is important to note that the limiting cases of this distribution are the exponential Gibbs distribution and the power-law distribution. The latter is known as the Pareto distribution [5] or Zipf's law [6] and is typical for a wide range of self-organizing systems [7] , phenomena of fracture theory [8] , for describing atmospheric cascades of cosmic rays [9] , and also for a large number of statistical models that arise in sciences such as economics [10] , biology [11] , network theory [12] , and linguistics [13] . The R´enyi distribution itself can be applied to the description of multifractal structures [14] , turbulence [15] , quantum entanglement [16] , statistical earthquake analysis [17] , and fractional diffusion [18] .

However, in order to write the explicit form of the R´enyi distribution for specific systems, it is necessary to set the parameter value q > 0 [1] involved in the * E-mail: a.m.savchenko@gmail.com R´enyi entropy functional. The question of establishing a connection between the parameter q and characteristics of statistical systems is open and the values of q are determined experimentally. In addition, the distribution involves the average energy whose calculation is a very cumbersome problem and often cannot be performed analytically. As it turns out, there is a general approach to solve this problem for a wide range of classical statistical models.

In this paper, we prove the energy equipartition theorem for the case of R´enyi statistics. The application of this theorem allows us to quite simply determine the average energy of a large number of statistical systems, examples of which are a harmonic oscillator, as well as classical and ultrarelativistic ideal gases. We consider a system with a model power-law Hamiltonian of the simplest form, which demonstrates the advantage of the method used to obtain the average energy over the method of direct calculation of this quantity. New useful relations for the R´enyi entropy and distribution obtained during our study are also presented. A q-formula is derived for the first time, which greatly simplifies the calculations in this theory.

An important point of this work is the study of the asymptotic transition of the R´enyi distribution to a power-law form [19] . Graphs of the limiting distribution clearly demonstrate the applicability of R´enyi statistics to a large number of phenomena, which is confirmed by numerous experimental data.

We emphasize that, in order to make these relations and the obtained results more descriptive, we will adhere to the standard notation, without resorting, as is often the case in studies on this topic, to the formalism of the so-called q-deformed functions, which, in our opinion, does not contribute to the disclosure of the physical meaning of what occurs.

We consider some statistical system in which it is possible to implement W microstates with probabili-

The R´enyi entropy functional of this system is defined as follows:

It generalizes the famous Boltzmann-Gibbs-Shannon entropy functional. In fact, it is easy to see that

It is important to note that the Tsallis entropy [20] , which is introduced independently and axiomatically in many papers, is a special case of the R´enyi entropy for |1 − q| 1. When the normalization i p i = 1 is taken into account, the condition |1 − q| 1 is equivalent to the fact that i p q i − 1 1. Therefore, expanding (1) into a Taylor series with respect to i p q i near unity up to the first order, we obtain

Thus, we arrive at the above-mentioned Tsallis entropy, which is characteristic for nonadditive statistical systems [21, 22] , as it is generally accepted to date. This topic is addressed in the Internet resource [23], which contains both theoretical and experimental studies of models described by the entropy functional of this type. The choice of the entropy form (1) was substantiated by Shore and Johnson and is explained in [24] [25] [26] .

Following Jaynes' approach [27, 28] , we obtain the R´enyi distribution corresponding to the introduced entropy functional S R (p). Fixing the normalization condition W i p i = 1 and also the average energy U = W i p i H i of the considered system, we get the functional (4) in which α and β are constant Lagrange multipliers. The necessary condition for the extremum of functional (4) has the form

Multiplying this relation by p i and summing over the index i = 1, W , by using the definition of the internal energy U and the normalization condition for the distribution, we obtain an expression for the factor α:

Substituting (6) into (5) and expressing p i , we have the relation

To find an expression for the last factor in the righthand side of (7), we use the normalization condition for the distribution. Hence,

Thus, the solution to the problem of maximizing the functional (4) is the R´enyi distribution

where (10) plays the role of a statistical sum (partition function).

We should specially mention the case of classical systems for which the spectrum of feasible energy values is continuous. Then we deal with a probability density of the form (12) where r = (r 1 , r 2 , . . . , r N ) are the particle coordinates, p = (p 1 , p 2 , . . . , p N ) are the particle momenta, N is the number of particles, and X is the volume occupied by the system in the phase space of states. An element of integration over the phase space has the form

where γ i is understood as the number of internal degrees of freedom of a particle that is not subject to the classical transition with index i and N ! is the number of permutations of identical particles. Despite the difference in the cases of continuous and discrete distributions, it is convenient to understand i = 1, W as an index, which can take on both a discrete and a continuous spectrum of values; here the summation means integration in the second case. This allows us to use the notation p = {p i } in the context of both distribution types.

We also note the fact that the R´enyi distribution transforms into the canonical Gibbs distribution in the limit q → 1

Comparing the statistical R´enyi entropy S (R) (p (R) ) with the thermodynamic Clausius entropy, we can show that the Lagrange multiplier β has the meaning of the inverse temperature of the system β = 1/θ, which will be considered in one of the subsequent sections. Thus, we have come to the most general form of the statistical distribution, but it does not have a number of properties convenient for the calculations inherent in the Gibbs exponential, namely, the property of factorizing and preserving the shape of the distribution for differentiation with respect to a parameter. Therefore, the following formula becomes useful in the process of calculations using the R´enyi distribution.

Let us pay attention to the definition of the average energy

On the other hand, due to the distribution normalization condition W i p

Thus, we come to the relation

where the notation ΔH i = H i − U is introduced for brevity of calculation records. Substituting the explicit form of the R´enyi distribution (9), we find

Assuming β = 0 and q = 1, we multiply this expression by β(1 − q)/q and add the statistical sum Z (R) written in explicit form (10) to both sides of the equality. We then obtain the so-called q-formula, which is very useful for calculations:

moreover, for q → 1, this formula remains valid and becomes a trivial equality. At first glance, the established equality may seem strange. In fact we need to remember that the parameter β depends on U , which ensures the validity of this relation. Moreover, for given U , the obtained equality can be considered as an equation in the unknown parameter β; however, this relationship between the quantities β and U arises even when the problem for the conditional extremum of the R´enyi entropy functional is solved. It is also important to note that the q-formula is valid only if the partition function and the expression for the average energy converge. In turn, these requirements are equivalent to the condition for the existence of the R´enyi distribution.

The relation plays an important role in the derivation of properties of the R´enyi distribution and entropy.

For convenience, we will use the notation S (R) below as a short notation for S (R) (p (R) ).

Proof. The definition of the statistical sum (10) and q-formula (21) imply that

Therefore,

It may be noted that a similar form of the expression for the entropy is also valid in the case of the Gibbs statistics. To demonstrate this, let us first draw attention to the fact that the R´enyi distribution contains a dependence on the energy deviations from the average value ΔH i = (H i − U ). Let us reduce the canonical distribution (14) to a similar kind of dependence, which is easy to do due to the exponential factorization property:

which corresponds to the form of expression (22) .

Using the obtained form of the R´enyi entropy (22), one can easily obtain the following differential relation.

Lemma 2. The R´enyi entropy S (R) satisfies the equality

Proof.

We note that the q-formula was used in the second equality. It is seen from (10) and (22) 

. Therefore, we can show in a completely similar manner to how it is done for S (G) [29] that the statistical R´enyi entropy S (R) under study corresponds to the thermodynamic Clausius entropy function in the laws of thermodynamics.

We consider a system whose state is specified by a set of parameters (U, x, N ), where x = (V, a), U is the average energy, N is the number of particles, V is the volume of the system, and a are the external fields acting on the system. Then, the energy levels {H i } W i=1 depends on the parameters x and N . Therefore, expression (10) for the partition sum implies that U, x, N ) . To avoid the encumbrance of the calculations, we omit the index (R) for a time. We write the total entropy differential

Let us consider each partial derivative separately. To calculate them, we will use the obtained q-formula (21) and also the expression for entropy (22):

We introduce the notation for the resultant generalized force

Then

where X is the average generalized force. Similarly, we find that

Let us consider the resulting construction in more detail. Taking the fact that the minimum step in N is equal to 1 into account, we have

This sum is the average over the distributions p i of the change in the energy levels H i when one particle is added in the isolated system, whereas the system does not receive heat and does not perform work. Therefore,

where μ is the chemical potential;

Thus, the total differential of the R´enyi entropy in the variables (U, x, N ) has the form

Noting that, by construction, Xdx is the work produced by the system and recalling the first law of thermodynamics, we come to the relation

in which δQ is the amount of heat received by the system. It is seen from (38) that β is an integrating factor for the expression of the first law of thermodynamics. However, as we know from the second law of thermodynamics, this integrating factor is equal to the inverse temperature of the system; therefore, for any admissible q,

Thus, we come to the conclusion that the statistical R´enyi entropy introduced into consideration corresponds to the Clausius thermodynamic entropy. This fact allows us to use all thermodynamic relations in the context of the R´enyi statistics. According to (39), the R´enyi distribution can be written as

D-dimensional phase space X = (x 1 , x 2 , . . . , x D ) and Hamiltonian H(X), which is in the state of thermodynamic equilibrium, the following relation holds:

for any positive integer k ≤ D, in which the averaging is performed over the R´enyi distribution.

Proof. Let us prove the validity of the stated theorem. For this, it is convenient to use the qformula (21) . With its use, taking the normalization condition W i p i = 1 and the form of the R´enyi distribution (9) into account, we obtain the expression

For the convenience of further reasoning, we introduce the notation:

Thus, we come to the relation

The considered theorem is valid for classical systems with a continuous energy spectrum; therefore, it is necessary to pass from the discrete case with H i to the function H(x 1 , x 2 , . . . , x D ), which continuously depends on its arguments. In this case, the summation over the states of the system turns into integration over the volume of the phase space X. The integration element is denoted as

where γ i is used to take the quantum degrees of freedom of a particle with index i into account, d is the dimension of the physical space, and (D/2d)! determines the number of permutations of identical particles whose inclusion is essential in the transition from the quantum case to the classical case, in which each particle is numbered. The factor (2π ) D/2 is the volume of a cell in the phase space X, where the same number of position and momentum variables is involved. Then expression (43) takes the form 1

Assuming that H depends explicitly on x k (k = 1, D), we apply the integration formula by parts with respect to the designated variable. Then,

where X k = (x 1 , x 2 , . . . , x k−1 , x k+1 , . . . , x D ), dΓ k = dΓ/dx k , and a and b are the boundaries of the region of integration with respect to the variable x k . We also make a remark about the derivative of the energy deviation from the mean value:

but U is independent of the variable x k ; hence,

As a result, we obtain

Let us consider the first integral in (49). The factor (1 − βλΔH) involved in the integrand determines one-to-one zero values of the probabilities in the R´enyi distribution. That is,

where p (R) (X 0 ) is the value of the probability density on the set X 0 ⊂ X.

In accordance with the physical meaning of the probability density of states, this function must be zero on the boundary of the region of possible values of the phase space variables; i.e., it has limited support. Therefore, in the case of finite values of a and b, the integral is equal to zero. In the case where the limits of integration contain +∞ or −∞, we deal with uncertainty of the form 0 · ∞; however, now we can use the fact that the system is physically bounded and all its parameters are finite quantities. Therefore, already for some finite but sufficiently large absolute values of a and b, a region of the phase space is reached such that the probability of finding the system in this region is equal to zero. Based on the above arguments, we can conclude that

and then we have the relation

Recalling that 1/λ − 1 = 1/(q − 1) and β = 1/θ, we get

where the averaging is performed over the R´enyi distribution. This completes the proof.

The proved theorem makes it possible to easily obtain the average internal energy of a large number of statistical systems. Moreover, the same form of virial relations in the Gibbs and R´enyi statistics leads to familiar expressions for the internal energy. As an example, U osc = θ for a one-dimensional harmonic oscillator, U 0 = (3/2)Nθ for a free monatomic gas, and U R 0 = 3Nθ for an ultrarelativistic monatomic gas.

In order to demonstrate the advantages of using the energy equipartition theorem, we consider the important case of a model power-law Hamiltonian of one variable

where C and k are some positive constants. This problem was considered in [30] , where a method of direct calculation of the average energy value is outlined. Let us demonstrate how much this process can be simplified due to the equipartition theorem. It follows from the proved theorem that

However, it should be noted that such a method for calculating the average energy does not allow one to obtain the corresponding convergence conditions for U , which naturally follow from the direct calculation procedure. Therefore, in order to obtain the mentioned restrictions, we present the appropriate calculations. By definition,

We note that the normalization Z has no index R, since in this case Z is the normalization integral but not the partition function that appeared in the previous sections.

To perform the corresponding calculations, we use the value of the table integral [31] of the form

with the convergence conditions:

First, we consider an expression for the normalization Z, which has a structure similar to the integral I. From the form (57), we can conclude that Z = I for

Thus,

under the convergence conditions:

We proceed similarly with the integral

which takes the form (57) for

Then, using expression (57), we obtain

with the convergence conditions:

As a result of taking the integrals, expression (55) takes the form

Recalling a useful property of the Γ-function: Γ(z + 1) = zΓ(z), we use it to simplify the resulting expression

Then, from (64), we obtain the average energy

Remembering that 1/β = θ, we obtain the result predicted by the energy equipartition theorem

Thus, by direct computation, we have verified the validity of the energy equipartition theorem and also obtained the convergence condition q > q min = 1/(1 + k). Thus, the R´enyi distribution for the system with model Hamiltonian (53) looks as follows:

where the notation C u = C/U is introduced.

From the practical point of view, the limiting case of this distribution for q → q min is of particular interest. Therefore, we consider it in more detail. Since the mean energy integral diverges for q = q min , we introduce a positive parameter ε → 0 such that q = q min + ε.

The normalization function of this distribution has the form

We note that the resulting normalization coincides with the approximate value of the normalization function (59) near q = q min . However, it should be remembered that if we approximate distributions the function Z may not coincide with the approximation of the original normalization function, since generally the procedure of approximation of the distribution violates the original normalization.

Let us analyze the resulting approximation. Obviously, for ε = 0, distribution (68) takes on a powerlaw form

However, it is essential to recall that for q = q min (ε = 0), the average energy diverges, and hence, the R´enyi distribution only asymptotically approaches the power form as ε → 0.

Now we consider the case x → +∞. It is easy to see from (68) that in this limit we obtain the power law dependence

Passing to the logarithmic scale, it is easy to observe that the slope of the graph of the dependence of ln p (R) on ln x in the considered limit x → +∞ is determined by the expression −(k + 1) 1 + ε k+1 k . In the region of small values of the variable x (as x → 0), distribution (68) reaches the constant

Let us demonstrate the general nature of distribution (68) depending on the parameters ε and k. For definiteness, we choose C = 1 and β = 0.01. The corresponding graphs are shown in Figs. 1 and 2.

One can see that a characteristic feature of the R´enyi distribution, which distinguishes it from the power-law distribution, is the presence of a so-called plateau observed in the region x 1. In this case, it depends on the parameter of the Hamiltonian k how smooth or abrupt the transition from a constant to a power-law dependence will be.

The experimental study of systems, which are usually attributed to the power distribution law, often leads to statistics corresponding to the particular case of the R´enyi distribution under consideration. Many examples for dependences of this kind can be found in [5, 32] , where the statistics of a large number of systems of different nature are collected and analyzed: from earthquakes and forest fires to the population of cities and the citation rate of papers. The graphs from [32] most similar in form with dependence (68) are shown in Fig. 3 . As an example of the appearance of the obtained distribution (68), we also note the results of study of the statistics of the descent of rice avalanches in the experiment examined in [33] [34] [35] . Furthermore, this experience is considered in the book [7] . Another amazing example is the statistics of vortex avalanches (magnetic flux avalanches) in high-temperature superconductors [36] . All this suggests the idea of the applicability of the R´enyi distribution to a wide range of systems and phenomena.

In the paper, the energy equipartition theorem over the degrees of freedom of classical statistical systems has been proved for the first time in the case of the R´enyi distribution obtained by maximizing the R´enyi entropy functional. Thus, we succeeded in expanding the range of applicability of this theorem to the class of statistical models described by the R´enyi distribution, which generalizes the canonical Gibbs distribution. The proved theorem is a powerful tool that makes it possible to very quickly obtain the value of the average energy for a large number of statistical models, which enables us to avoid the time-consuming problem of direct calculation of this characteristics of systems.

During the study of the properties of the R´enyi distribution, some useful relations were obtained, among which it is necessary to distinguish the so-called qformula, which played a central role in the proof of the above theorem. The equality between the logarithm of the partition function of the R´enyi distribution and the R´enyi entropy was deduced, which made it possible to establish the relationship of the derivatives of the entropy and the average energy with respect to temperature.

A connection is established between the statistical R´enyi entropy and the thermodynamic Clausius entropy function. As a result, the physical meaning of the Lagrange multiplier β involved in the R´enyi distribution is revealed.

Using an example of a system with a model Hamiltonian H = Cx k , a direct method for calculating the average energy is demonstrated, which allows us to obtain a constraint on the distribution parameter q and also led to a result that confirms the equipartition theorem.

Given the model, we have studied the behavior of the distribution for q close to q min . A characteristic and important feature of this case is the presence of a horizontal section, that is, a plateau in the area of small values of x. Figure 1 demonstrates that many empirical data are well approximated by the power-law distribution only for sufficiently large values of the random variable. In the area of small values, in turn, we observe a clear deviation from this distribution. This phenomenon of deformation of the power-law distribution indicates the applicability of R´enyi statistics to a wide range of phenomena related to various fields of science.

It should be noted that the q-statistics is a promising research direction related to the study of complex and self-organizing systems. This is confirmed by the progress in the generalized statistics applied to the analysis of the most pressing challenges, for example, the COVID-19 epidemics to which [37] is devoted.

Proceedings of the 4th Berkeley Symposium on Mathematical Statistics and Probability

Probability Theory

Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology

How Nature Works: The Science of Self-Organized Criticality

Introduction to Nonextensive Statistical Mechanics: Approaching a Complex World

Nonextensive Statistical Mechanics and its Applications

Statistical Physics. Theory of Equilibrium Systems

Tables of Integrals, Sums, Series and Productions

The authors are grateful to Associate Professors G.V. Koval and V.A. Gribov for valuable comments on this paper.