key: cord-0129244-ubbk99jt
authors: Teza, Gianluca; Stella, Attilio L.
title: Exact coarse-graining preserves entropy production out of equilibrium
date: 2020-03-19
journal: nan
DOI: nan
sha: bc4bafb533398fb34c68fa838f4b987cb4c462c5
doc_id: 129244
cord_uid: ubbk99jt

The entropy production rate associated to broken time-reversal symmetry provides an essential characterization of nano-systems out of equilibrium, from driven colloidal particles to molecular motors. Limited access to the dynamical states is generally expected to hinder the correct estimation of this observable. Here we show how memoryless jump processes can be coarse-grained exactly preserving its average and fluctuations at stationarity. This supports univocal applicability of fluctuation theorems for entropy and allows inference of the genuine thermodynamics together with inaccessible process details.

Entropy production out of equilibrium, measured, e.g., from the heat dissipated by a mesoscopic system into a thermostat, is a key to interpret experiments involving nano-manipulation or molecular motors [1] [2] [3] [4] [5] [6] . When dealing with active matter, this production may be the only possible indicator of out of equilibrium conditions [7, 8] . Fluctuations of the rate of produced entropy are expected to obey symmetry properties which allow to estimate free energy differences or binding energies at molecular level [1, [9] [10] [11] . Theorems validating such properties have been proved for specific models [12] [13] [14] [15] , but do not hold if only part of the states of the mesoscopic system is experimentally accessible. Indeed, applicability of these theorems requires that all slow transitions between mesostates are detectable, and that transitions inside each mesostate are very fast [16, 17] . Under these conditions entropy production can be recovered from a description without memory in the framework of stochastic thermodynamics [18] .

On the other hand, in experiments where not all mesoscopic details are accessible, like with molecular motors, non-exponential dwelling or residence time distributions have been often measured. [19] [20] [21] . These memory effects were identified as revealing features of the underlying chemo-mechanical transitions, but not put in direct relation to entropy production.

If only partial, coarse-grained information is experimentally available, the average entropy production one can record is generally expected to be lower [17] . Estimates of partial entropy production pertaining to the accessible parts of the systems, or lower bounds for the average full productions, have been actively studied recently [7, 8, [22] [23] [24] [25] . However, these results are of limited help for a complete thermodynamic inference [26] , since precise insight on the possible effects of coarse-graining on detectable entropy production is still missing [17] . * attilio.stella@pd.infn.it So far, the inference of entropy production from the coarse-grained dynamics of Markov processes governed by Master equations could be attempted only in cases in which memory effects amounted to small perturbations [27, 28] . Thus, establishing the effects of coarsegraining on the detectable entropy production remains an open fundamental issue, also in view of the relevance of these processes for the description of experiments.

In this article we present an exact coarse-graining by decimation of Markov jump processes which keeps precise record of how the entropy production evolves in time. We also show that its average and higher fluctuation moments remain invariant at stationarity. Non-Markovian residence time distributions typically recorded in experiments can also be analyzed in terms of a decimation of Markovian trajectories in state space. This enables thermodynamic inference of the full entropy fluctuations [17, 26] by reconstruction of the underlying dynamics.

We start by considering a Markov process on a linear periodic network with states i = 1, 2, . . . , N and rates of jump W ji from state i to state j. We assume nonzero rates only for nearest neighbor jumps, and put W ji = r or W ji = l, for all right or left jumps, respectively (Fig. 1a ). If states, e.g., refer to positions of a particle on a lattice with spacing L, r and l can be linked to L, to a uniform driving force f , and to the thermal bath temperature T , by the local detailed balance condition r/l = e f L/k B T [29] . Along a trajectory of the process in which jumps take place at times t k (k = 1, 2, . . . n; 0 < t 1 < t 2 < . . . t n < t) from states i k−1 to states i k (i 0 state at t = 0) the entropy produced is (k B = 1) [15, 30] :

The probability P i (t) for the system to be in state i at time t, is marginal of the probability P i (S, t) that the system is in state i at time t with an entropy S accumulated along all possible trajectories. Thus we can write:

where the shifts in the S arguments on the r.h.s. account for the entropy gains associated to jumps according to Eq. 1. Summing Eq. 2 over all S values yields the Master equation satisfied by P i (t) with a unique stationary solution reached for t → ∞ from arbitrary initial conditions [31] . For t → ∞, P i (S, t) is consistent with a large deviation principle [32] for the entropy production rate σ = S/t. Indeed, indicating by Q(S, t) = Σ i P i (S, t) the probability of having produced a total entropy S at time t, the scaled cumulant generating function (SCGF) ε can be extracted from the function G(λ, t) = S e λS Q(S, t) ∼ t→∞ e ε(λ,r,l)t [32] . By discrete Laplace-transforming Eq. 2 with respect to the entropy S we obtain a first order differential equation for G ∂ t + r + l + re λ log r/l + le λ log l/r G(λ, t) = 0 (3) whose solution, in the long time limit, provides us with the SCGF [33] ε = re λ log r/l + le −λ log(r/l) − (r + l) .

The probability Q(S, t) for t → ∞ concentrates on the value S = σ 0 t where σ 0 = ∂ε/∂λ| λ=0 = (r − l) log(r/l), while higher order derivatives ∂ n ε/∂λ n | λ=0 give the scaled cumulants of σ describing its fluctuations for long times. Since ε satisfies the fluctuation theorem [15] , i.e. ε(λ − 1, r, l) = ε(−λ, r, l), the probability of σ further satisfies Pr(S/t = σ)/Pr(S/t = −σ) = e σt for t → ∞. Assuming N > 3 even, coarse-graining can be performed by eliminating from the system in Eq. 2 all odd (or even) states (Fig. 1a) . Indeed, after Fourier transforming in time Eq. 2 one obtains

, and the oddP i can be algebraically eliminated. Thus, upon reversetransforming, the even P i 's satisfy

(6) Eqs. 6 are not consistent with a Master equation due to the ∂ 2 t term. The quantity S i even P i (S, t) is not conserved, but stabilizes at a value 1/2 (equivalence of even and odd states) after a transient time 1/2(r + l). As we show below, in spite of the lack of strict normalization of the P i 's with even i, the coarse grained description provided by Eqs. 6 accounts correctly for the entropy production of the original system. Indeed, we can define Q (S, t) = i even P i (S, t) and write a differential equation of second order in time for G (λ, t) = S e λS Q (S, t), which becomes the function controlling entropy production in the coarse-grained system:

From the dominant long t behavior G ∼ e ε t of the solution, we argue that the rate S/t still obeys a large deviation principle [32] and find eventually the SCGF for the coarse-grained entropy production ε (λ, r, l) = ε (λ, r, l) [33] . Thus, the SCGF function remains the same as that of the original process satisfying the fluctuation theorem [15] . This is a first instance of our main, unexpected [17, 27, 28] result: decimation leaves exactly invariant the stationary spectrum of entropy production fluctuations. The memory effects implied by Eqs. 6 can be best analyzed by computing the probability densities of jump times between surviving states. These are the times spent by trajectories starting from an even state i before reaching either state i + 2, or i − 2. Distributions of such and related times are often recorded in experiments [19] [20] [21] . To perform this calculation we introduce a normalized jump rate w ji = W ji / k W ki . So, for the Markov process, the probability density of the time t of jump from Analytic results (red) are in agreement with data (dots) data from extensive simulations [33] .

state i to state j after arriving in i at t = 0 has the exponential form w ji e −t/τ /τ = 1 2π dω e −iωtπ ji (ω), with τ = (r+l) −1 . The Fourier transformπ ji allows to express as a series of convolutions the characteristic function of the probability density p r (t) of jump times from an even state i to the state i + 2 without visiting i − 2. To this purpose we consider the 3 × 3 matrix T k k (ω) =π k k (ω), restricted to the states {i − 1, i, i + 1}. In this way the characteristic function of p r (t) can be written as p r (ω) = dt e iωt p r (t) =π i+2,i+1 (ω)(I + T(ω)) −1 i+1,i . This expression sums the contributions of all trajectories performing that jump [33] . Analogously one obtainsp l (ω). Even if the P i 's in Eqs. 6 are constructed by recording the presence of the system exclusively in the even states, one can count the whole times of jump as residence times in state i. With this assumption the sum p r (t) + p l (t) = p(t) becomes equivalent to a non Markovian (non-exponential) distribution of residence time in a generic even state. This distribution is reported in Fig. 2a . The process resulting from this interpretation of the jump times qualifies as semi-Markov with timedirection independence, since p r (t) ∝ p l (t) [34] [35] [36] . The entropy production rate of such models has been determined recently [35, 36] and is remarkably consistent with our results for the linear chain [33] . As we show below, in more general cases such consistency does not hold. However, the densities p r,l (t), regarded as empirical data, can be exploited to identify the underlying Markov dynamics. Upon matching these densities with analytic results like those just derived, one can determine the rates of the undecimated model and infer the full entropy production with its fluctuations.

The linear network of Eq. 2 presents only one loop along which entropy is produced. A fundamental prob-lem left is to study situations in which coarse-graining erases loops producing entropy in the network. Indeed, detectable entropy production is generally expected to lack erased loops contributions and therefore to be lower in such cases [17, 28] .

Let us consider the network in Fig. 1b . It consists of a main loop of N states (blue), X 1 , X 2 , . . . , X N , with right\left nearest neighbor jump rates r\l. A secondary 3-state loop is further attached to each X i state along the main loop, by connecting it to a Y i (red) and a Z i (green) state. Rates c and a apply, respectively, to clockwise and anticlockwise jumps to nearest neighbors within secondary loops. The states Y i and Z i of each loop are those we want to decimate. Before decimation, upon summing over the index i specifying different X, Y and Z states equations analogous to Eq. 2 [33] , one gets: t) is the probability that a trajectory ends at time t in a generic X state with cumulated entropy S. The SCGF ε(λ, r, l, c, a) for entropy production of this process can be found by considering, for α = X, Y, Z, G α (λ, r, l, a, c) = S e λS P α (S, t) in Eqs. 8 and by diagonalizing the 3×3 matrix expressing the time derivative of the G α vector components. The dominant eigenvalue determines ε [33] which yields an average entropy production rate:

Here the first term is the contribution from jumps on the main loop, while the latter is relative to transitions occurring within the secondary loops. Decimation in this case is realized by simply eliminating P Y (S, t) and P Z (S, t) from the system of Eqs. 8 after Fourier transforming in time. Reverse-transforming yields a third-order differential equation in time for P X (S, t). Also in this case S P X (S, t) is not strictly normalized, but for large t it stabilizes to 1/3. Indeed, X,Y and Z states have equal total probability 1/3 at stationarity [33] . P X (S, t) plays here a role analogous to that of Q (S, t) in the linear chain decimation. For the function G (λ, t) = Σ S e λS P X (S, t) of the coarse-grained network we eventually obtain:

where α, β and γ are functions of the jump rates and of λ [33] . In the limit t → ∞ we get G ∼ e tε , where ε is the dominant root of the characteristic equation associated with Eq. 10. Remarkably, ε coincides with the SCGF ε of the original process [33] . So, also in this case , an observer is unable to distinguish between purely mechanical (r, l) and chemomechanical (RuR d , LuL d ) transitions between X states (blue).

Matching pr(t) and p l (t) (b) allows to recover all rates, full entropy production and ATP consumption. ε = ε, maintaining validity of the fluctuation theorem.

In this example, the elimination of secondary loops leads to terms in Eq. 10 that represent additional entropy gains originating from self-jumps on X states in the coarsegrained dynamics (Fig. 1b) . Such contributions take into account what the removed secondary loops were producing in the underlying Markov description [33] . Thanks to them, contrary to previous expectations [17, 28] , our decimation can keep track of the full entropy production. Decimation of the process trajectories, carried on along lines similar to those illustrated for the linear chain, allows to obtain non-exponential p r (t) and p l (t) for the jumps between neighboring X states [33] . Also in this case the coarse-grained dynamics can be regarded as semi-Markov with time-direction independence (p r (t) ∝ p l (t)). In panel b of Fig. 2 we compare the residence time distribution p(t) = p r (t) + p l (t) with a histogram based on simulations.

Evaluating the rate of entropy production of this semi-Markov model according to Refs. [35, 36] would account only for the partial contribution coming from direct transitions between surviving X states, with average log(r/l)(r − l)/3 < σ 0 . Also procedures setting lower bounds to the average entropy production rate [7] could not improve the estimate of this average, due to the timedirection independence of the process. However, even though coarse-graining gives access only to the densities p r (t) and p l (t), the calculations leading to these functions still allow to determine the full entropy production. This can be done by matching the empirical probability densities of jump times with those of the candidate model for the fine-grained process. Its Markovian dynamics can be precisely determined together with the full spectrum of entropy production. The marked difference between the residence time distributions reported in panel a and b of Fig. 2 , exemplifies a useful hint helping in an attempt to guess the underlying hidden Markov network.

Our coarse-graining can be applied, e.g., to molecular motors. A simplified version of the model in Ref. [5] is reported in Fig. 3 , where the red dots in the periodic network indicate intermediate hidden Y states allowing ATP hydrolysis activated jumps between the blue X states. The positions of the X states are recorded in experiments. The transition rates reported in Fig. 3 satisfy local detailed balance conditions linking them to the load force f , the difference in chemical potentials ∆µ = µ(AT P ) − µ(ADP ) − µ(P ), the temperature T and the spacing L between X positions [33] . Elimination from the equations of the probabilities referring to Y sates, leads to the following system for the probabilities of the X states

The various terms on the r.h.s. account for the entropy contributions due to both direct and ATP activated transitions. Exact calculation show that this coarse-grained description accounts for the total entropy production also in this case [33] . The process resulting from analysis of the probability densities of jump times is semi-Markov with time-direction dependence, since p r (t) and p l (t), now reported in Fig. 3b , are not simply proportional. The methods of Refs. [35, 36] could not even be applied to compute an entropy production of such semi-Markov process. On the other hand, if the jump time probability densities are extracted from some empirical time series, one can recover the rates of the original Markov model by matching them with the results from a decimation of trajectories letting only X states survive. So, provided the assumption of underlying Markov dynamics is correct, a successful matching allows to recover the full entropy produced in the experiment.

The coarse-grainings presented above can be performed on networks with arbitrary topology and with states surviving decimation which are not simply equivalent up to translations. One can also keep simultaneous record of other currents besides the entropy production rate [37] .

Summarizing, we showed that, in the context of Markov jump processes, an exact coarse-graining, taking into account memory effects, and guaranteeing invariance of average and fluctuations of the full entropy production at stationarity, is possible. While an extension of the type of exact results presented here to other mod-els is an open program, our findings suggest that future investigations in this field, both theoretical and experimental, should focus on the memory effects associated with coarse-graining, which revealed essential to guarantee entropy production invariance. These effects, when analyzed for coarse-grained trajectories, were also crucial for realizing the fine-graining transformations at the basis of our inference strategy. Indeed, after detecting non-Markovian probability densities for the times of jump in an experimentally studied process, one can try to match them with those of a guessed underlying Markov process. If successful, this enables to determine the correct full entropy production, while uncovering hidden states and mechanisms. An assumption at the basis of this strategy is that Markov jump dynamics is adequate to describe what underlies the coarse-grained level. Such dynamics should in principle result from coarse-graining of a more microscopic description, and the choice of rates may reveal crucial for the very possibility of describing certain phenomena [38] .

Arrow of time in active fluctuations

Proceedings of the National Academy of Sciences of the United States of America

Proceedings of the National Academy of Sciences of the United States of America

Stochastic processes in physics and chemistry

Supplemental material

Random walks and random environments: random walks

We acknowledge Carlo Vanderzande, Marco Baiesi and Stefano Iubini for collaboration on related subjects. We thank David Mukamel and Oren Raz for discussions. G.T. is supported by a research grant from the Center of Scientific Excellence at the Weizmann Institute of Science.