Basic Ideas of Information Thermodynamics


Carnot cycle, I., II., III. Principle of Thermodynamics, Heat entropy, Observation, Information entropy, Transfer channel, Transinformation, Noise.

How to Cite

Bohdan Hejna. (2019). Basic Ideas of Information Thermodynamics. Journal of Basic & Applied Sciences, 15, 6–26.


We apply a certain unifying physical description of the results of Information Theory. Assuming that heat entropy is a thermodynamic realization of information entropy, we construct a cyclical, thermodynamic, average-value model of an information transfer chain as a general heat engine, in particular a Carnot engine, reversible or irreversible. A working medium of the cycle (a thermodynamic system transforming input heat energy) can be considered as a thermodynamic, average-value model or, as such, as a realization of an information transfer channel. We show that for a model realized in this way the extended II. Principle of Thermodynamics is valid and we formulate its information form.
In addition we solve the problem of a proof of II. Principle of Thermodynamics. We state the relation between the term of information entropy, introduced by C. Shannon (1948), and thermodynamic entropy, introduced by R. Clausius (1850) and, further, explain the Gibbs paradox. Our way to deal with the given topic is a connection of both the mathematical definitions of information entropies and their mutual relations within a system of stochastic quantities, especially with thermodynamic entropies defined on an isolated system in which a realization of our (repeatable) observation is performed [it is a (cyclic) transformation of heat energy of an observed, measured system].
We use the information description to analyze the Gibbs paradox, reasoning it as a property of such observation, measuring an (equilibrium) thermodynamic system. We state a logical proof of the II. P.T. as a derivation of relations among the entropies of a system of stochastic variables, realized physically, and, the Equivalence Principle of the I., II. and III. Principle of Thermodynamics is formulated.


Bennett ChH. The thermodynamics of computation – a Review. Int J Theor Phys 1982; 21(12): 905-940.

Brillouin L. Science and Information Theory; Academia Press: New York 1963.

Cover TM, Thomas JB. Elements of Information Theory; Wiley: New York 1991.

Gershenfeld N. Signal entropy and the thermodynamics of computation. IBM Systems Journal 1996; 35(3/4): 384.

Hála E. Úvod do chemické termodynamiky; Academia: Praha 1975.

Hašek O, Noži ka J. Technická mechanika pro elektrotechnické obory II.; SNTL: Praha 1968.

Hejna B, Vajda I. Information transmission in stationary stochastic systems. AIP Conf Proc 1999; 465(1): 405-418.

Hejna B. Informa n kapacita stacionárnch fyzikálnch systému. Doktorská diserta n­ práce, ÚTIA AV R, FJFI VUT, Praha 2000.

Hejna B. Tepelný cyklus a p enos informace. Matematika na vysokých školách: Determinismus a chaos; J MF, VUT: Praha 2005; pp. 83-87.

Hejna B. Thermodynamic model of Noise Information Transfer. In AIP Conference Proceedings, Computing Anticipatory Systems: CASYS’07 – Eighth International Conference; Dubois, D., Ed.; American Institute of Physics: Melville, New York 2008; pp. 67-75.

Hejna B. Informa n­ význam Gibbsova paradoxu. Matematika na vysokých školách: Varia n­ principy; J MF, VUT: Praha 2007; pp. 25-31.

Hejna B. Proposed Correction to Capacity Formula for a Wide-Band Photonic Transfer Channel. In Proceedings of International Conference Automatics and Informatics’08, VII-1 – VIII-4; Atanasoff, J., Ed.; Society of Automatics and Informatics: Sofia, Bulgaria 2008.

Hejna B. Gibbs Paradox as Property of Observation, Proof of II. Principle of Thermodynamics. In Computing Anticipatory Systems: CASYS’09 - Nineth International Conference; Dubois, D. M., Ed.; 2009.

Hejna B. Informa n Termodynamika I., Rovnovážná termodynamika p enosu informace 2010. ISBN 978-80-7080-747-7, VŠCHT Praha, Praha.

Horák Z, Krupka F. Technická fyzika; SNTL/ALFA: Praha 1976.

Jaynes ET. Evolution of Carnot’s Principle. Reprinted in Ericksen & Smith 1988; 1: 267-282.

Jaynes ET, Smith CR, Ericksen GJ, Neudorfer PO. The Gibbs Paradox. Kluwer Academic Publishers 1992; 1-22.

Kal ík J, Sýkora K. Technická termomechanika; Academia: Praha 1973.

Koš ál K. Statistická fyzika: Vybrané partie; VUT: Praha 1973.

Kotek Z, Vysoký P, Zdráhal Z. Kybernetika; SNTL: Praha 1990.

Kvasnica J. Termodynamika; SNTL: Praha 1965.

Landauer M. Irreversibility and Heat Generation in the Computing Process. IBM J Res Dev 2000; 44(1/2).

Maršák Z. Termodynamika a statistická fyzika; VUT: Praha 1995.

Moore WJ. Fyzikáln­ chemie; SNTL: Praha 1981.

Prchal J. Signály a soustavy; SNTL/ALFA: Praha 1987.

Shannon CE. A mathematical theory of communication. The Bell Systems Technical Journal 1948; 27: 379-423, 623-656.

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2019 Journal of Basic & Applied Sciences