|
NEW
PRINCIPLES OF COMPUTER PROCESSING
Introduction
For the processing of the experimental data of water and
liquids it was developed a set of new principles of parameters
calculations and a set of new GDV programs. The most important
parameter was entropy that was derived from presenting GDV-gram
as a curve (or vector) by calculating some data along the
radii coming along 360°. Herewith we present some ideas
and principles of operation of the new programs.
Basic
notions of probability theory. Entropy
The basic notion of probability theory is the notion of experiment
which may be repeated many times under the same conditions
and may produce different results, or outcomes. Let us assume
for simplicity that the number of different outcomes is finite
and denote them by A1,...,AN.. Moreover, let us assume that,
if we repeat our experiment (we call it a) many times, the
frequency of outcome Aj turns out to be proportional to positive
number pj, and, moreover, p1 +.. .+pN = 1. Then we say that
outcome Aj has probability p(Aj ) = pj. We remind the simplest
possible experiment: tossing the coin. It is normally assumed
that there are only two possible events: the "head"
and "tail"; under standard conditions probabilities
of these events equal 1/2.
Notion
of entropy naturally arises if we ask the following question:
Is it possible to give a natural measure of uncertainty e(a)
of outcome of experiment a?. For example, in coin experiment
the coin may be forged in such a way that the "head"
event has probability 0,99 and "tail" event - only
0,01. It is natural to suppose that the measure of uncertainty
of this second experiment should be essentially smaller then
the measure of uncertainty of experiment with non-forged coin.
If probability of "head" equals 1, there is now
any uncertainty in the experiment outcome at all, and the
measure of uncertainty of such experiment should be minimal
(it is natural to put it equal zero). In general, it is intuitively
clear that the experiment with all probabilities pj equal
1/n contains bigger uncertainty then experiment with any other
values of pj. What are the other requirements which we would
like to impose on uncertainty measure?
Consider
two independent experiments  and
ß. Denote possible outcomes of experiment ß by
B1, ..., BM and corresponding probabilities by q1 = p(B1),...,
qM = p(BM). Consider experiment ß;
outcomes of this experiment are formal products Aj Bk with
assigned probabilities pj qk. It is clear that uncertainty
of experiment ß
should be greater then degree of uncertainty of, say, experiment
a, since experiment ß introduces additional uncertainty.
It is natural to require that uncertainty of ß
should equal to the sum of uncertainties of independent experiments
and ß:
To
be able to get universal numerical value of  (a)
we should require that it does not depend on the nature of
outcomes Aj i.e. it should be function only of probabilities
pj: (a) = (p1,..,pn).
Moreover, consistency requires the invariance of (p1,..,pn)
with respect to an arbitrary transposition of numbers pj.
Finally, we would like (p1,..,pn)
to be always non-negative and continuous function of probabilities
pj (since small variation of pj should intuitively lead to
only small variation of uncertainty of experiment).
It
turns out that the measure of uncertainty of experiment satisfying
all these requirements does really exist; supplementing them
by some more technical condition (see [Yaglom, 1960]) it is
possible to claim that solution is also unique up to an arbitrary
positive multiplier and has the form:
where
C > 0 is an arbitrary constant. Value e(a) is called the
entropy of experiment a. It is worth making one technical
comment: the events with very low probabilities give very
little contribution to .
This is not completely obvious since ln p —> - 
as p —> 0. However, it is easy to prove that already
the product (pln p) tends to 0 as p tends to 0.
The
notion of entropy in probability theory as universal measure
of uncertainty of experiment was first introduced by Shannon
[1948] in 1949 within the framework of signal transmission
theory. Since that this notion was proved to play very fundamental
role in different application areas of probability theory
like coding theory, linguistic, image processing, statistics
and so on. We do not mention here the applications of the
notion of entropy in physics and biology, which will be considered
in the sequel.
Let
us summarise once more the main properties of the entropy
(2.2):
- The
entropy is
continuous symmetric non-negative function of probabilities
p1,... ,pN invariant with respect to any transposition of
numbers p1,... ,p
 .
- The
entropy takes its minimal value
= 0 if and only if one of probabilities equals 1 and all
others equal 0, i.e. the experiment
does not contain any uncertainty.
- For
fixed N the entropy takes its maximal value for experiment
with equal probabilities pj = 1/N; then
- Obviously,
the entropy of equal probability experiment infinitely and
monotonically increases with increasing of N.
- Entropy
of product of two (or more) independent experiments satisfies
addition law (2.1).
- Function
e satisfies the following functional equation [Yaglom, 1960]:
which
is closely related to addition law (2.1).
Notion
of information »
«
Back to Home
gdvtechnique.com
@ 2001. All rights Reserved
|
