熵 · 易 · Shannon entropy (1948)

The entropy of the oracle

How much information each method gives, in bits

Compare the entropy of each method bar by bar and separate the yin and yang value from the movement.

Shannon entropy, H = minus the sum of p·log₂p, measures information in bits. A uniform hexagram is exactly 6 bits, the maximum for 64 states. But not every method gives the same: a line of coins has more entropy than a yarrow one, and the whole difference is in the movement (old or young), because the pure yin/yang value is exactly 1 bit in both. On the first page of that 1948 article the word bit appears for the first time.

Entropy of one line: value + movement

light = yin/yang value (1 bit in both) · solid = movement (old/young)

6 bits

uniform hexagram (maximum)

1.8113

line of coins

1.7490

line of yarrow

0.0623

difference (all of it in the movement)

In the long run: the stationary distribution

Chaining consultations (the Markov chain), the equilibrium distribution of the coins is uniform and keeps the 6 bits; that of the yarrow skews to yin and drops to 4.8677 bits = 6·H(1/4). The ancient method not only changes where the series tends, but how much information it retains.

The optimal code: Huffman

How many bits does it cost to transmit one line with the best possible code? Huffman's, built by the site (not hard-wired), gives an expected length of 1.875 bits with coins (30/16) and 1.8125 bits with yarrow (29/16). The finding: the ancient method not only has less entropy, it also compresses better. And both satisfy Shannon's source coding theorem: H ≤ L < H+1, the optimal length stays within one bit of the entropy.

1.8750

Huffman coins (30/16)

1.8125

Huffman yarrow (29/16)

H ≤ L < H+1

source coding (Shannon)

+0.0625

coins costs more per line