Suppose we receive a series of 8-bit words representing some symbols or event outcomes, and we notice that, on average, about 70% of the received bits are “1”, and only 30% are “0”. What, if anything, does that tell us from an information theoretic point of view? Question 2Select one: Nothing. We would need to know the probability of received words before we could calculate the entropy. Since 1 and 0 are not equi-probable, we know that we could send the sequence of symbols with fewer bits using a suitable code word assignment method. Plugging the probabilities into the information theory equations would give us the optimum encoding method we should use. (JUST choose the right option)

Suppose that the output of a process that generates independent symbols {A, B, C, D, E} with known probabilities {in the same order: 0.2, 0.33, 0.1, 0.07, 0.3 } needs to be communicated. What is the entropy (bits/symbol)? (Your answer should be rounded off to the nearest first decimal point. Type a number only - no units.)

Entropy in communication refers to the amount of uncertainty. True False

Consider a process that generates a set of symbols with a known probability distribution. We calculate the entropy assuming that the symbols are IID (Independent and Identically Distributed), but then discover that there are certain relationships between the symbols that mean they are NOT truly independent. Compared with our first calculation, would you expect that the true entropy is actually Question 6Select one:

A binary source transmits either a symbol "1" or "0" at discrete time intervals. The source could transmit independently either symbol with equal probability. What is the probability of transmitting exactly two "1"s in the first five transmitted symbols?

The average number of bits/symbol is obtained by calculating the sum of all piLi, where pi is the probability of symbol i and Li is the length (in bits) of the code word used to represent symbol i. If we use a fixed length code word (e.g. each symbol is represented by a 4-bit word), then the average number of bits/symbol is independent of the probability distribution.Question 1Select one:TrueFalse