Unlock document.
This document is partially blurred.
Unlock all pages and 1 million more documents.
Get Access
Reflective)statement.)Week)3)
!
Entropy!is!the!measure!of!randomness!in!the!event.!However,!in!information!
theory,!entropy!is!the!average!amount!of!information!contained!in!each!message!
received.!It!is!very!interesting!to!know!that!amount!of!information!that!is!
received!will!depend!on!the!probability!of!the!appearance!of!the!required!
symbol.!!
Based!on!this,!we!can!find!a!relation!between!the!amount!of!information!and!the!
randomness!in!the!symbols.!If!the!randomness!in!the!symbols!will!increase!then!
the!amount!of!information!that!one!will!have!to!ask!to!predict!the!required!
symbol!will!also!be!more.!Thus!an!increase!in!randomness!or!entropy!will!result!
in!an!increase!in!the!amount!of!information!received.!!Therefore,!we!can!say!that!
randomness!and!the!amount!of!information!received!are!directly!proportional!to!
each!other.!According!to!Claude!Shannon,!an!event!will!have!the!greatest!entropy!
or!will!gather!the!maximum!information!when!all!possible!outcomes!are!equally!
likely.!!
For!example,!lets!take!the!weather!of!a!place!over!a!period!of!31!days.!If!the!
weather!is!random!and!unpredictable!then!we!will!have!to!store!the!entire!
information!or!we!will!have!to!ask!more!information!in!order!to!get!the!right!
result.!On!the!other!hand,!if!we!take!a!place!where!rain!is!rare,!we!would!need!
the!same!information!as!compared!to!the!previous!place!but!this!information!will!
be!compact!and!less!as!compared!to!above!because!the!chances!of!rain!are!less!
and!the!information!can!simply!be!put!into!a!pattern.!This!shows!how!
randomness!increases!the!amount!of!information!required!to!reach!a!particular!
symbol.!Getting!to!know!how!the!amount!of!information!received!was!dependent!
