COURSE UNIT TITLE

: INTRODUCTION TO INFORMATION THEORY

Description of Individual Course Units

Course Unit Code Course Unit Title Type Of Course D U L ECTS
IST 3121 INTRODUCTION TO INFORMATION THEORY ELECTIVE 3 0 0 5

Offered By

Statistics

Level of Course Unit

First Cycle Programmes (Bachelor's Degree)

Course Coordinator

PROFESSOR DOCTOR ÖZLEM EGE ORUÇ

Offered to

Statistics
Statistics(Evening)

Course Objective

The objective of the course to expose students to the fundamental elements and practices of information theory, covering both theoretical and applied issues of recognized importance in data/information compression, transmission, storage and processing.

Learning Outcomes of the Course Unit

1   Describing Entropy
2   Obtaining joint, conditional, relative entropy and mutual information
3   Understanding chain rules
4   Describing Jensen s inequality, its consequences and Fano s inequality
5   Understanding coding theorem and algorithms
6   Calculating differential and maximum entropy
7   Understanding how to use information theory in statistics

Mode of Delivery

Face -to- Face

Prerequisites and Co-requisites

None

Recomended Optional Programme Components

None

Course Contents

Week Subject Description
1 Elements of Probability
2 Entropy
3 Joint Entropy and Conditional Entropy
4 Relative Entropy and Mutual Information
5 Chain Rules for Entropy, Relative Entropy and Mutual Information
6 Jensen s Inequality and Its Consequences
7 Fano's inequality
8 Examples of Codes
9 Examples of Codes Continued
10 Source Coding theorem
11 Source Coding Algorithms
12 Differential Entropy
13 Information Theory in Statistics
14 Maximum Entropy

Recomended or Required Reading

Textbook(s):
T. Cover, J. Thomas, Elements of Information Theory, , Wiley, 2006
Supplementary Book(s):
R. W. Yeung. A First Course in Information Theory. Springer, New York, 2002

Planned Learning Activities and Teaching Methods

Lecture, homework assignments, problem solving

Assessment Methods

SORTING NUMBER SHORT CODE LONG CODE FORMULA
1 MTE MIDTERM EXAM
2 ASG ASSIGNMENT
3 FIN FINAL EXAM
4 FCG FINAL COURSE GRADE MTE * 0.40 + ASG * 0.10 + FIN * 0.50
5 RST RESIT
6 FCGR FINAL COURSE GRADE (RESIT) MTE * 0.40 + ASG * 0.10 + RST * 0.50


Further Notes About Assessment Methods

None

Assessment Criteria

Evaluation of exams and homework

Language of Instruction

English

Course Policies and Rules

Attendance to at least 70% for the lectures is an essential requirement of this course and is the responsibility of the student. It is necessary that attendance to the lecture and homework delivery must be on time. Any unethical behavior that occurs either in presentations or in exams will be dealt with as outlined in school policy. You can find the undergraduate policy at http://web.deu.edu.tr/fen

Contact Details for the Lecturer(s)

DEU Faculty of Sciences Department of Statistics
e-posta:ozlem.ege@deu.edu.tr
Tel: 0232 301 85 58

Office Hours

To be announced.

Work Placement(s)

None

Workload Calculation

Activities Number Time (hours) Total Work Load (hours)
Lectures 13 3 39
Preparations before/after weekly lectures 12 1 12
Preparation for midterm exam 1 24 24
Preparation for final exam 1 32 32
Preparing assignments 1 5 5
Final 1 2 2
Midterm 1 2 2
TOTAL WORKLOAD (hours) 116

Contribution of Learning Outcomes to Programme Outcomes

PO/LOPO.1PO.2PO.3PO.4PO.5PO.6PO.7PO.8PO.9PO.10PO.11PO.12PO.13PO.14
LO.155433
LO.255433
LO.355433
LO.455433
LO.555433
LO.655433
LO.755433