Teaching | Avik Kar

Markov Decision Processes

Module 1A — Foundations of Probability
Module 1B — Stochastic Processes and DTMC
Module 1C — Foundations of Optimization
Module 2A — Introduction to Markov Decision Processes
Module 2B — Finite-Horizon Markov Decision Processes
Module 2C — Infinite-Horizon Discounted Markov Decision Processes
Module 2D — Infinite-Horizon Average Reward Markov Decision Processes
Module 3A — Constrained Markov Decision Processes
Module 3B — Partially Observable Markov Decision Processes

Random Processes (E2 202)

08 Aug 2022 Tutorial 01 — Cardinality. Probability Space. Limits of Sets. Continuity of Probability.
15 Aug 2022 Tutorial 02 — Law of Total Probability. Independence of Events. Conditional Probability. Random Variables.
22 Aug 2022 Tutorial 03 — Random Vectors. Transformation of Random Variables and Vectors.
29 Aug 2022 Tutorial 04 — Random Processes. Expectations.
05 Sep 2022 Tutorial 05 — Moments. Correlation. L^p Space. Inequalities.
12 Sep 2022 Tutorial 06 — Generating Functions. Conditional Expectation.
19 Sep 2022 Tutorial 07 — Almost Sure Convergence. Convergence in Probability. Borel-Cantelli Lemma.
24 Oct 2022 Tutorial 08 — L^p Convergence. Convergence in Distribution. Problems on Convergence of RV.
26 Sep 2022 Tutorial 09 — Law of Large Numbers. Central Limit Theorem.
10 Oct 2022 Tutorial 10 — Introduction to Discrete Time Markov Chains.
17 Oct 2022 Tutorial 11 — DTMC: Recurrent and Transient States.
24 Oct 2022 Tutorial 12 — DTMC: Communicating Classes. Invariant Distribution.