Final Exam Study Guide


The final exam is scheduled in the classroom on the date scheduled by the University (see the course schedule on Learning Suite). You will have a 3 hour time limit. The exam is closed book, and you may have no notes. No calculators or digital assistants (you won't need one). I think a well-prepared student will be able to complete the exam in two hours. You're going to do well!

The exam is comprehensive. The format is short answers, worked mathematical solutions, and possibly some T/F.

The difficulty level is comparable to the difficulty of the mid-term exam.

You will be expected to show your work. You will be graded not only on the correctness of your answers, but also on the clarity with which you express your rationale for your answers; be neat. It is your job to make your understanding clear to the graders; non-neat work is likely to earn a lower grade. If using a pencil (rather than a pen) helps you be neat, please plan accordingly.


I recommend the following activities to study the topics covered by the exam:

  • Review the lecture notes and identify the topics we emphasized in class. Focus on those topics listed below.
  • Compare the homework solution keys to your homework assignments, and make sure that you understand the major principles covered in the homework problems.
  • While you are reviewing the lecture notes, the homework solutions, and the topics in the mid-term study guide and this final study guide, I strongly encourage you to build the following lists:
    • Problems (e.g., classification, clustering)
    • Models (e.g., Naive Bayes, Gausian Mixture Model)
    • Algorithms (e.g., the Viterbi algorithm, the Expectation Maximization (EM) algorithm)
    • Theories (e.g., probability theory)
    • Methodologies (e.g., feature engineering, unsupervised learning)
  • Identify common themes and ideas within each of the lists. This will aid you in organizing your thoughts and making comparisons and contrasts.


The final exam is comprehensive and will cover a subset of the following topics as well as topics from the mid-term exam study guide:

  1. Steps of the Expectation Maximization algorithm
  2. Mixture models
  3. Mixture of multinomials model
  4. NO deriving new EM algorithms for new models
  5. Initialization for Expectation Maximization
  6. Computing the likelihood of the data according to a model
  7. Converting likelihood expressions into log-space
  8. Interpreting Hierarchical Bayesian models
  9. (Multivariate) Gaussian distributions
  10. Gaussian Mixture Models (GMMs)
  11. Sequence labeling
  12. Part-of-speech tagging
  13. Hidden Markov Models (HMMs)
  14. Independence assumptions in HMMs
  15. The Viterbi algorithm
  16. Components of a speech recognition system
  17. Application of HMMs in speech recognition
  18. Application of GMMs in speech recognition
  19. Formulating recognition problems in the source/channel (aka “noisy channel”) paradigm
  20. Language models as Markov chains
  21. Decoding as search
  22. Beam search as an approximation to the Viterbi algorithm
  23. The Monte Carlo principle
  24. Gibbs Sampling
  25. Justifying steps in the derivation of complete conditional distributions for Gibbs sampling
  26. NO novel derivations of complete conditional distributions for Gibbs sampling
  27. Document clustering with Gibbs sampling on a mixture of multinomials
  28. Metrics for clustering
  29. Topic modeling and topic discovery
  30. Latent Dirichlet Allocation (LDA): the generative story and model
  31. Inference in LDA using Gibbs sampling
  32. Strengths and limitations of joint models
  33. Strengths and limitations of conditional models
  34. Answering conditional queries using a joint model versus using a conditional model directly
  35. Maximum entropy classifiers / Logistic regression
  36. NO derivations of gradients of the likelihood (using differential Calculus) for gradient descent / ascent learning of maximum entropy model parameters
  37. The feature engineering cycle
  38. Pros and cons of Naive Bayes versus Maximum entropy as classifiers
cs-401r/final-topics.txt · Last modified: 2014/12/12 11:40 by ringger
Back to top
CC Attribution-Share Alike 4.0 International = chi`s home Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0