Expectation Maximization Part 2

This assignment is designed to:

• provide practice with essential ideas for using the expectation maximization (EM) algorithm
• help you become more fluent with the terminology and the techniques
• prepare you for the next assignment dedicated to using EM for document clustering

This is a mathematical and coding homework assignment. Where math is involved, show your work. Be clear and concise. Where code is required, include your code. Type your assignment.

Work through these exercises as soon as possible for their instructional content. If you have general questions about the assignment, please post on the Google Group. Finish early, and earn the early bonus.

Assume the context established by Assignment 5.75.

[40 points]

• Write code to compute the kind of results required earlier for questions #5 (EM), #7 (likelihood), and #8 (cluster assignment) in Assignment 5.75, and do so in a manner that it will accept an arbitrary set of input triples.
• Include your code.
• Now reproduce your output for the input specified in question #5.
• Show your output.

[20 points]

• Re-run your EM code for a different partially observed data set: $\langle HHH \rangle,\langle TTT \rangle,\langle HHH \rangle,\langle TTT \rangle$
• Produce a table as in questions #5 and #7 and cluster assignments as in question #8 in Assignment 5.75.
• Discuss what is different between the results from this problem instance and what you saw in question #9.

[20 points]

• Re-run your EM code on the same data set as question #10 but start with different initial parameter values as follows:
  • $\lambda = 0.3000$
  • $\beta_1 = 0.7000$
  • $\beta_2 = 0.7000$
• Describe your observations.

[10 points]

• Generalize your code to handle more interesting events, such as $\langle HTH \rangle$. (It is possible that no changes may be necessary, if you had the general case in mind.)
• Run your generalized code on a small example, and explain why the results are valid.

[10 points]

• Generalize your code to handle events drawn from a more interesting vocabulary, such as a small (4-word) English dictionary, instead of $\{H,T\}$.
• Run your generalized code on a small example, and explain why the results are valid.

Submit a .pdf document through Learning Suite.