Reinforcement Learning Challenges: frozen lake and tic tac toe

This is mostly here as a reminder to get to these topics because I know you want to cover them eventually.
I think it a good starting point is recreating the game from openAI gym, frozen lake and building an agent that could play it completely from scratch, maybe even animate the playing.
Also, I think it would be cool to build a deep q agent with tf.js that could play tic tac toe, I know tic tac toe can be solved with min-max but I think it is a good starting point for deep q learning because it has a relatively small state space of 3^9.