CSE 517A: Machine Learning

Project Descriptions

If you don't have any strong opinions about what sort of project you want to attempt for the class, we've got some suggestions for you. You can either pick one of the projects from the list below and do it as described, or use it as a starting point for a project proposal of your own. If you're going to attempt a project as described here, all you have to do is to tell me which one you're going to do. If you want to modify a project, you need to write a proposal on your own. If the modification is small, you don't need to bother with a full proposal. Just identify the bit you're going to change, and describe it in detail.

What We're Expecting

Some of the project descriptions below are quite short. In particular, they will lack a well-specified set of sets towards accomplishing the final goal. This is on purpose, since we want to make sure that you've learned something about setting up and conducting machine learning experiments this semester. For each of the projects, the first thing you should do is sit down and set out the steps that you will take to complete the project. If you have any questions about this, or want to talk about it, then get in touch with either the TAs or the instructor.

We're also a little vague about what we're expecting you to show in some of the projects. Again, this is because we're leaving the choice of the actual experiments up to you. In general, you should decide what question you're trying to answer. Then, ask what information you can provide to answer that question. When you're working on the projects, try to follow the spirit of the question. We're presenting you with a chance to show us what you can do. If you do the absolute minimum of work, you'll probably end up with the absolute minimum grade.

We're going to ask you to write a short report on your project. For most projects, this will be somewhere between 3 and 5 pages. Please be concise and to the point, since we have 50 of these things to read. We'll give you a little more information on the writeup shortly, and will provide you with an example of the sort of thing we're after.

Yes, these instructions are a bit vague. If you have any doubts or questions, please, please ask us before you hand in your final report. We're more than happy to discuss what we're after, and to give you some guidance.

Just to be explicit, you are allowed to use code you find on the web for the project. However, you must attribute your sources, and tell us what you used. There's no penalty for not writing everything yourself, but you should be honest about where things come from. As always, copying the results of experiments, and claiming them as your own is not allowed. For most of the projects, we're less interested in you implementing and algorithm that in seeing you apply it, and showing that you know what's going on. To this end, it probably doesn't make sense to implement "large" systems, such as a genetic algorithm, by hand. Searching for available code for these is a much better option. You can use any of the popular free software sites (such as SourceForge) or Google) to find this code, or we can offer pointers. If you have doubts about the quality of any code you find, feel free to check with us.

Projects

Here are some example projects. If you feel that you want to modify one of these, then feel free. Decide what modifications you want to make, write them up, and send them to us soon. If you're happy just doing one of the projects without changing it, email the instructor, and tell us which project you're doing. If there's any data sets or domains left open for you to choose, tell us which ones you're planning on using, so we can make sure they're appropriate.

Algorithm Bakeoff

Pick two or more of the machine learning algorithms that we covered this semester, and run it on a number (at least 5) data sets. Investigate the following things, for each algorithm:
  1. How does the performance vary as you vary the parameters?
  2. How does the performance vary with the different data sets?
  3. Why does the performance vary in the way that it does?
Try to find some data sets which are easy and some that are hard for each of the algorithms, and discuss why they are hard/easy. You can find the data sets at the UCI Machine Learning Repository, or at one of the other on-line data repositories. Remember to cite your sources.

Class Data Set

Learn something interesting about the data set we collected in class. Apply the algorithms we used in class to see if you can predict one attribute from another (or set of others). Are there any strange correlations. You might want to use more than one algorithm, since there is more than one type of data in this data set. We will send this data set to anyone who is interested in this project.

Reinforcement Learning for Control

Implement Q-learning for a standard reinforcement learning testbed problem. You can find four of these problems on Remi Munos' old web site. You should pick one of these domains, implement and test it. Then apply Q-learning to the project, and learn a control policy for the problem. You should discuss how you discretize the state space, what your actions are, your choice of parameters, and anything else you think is important. You should also do a literature search to see what techniques people have already applied to these domains, and how successful they were. You should be careful to report your results using the same metrics at the papers, and discuss your results in light of theirs. Google scholar is a good place to look for prior work. Your goal in this project is to learn as good a policy as you can. You should implement some or all of the speedups we covered in class, and should also look into the literature to see if there are other appropriate techniques.

More Reinforcement Learning for Control

This is essentially the same as the previous problem, except that it's a bit harder. We have a number of models of walking and swimming robots. The goal is to take these and learn a better locomotion strategy, using reinforcement learning. These are models that we're actively using in our own research, and you can get them from Tom Erez, who will also be able to fill you in on the sorts of systems we're interested in. The goal in this project is to learn more effective gaits and, hopefully, to outperform the systems that we currently have in place. If you're interested in joining my lab group, this is a good project to get you familiar with the sorts of projects we have.

Genetic Algorithms for Control

Use genetic algorithms to learn good control policies for the domains on Munos' web page, or one of our models. This will require you to think hard about how you're going to represent the policy: the easiest way is to assume some maximum number of steps, and have one action for each step. This has the disadvantage that it ignores the state the agent is in, and just forms an open loop set of actions. More sophisticated representations of the policy are likely to work better.

Advanced Features

Take one of the algorithms we covered in class, and add in some of the more advanced features discussed in the literature. For example, for decision trees, you can add in gain ration splitting, pruning, and non-axis-aligned splitting. You should evaluate the basic algorithm and the enhanced versions on a number of data sets (either the class data, or sets chosen form some repository). Report the results of using the enhancements, and discuss why you got the results you did on the evaluation data sets.

If you're going to do this project, tell us which algorithm that you're planning on using, and which enhancements you're going to implement and test.

Pick a Research Paper

Pick a machine learning research paper, and replicate the results in it. You can either look for a paper yourself, or we can suggest one if you give us some broad parameters (RL is cool, and I like the idea of speedups). To get full points on this, you should be able to completely replicate the results reported in the paper, and generate the same (or very similar) graphs and performance results. If you feel inspired to add something to the implementation, and improve on the results in the paper, that's even better.

Learn to Play a (Simple) Game

Use reinforcement learning to play a simple game, such as tic-tac-toe, against either a human or a machine opponent. There are a number of suitable games, and a large amount of previous work on this (which you are expected to be aware of). Possible games include:
  1. Simple card games
  2. Tic-tac-toe
  3. Simple chess end-games
  4. Simple versions of checkers
  5. The prisoner's dilemma
  6. Rock-paper-scissors (for which there is an international competition)
To learn properly, your opponent must have a (mostly) policy. Think about what you're going to evaluate against (random player, computer player, learning player).

If more than one person is interested in the same game, we can have a competition. If there's sufficient interest in game-playing, we will have a help-session dedicated to talking about the issues involved.

Genetic Algorithms for Learning Neural Network Weights

Implement a GA that learns the weights for a particular neural network. Compare the results of this learning (in times of convergence time, performance over time, etc) with the standard backpropagation algorithm. Implement a hybrid scheme which interleaves steps of the GA with steps of backprop, and report on the effects that this has. You should evaluate your algorithms on a number of data sets.

Planetary Data Sets

We have a number of data sets from space probes and planetary rovers, which you can browse at the PDS Geosciences Node. There are a number of very cool learning projects associated with these data. If you're interested in a project based on these, get in touch with the instructor, and we can arrange for a briefing on what's possible. These projects are likely to be somewhat more tricky, but they are based on real data from an on-going exploration project. Experience with image analysis (especially hyper-spectral images) is a huge plus for this project.

Neurological Data

We have a collection of brain activity recordings from surgical patients at the medical school. These data represent brain activity while the patient is performing some manipulation tasks, moving an on-screen cursor to a particular target and then hovering over it. The goal of the project is to learn predictive patterns of activity in the brain that signal reaching the target. This is part of an on-going brain-computer interface. If you're interested in this project, get in touch with me, and we will take things from there. Experience in signal processing and/or some knowledge of basic neuroanatomy is a plus, but not required for this project.