Discuss approaches to the Netflix prize using Python, getting started with [[http://pyflix.python-hosting.com/|PyFlix]] for new people, algorithm + code performance, etc

Some Netflix code in Python will be shown/run (KNN, NMF, ARTmap, SVD, etc).

I will be posting the code later this month on my blog: [[http://www.datawrangling.com|Data Wrangling]]


Some links for those just getting started:
 *[[http://www.netflixprize.com/teams|Register a Team]] in order to [[http://www.netflixprize.com/download|download the Netflix data]]
 *[[http://pyflix.python-hosting.com/|PyFlix]] library for efficiently handling the dataset.
 *[[http://www.grouplens.org/node/73|Movielens dataset]] - smaller dataset to debug your code with...

Some approaches:
 *[[http://sifter.org/~simon/journal/20061211.html|Simon Funk approach]]
 *[[http://www.timelydevelopment.com/demos/NetflixPrize.aspx|Timely Development code for Simon Funk approach]]
 *[[http://www.netflixprize.com/community/viewtopic.php?pid=4712#p4712|Netflix forum KNN discussion]] - includes numpy, weave specifics
 *[[http://devlicio.us/blogs/billy_mccafferty/archive/2006/11/07/netflix-memoirs-using-the-pearson-correlation-coefficient.aspx|Basic KNN in SQL]]
 *[[http://mainline.brynmawr.edu/Courses/cs380/fall2006/TiVo.pdf|Tivo KNN paper]]
 *[[http://www.erikshelley.com/netflix/|Erik Shelly's approach]]
 *[[http://www.tillberg.us/netflixprizejumpstart|Dan Tillberg's page]]
 *[[http://www.logarithmic.net/pfh/blog/01176798503|Paul Harrison's approach]] - using numpy and weave
 *[[http://www.siam.org/meetings/sdm06/proceedings/059zhangs2.pdf|Dartmouth paper]] - using EM/NMF approach with Movielens data
 *[[http://www.research.att.com/~volinsky/netflix/ProgressPrize2007BellKorSolution.pdf|BellKor paper]] - Progress prize winner
 *[[http://code.google.com/p/canopy-clustering/|Hadoop MapReduce code]] for working with the Netflix data
 
More here:

 *http://del.icio.us/pskomoroch/netflixprize
 *http://del.icio.us/pskomoroch/collaborative%2Bfiltering

Performance pointers:

 *http://www.scipy.org/PerformancePython
 *http://wiki.python.org/moin/PythonSpeed/PerformanceTips
 *http://www.scipy.org/Weave
 *If you need to go parallel for Netlfix, [[http://www.datawrangling.com/pycon-2008-elasticwulf-slides.html|ElasticWulf]] public Amazon EC2 images come with mpi4py, IPython1, pyflix, numpy, scipy, weave, pyrex, etc. already installed and configured.  The [[http://code.google.com/p/elasticwulf/|python code]] for launching your own beowulf on EC2 using the images is on google code.

Parallel Programming is useful for lots of ML algorithms.  [[http://www.dehora.net/journal/2005/02/two_classic_hardbacks.html|How to Write Parallel Programs]] is a good book. [[http://www.amazon.com/How-Write-Parallel-Programs-Course/dp/026203171X/|Amazon]]  Consider jython, since ML is often CPU-bound, and jython has no GIL.