Python, pycuda, gpu computing et. al.

I have often thought about why is it that nature is so fast whereas our simulations of it are so annoyingly slow and my mind always goes to the same sort of conclusion, nature is a massively parallel system, a seamless collection of very simple processes all taking place at the same time. It then appears that a computation strategy which mimics this parallelism to a certain extent will serve to gain much in terms of sheer computational power and efficiency. Of course there must be incredible challenges or it would be done by now but there you are. Now GPU (graphical processing unit) computing is something which looks suspiciously relevant in this scenario and I have begun exploring it a little for my own purposes. GPU computing allows one to execute massive number of relatively simple processes all at the same time. The catch is that the problem often has to be recast to make it suitable for treatment within the GPU paradigm but if this can be done, and done well, it may lead to significant improvements in efficiency.

Having said that any researcher worth his salt should begin coding in C or fortran and then exploit NVIDIA's CUDA (Compute Unified Device Architecture) to do computations on the NVIDIA GPUs present in most modern computers. I, however, am not a researcher worth my salt, at least not yet by that definition. I code in MATLAB and have recently moved to Python, driven by my pursuit to rid myself of all software which is not exactly free. Python coupled with Numpy is a formidable competitor to MATLAB and it makes sense to eventually port everything to it, if not C and Fortran. This still leaves the issue of parallelization. Fortunately NVIDIA has begun supporting python through PyCuda and I have spent the better part of the last two days trying to figure out how to make it all work. To use Python+Numpy+PyCUDA I needed to install a few things on my machine (Windows 7). Here is a list:

1. Python 2.7.5 from http://www.python.org/getit/ (I used the x86-64 installer which is suitable for 64 bit Win7). I went to the directory where Python is installed and copied its address to the PATH variable. This allows you to run the Python interpreter from the command line.

2. Numpy. A great list of Python packages is at http://www.lfd.uci.edu/~gohlke/pythonlibs/ with numpy being at http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy. I initially made the mistake of using a comprehensive Python distribution (Anaconda) which contains both Python and Numpy. It, however, did not make the correct registry entries and, therefore, gave me incredible grief when I was trying to install CUDA. I found it simpler just to do everything myself. Additional necessary packages which I installed from this page were Pytools, Setuputils, matplotlib (and its dependencies).

3. PyCuda can also be installed from http://www.lfd.uci.edu/~gohlke/pythonlibs/#pycuda

4. It's nice to have an Integrated Development Environment (IDE) for Python and I have installed Eclipse for this purpose. You can download it at http://www.eclipse.org/ and look at http://www.vogella.com/articles/Python/article.html for an excellent tutorial on how to get it running and making Python codes.

5. Visual Studio. I haven't been able to figure out why this is required yet but apparently it is essential. I installed VS 2012 Professional from http://www.microsoft.com/visualstudio/eng/downloads.

6. Finally it is time to install CUDA. This can be done from https://developer.nvidia.com/cuda-downloads, choosing the appropriate version for your OS.

Once all this is done you just have to write a python code appropriately configured to use CUDA and voila, it won't run! The trouble is that certain connections which let python know which compiler to use to invoke the CUDA parts of the code don't exist yet. To make those connections consult this: http://wiki.tiker.net/PyCuda/Installation/Windows. However, I had to make appropriate changes to the PATH instructions because my VS is 2012 and not 2008. For some reason my VS installation was also not in the same directory as shown in that link. Moreover, it was a bit of a trouble finding nvcc.profile and then editing it given that it is a read only file to begin with. Anyway, I'm all set now!