High Performance Python tutorial v0.2 (from EuroPython 2011)

My updated High Performance Python tutorial is now available as a 55 page PDF. The goal is to take you on several journeys which show you different ways of making Python code run much faster (up to 75* on the CPU, faster with a GPU).

This is an update to the 49 page v0.1 I published three weeks ago after running the tutorial at EuroPython 2011 in Florence.

Topics covered:

• Python profiling (cProfile, RunSnake, line_profiler) – find bottlenecks
• PyPy – Python’s new Just In Time compiler, a note on the new numpy module
• Cython – annotate your code and compile to C
• numpy integration with Cython – fast numerical Python library wrapped by Cython
• ShedSkin – automatic code annotation and conversion to C
• numpy vectors – fast vector operations using numpy arrays
• NumExpr on numpy vectors – automatic numpy compilation to multiple CPUs and vector units
• multiprocessing – built-in module to use multiple CPUs
• ParallelPython – run tasks on multiple computers
• pyCUDA – run tasks on your Graphics Processing Unit
• Other algorithmic choices and options you have

The improvement over the last version (v0.1) is that I’ve filled in all the sections now including pyCUDA (there are still a few IAN_TODOs marked, I hope to finish these in a future v0.3). I’ve also added a short section on Algorithmic Choices, link to the new Cython prange operator and show the new numpy module in PyPy.

The source code is on my github page. The original slides are on slideshare too. If you’re after a challenge then at the end of the report I suggest some ported versions of the code that I’d like to see.

The report is licensed Creative Commons by Attribution (please link back here) – I’ll also happily accept a beer if you meet me in person! If you’re curious about this sort of work then note that I offer A.I. and high performance computing consulting and training via my Mor Consulting.

Update – ShedSkin 0.9 adds faster complex number support. I haven’t added it to the report yet, evidence in the ShedSkin Group suggests it gets closer to the non-complex-number version (i.e. you don’t have to do more work but you get a nice speed boost whilst still using complex numbers).

Ian applies Artificial Intelligence as an Artificial Intelligence Researcher for companies (

), co-founded the

A.I. datamining toolkit, co-authored

, programs Python, writes

and is also a sea-side dweller and consumer of fine coffees.