Archive | November 2006

Parallel Computing Needs to be More Accessible

There are two truths about parallel computing.

1. Parallel computing is hard.

To quote from a February 2005 article by Herb Sutter in Dr. Dobb’s Journal:

Probably the greatest cost of concurrency is that concurrency really is hard: The programming model, meaning the model in programmers’ heads that they need to reason reliably about their program, is much harder than it is for sequential control flow.

2. Parallel computing is going mainstream.

To quote Sutter again:

… if you want your application to benefit from the continued exponential throughput advances in new processors, it will need to be a well-written, concurrent (usually multithreaded) application. And that’s easier said than done, because not all problems are inherently parallelizable and because concurrent programming is hard.

Because 1 and 2 are to some extent in opposition, we have an escalating situation.

So, this means we have to do a better job of making parallel computing, well, less hard – i.e., more accessible.

Since returning to York University last April, this has become resoundingly clear to me. In fact, it is resulting in an Accessible Parallel Computing Initiative. I hope to be able to share much more about this soon. For now, you can read over my abstract for the upcoming HPCS 2007 event:

Accessible Parallel Computing via MATLAB

Parallel applications can be characterized in terms of their granularity and concurrency. Whereas granularity measures computation relative to communication, concurrency considers the degree of parallelism present. In addition to classifying parallel applications, the granularity-versus-concurrency template provides some context for the strategies used to introduce parallelism. Despite the availability of various enablers for developing and executing parallel applications, actual experience suggests that additional effort is required to reduce the required investment and increase adoption. York University is pioneering an investment-reducing, adoption-enhancing effort based on the use of MATLAB, and in particular the MATLAB Distributed Computing Toolbox and Engine. In addition to crafting an appropriate environment for parallel computing, the researcher-centric York effort places at least as much emphasis on the development and execution of parallel codes. In terms of delivery to the York research community, MATLAB M-files will be shared in a tutorial context in an effort to build mindshare and directly engage researchers in parallel computing. Although MATLAB shows significant promise as a platform for parallel computing, some limitations have been identified. Of these, support for threaded applications in a shared-memory context and limited support for the Message Passing Interface (MPI) are of gravest concern.

As always, I welcome your feedback.