In the previous post, I described why parallel programming is hard. Now I am going to start describing techniques for writing parallel code that works correctly.

First some definitions.

• thread safe: code that works correctly even when called in parallel.
• critical section: an area of code that will not work correctly if run in parallel.
• shared: a resource that can be accessed by more than one thread.
• mutex: a programming tool that controls access to a section of code

Green is definitely the color of the 21^st century. Recently, I was attending the annual conference of the Society of Instrumentation and Control Engineers. The keynote was delivered by Dr. Tariq Samad of Honeywell and the President of the IEEE Control Systems Society.  The talk was on various dimensions in advanced control – past, present, and future, and in particular Dr. Samad summarized some fascinating work being done in the natural resources industry on advanced control.  Through his very interesting and engaging talk, my generally conservative brain went into green mode.

Dr. Samad gave a couple of examples of massive engineering undertakings that deployed highly sophisticated control strategies at unprecedented levels of innovation and complexity. The Olympic Dam mining operation in Australia is the largest PC-based deployment of digital control techniques in history, with over 500,000 I/O points. There are major applications of model-predictive control (control strategies where the controller has inherent knowledge of plant dynamics) in traditional coal power plants that will immediately reduce the harm from these plants and set the stage for the introduction to alternate power generation.

In my previous post, I tried to convince you that going parallel is necessary for high performance applications. In this post, I am going to show what makes parallel programming different, and why that makes it harder.

Here are some definitions used in this post:

• process: the operating system level representation of a running executable. Each process has memory that is protected from other processes.
• thread: within a single process each thread represents an independent, concurrent path of execution. Threads within a process share memory.

We think of a function as a series of discrete steps. Lets say a function f, is composed of steps f1, f2, f3... e.g.

Selling a company is emotionally wrenching. It was even more intensive for us at Maplesoft since we had a large number of founders who had been actively involved with the company for 20+ years. The decision for founders to sell a company so that it can move to the next stage is truly massive.

We had the luxury of a number of suitors with essentially equivalent initial financial offers, but this also destined us for a long process with lots of discussion and many twists and detours along the way. I remember the saying: that which does not kill you makes you stronger ;-). This was also my 2nd time going through the full process and maybe I can offer some advice to novices, noting the old saying that free advice is not always worth the price.

Sometime in 1992 I was offered the title of “Applications Engineer” at Maplesoft. I was the company’s very first employee to hold this title and it was my first real job.  I was thrilled! Imagine, if you will, an impoverished student who had been living on the most pitiful of incomes for almost ten years, all of a sudden being offered a great salary and the chance to travel and meet interesting people around the world! And for the most part, all I had to do was show people how great this thing called Maple was.

Computers with multiple processors have been around for a long time and people have been studying parallel programming techniques for just as along. However only in the last few years have multi-core processors and parallel programming become truly mainstream. What changed?

Here are some definitions for terms used in this post:

• core: the part of a processor responsible for executing a single series of instructions at a time.
• processor: the physical chip that plugs into a motherboard. A computer can have multiple processors, and each processor can have multiple cores
• process: a running instance of a program. A process's memory is usually protected from access by other processes.
• thread: a running instance of a process's code. A single process can have multiple threads, and multiple threads can be executing at the same on multiple cores
• parallel: the ability to utilize more than one processor at a time to solve problems more quickly, usually by being multi-threaded.

For years, processors designers had been able to increase the performance of processors by increasing their clock speeds. However a few years ago they ran into a few serious problems. RAM access speeds were not able to keep up with the increased speed of processors, causing processors to waste clock cycles waiting for data. The speed at which electrons can flow through wires is limited, leading to delays within the chip itself. Finally, increasing a processor's clock speed also increases its power requirements. Increased power requirements leads to the processor generating more heat (which is why overclockers come up with such ridiculous cooling solutions). All of these issues meant that is was getting harder and harder to continue to increase clock speeds.  The designers realized that instead of increasing the core's clock speed, they could keep the clock speed fairly constant, but put more cores on the chip. Thus was born the multi-core revolution.

My name is Darin Ohashi and I am a senior kernel developer at Maplesoft. For the last few years I have been focused on developing tools to enable parallel programming in Maple. My background is in Mathematics and Computer Science, with a focus on algorithm and data structure design and analysis. Much of my experience with parallel programming has been acquired while working at Maplesoft, and it has been a very interesting ride.

In Maple 13 we added the Task Programming Model, a high level parallel programming system. With the addition of this feature, and a few significant kernel optimizations, useful parallel programs can now be written in Maple. Although there are still limitations and lots more work to be done on our side, adventurous users may want to try writing parallel code for themselves.

To encourage those users, and to help make information about parallel programming more available, I have decided to write a series of blog posts here at Maple Primes. My hope is that I can help explain parallel programming in general terms, with a focus on the tools available in Maple 13. Along the way I may post links to sites, articles and blogs that discuss parallel programming issues, as well as related topics, such as GPU programming (CUDA, OpenCL, etc).

My next post, the first real one, I am going to explain why parallel programming has suddenly become such an important topic.

It’s been nearly ten years since I first walked onto the University of Waterloo campus as a freshly minted undergraduate, bright-eyed and bushy-tailed and eager to learn all about electrical engineering. I guess it’s hard to believe the speed with which time passes. It’s actually a bit astonishing how much I can still remember about orientation, or “frosh” week, like 4 a.m. fire drills, a very messy obstacle course, sitting with 800 other young engineering students in a lecture hall, and above all, meeting new friends.

Recently, we were asked by a designer of thrill rides if we could help them define a design tool that would allow them to push the envelope in rider experience, while considering engineering constraints and, of course, rider safety.

This week I decided to do some research and find out the details of how to make model animations with MapleSim, by adding in CAD drawing files of the component parts. To see what I mean, take a look at this quick animated movie that shows a robot arm with five degrees of freedom:

At the recent Vehicle Dynamics Expo in Stuttgart, I presented an example that demonstrates the speed with which you can perform the complete model-development-to-HIL process for a vehicle stability controller using MapleSim. The process begins with the development of a full-chassis vehicle model in MapleSim. This is a detailed model that includes the geometries for a double-wishbone suspension at the front and semi-trailing arms at the rear, with Fiala models for the tires. The stability controller, or Electronic Stability Program (ESP), is a predictive model based on a simplified vehicle model (referred to as the “bicycle model” since it only uses one wheel at the front and rear). When activated, the controller estimates what the desired yaw rate should be from the simple model, compares this with the actual yaw rate, and applies a braking force proportional to the difference to the appropriate front tire.

Through the landmark book, Zen and the Art of Motorcycle Maintenance, Robert Pirsig introduced generations of engineers to formal metaphysics. This engaging story chronicles the journey of a man and his teenage son on a single motorcycle through America. Through their encounters with challenges of all sorts, the man explores and wrestles with the notion of “quality”, in both the mechanical sense – the quality of his machine, and the human sense – the quality of a person or a relationship. I’m a big fan of interdisciplinary education and I was always thrilled to find out that this book is actually mandatory reading at many engineering universities. Today, I think I have a much better sense of where this thrill comes from.

A few weeks ago I made a short trip down to Lincroft, New Jersey, to deliver a Maple training course to a group of math professors at Brookdale Community College. I’d been to Manhattan before, but never New Jersey, and didn’t really know what to expect. I was pleasantly surprised to fly in over a lush and verdant landscape, with temperatures hovering in the mid-70s. My host, Barbara, had graciously invited me to her home for dinner, and we had a great conversation about teaching math, the Verrazano Bridge, dealing with deer in the backyard (Barbara was originally a Brooklyn girl, used to the concrete jungle!), and of course, what to expect the next day at the college.

Last week was rather crazy…

Monday afternoon, I’m sitting in a canoe on a beautiful lake in the wilds of Ontario…

…Tuesday morning, I’m in Stuttgart for the Vehicle Dynamics Expo to introduce MapleSim 2 to the many automotive engineers that have converged from just about every European nation, and beyond, to learn about new technologies and methodologies for the design of vehicle chassis systems, including  suspensions, steering, tires (or tyres, depending where you come from), and braking systems. One hot topic of discussion is the rapid development of vehicle stability controllers, given that all new passenger vehicle designs must now by law include active stability control. This is very timely for us because we are able to show our hardware-in-the-loop (HIL) demonstration that includes a full-vehicle model developed in MapleSim and running on dSPACE and National Instruments PXI real-time platforms, with a MotoTron prototype controller interfaced to the vehicle model via a CANbus interface. I’ll describe this in more detail in a later blog post.

“Keep Austin weird” is probably the best civic slogan I have ever encountered. Austin, Texas is one of the most charming cities in the US. It’s the capital of the state of Texas and also the self-professed live music capital of the world. In addition, the University of Texas at Austin is the largest university in the US, and its influence on the technology sector has spun off a very vibrant high technology business center as well. This mix of government, the arts, academia, and technology is the quintessential recipe for a very dynamic, vibrant, and yes, weird (in a good way) community.