parallelization - Are built-in Mathematica functions already parallelized?

I've been noticing something strange since updating to Mathematica 8, and that is that occaisionally I'll see that the MathKernel is using up to 800% CPU in my Activity Monitor on OS X (I have 8 cores). I have no Parallel calls whatsoever, and this is in a single kernel, not across multiple kernels. My code is pretty much only Interpolates, Maps, Do loops, and plotting routines.

I'm curious if some of the built-in Mathematica routines are in fact already parallel, and if so, which ones?

Answer

Natively multi-threaded functions

A lot of functions are internally multi-threaded (image processing, numerical functions, etc.). For instance:

In[1]:= a = Image[RandomInteger[{0, 255}, {10000, 10000}], "Byte"];

In[2]:= SystemOptions["ParallelOptions"]

Out[2]= {"ParallelOptions" -> {"AbortPause" -> 2., "BusyWait" -> 0.01,
    "MathLinkTimeout" -> 15., "ParallelThreadNumber" -> 4, 

   "RecoveryMode" -> "ReQueue", "RelaunchFailedKernels" -> False}}

In[3]:= ImageResize[a, {3723, 3231}, 
   Resampling -> "Lanczos"]; // AbsoluteTiming

Out[3]= {1.2428834, Null}

In[4]:= SetSystemOptions[
 "ParallelOptions" -> {"ParallelThreadNumber" -> 1}]


Out[4]= "ParallelOptions" -> {"AbortPause" -> 2., "BusyWait" -> 0.01, 
  "MathLinkTimeout" -> 15., "ParallelThreadNumber" -> 1, 
  "RecoveryMode" -> "ReQueue", "RelaunchFailedKernels" -> False}

In[5]:= ImageResize[a, {3723, 3231}, 
   Resampling -> "Lanczos"]; // AbsoluteTiming

Out[5]= {2.7461943, Null}

Functions calling optimized libraries

Mathematica surely gets benefit from multi-threaded libraries (such as MKL) too:

In[1]:= a = RandomReal[{1, 2}, {5000, 5000}];

In[2]:= b = RandomReal[1, {5000}];

In[3]:= SystemOptions["MKLThreads"]

Out[3]= {"MKLThreads" -> 4}

In[4]:= LinearSolve[a, b]; // AbsoluteTiming


Out[4]= {4.9585104, Null}

In[5]:= SetSystemOptions["MKLThreads" -> 1]

Out[5]= "MKLThreads" -> 1

In[6]:= LinearSolve[a, b]; // AbsoluteTiming

Out[6]= {8.5545926, Null}

Although, the same function may not get multi-threaded depending on the type of input.

Compiled function

CompiledFunctions and any other functions that automatically use Compile can be multi-threaded too, using Parallelization option to Compile.

Caution

1. 
   

   Measuring timing with AbsoluteTiming for multi-threaded functions could be inaccurate sometimes.

   
   


2. 
   
   

   The performance gain is usually not direct proportion to the number of threads. It depends on a lot of different factors.

   
   


3. 
   

   Increasing number of threads (by using SetSystemOptions ) more than what your CPU support (either physical or logical cores) is not a good idea.