Skip to main content

crash - Mathematica, MKL_NUM_THREADS, and GNU/Linux Process Limits


I am using Mathematica 10.3 on a large GNU/Linux system (CentOS 6.6) with 40 processors and 250 GB of RAM. It is a delight to watch it chew through linear algebra problems using all the processors.


However, it seems that the system becomes unstable, and perhaps even the Mathematica Kernel crashes, after evaluating expressions that use all of these resources. (A built-in example is to evaluate the NDEigensystem help notebook on the 40-core machine.) It seems as though the system has run out of permissible user processes.



When I issue (zsh):


 ~> limit

 cputime         unlimited
 filesize        unlimited
 datasize        unlimited
 stacksize       10MB
 coredumpsize    0kB
 memoryuse       unlimited
 maxproc         1024

 descriptors     1024
 memorylocked    64kB
 addressspace    unlimited
 maxfilelocks    unlimited
 sigpending      256546
 msgqueue        819200
 nice            0
 rt_priority     0

which leads me to believe that as many settings as are relevant are unlimited.



I have also explored various settings for OMP_NUM_THREADS and MKL_NUM_THREADS--setting these to smaller settings, e.g. MKL_NUM_THREADS=8, does not seem to exhibit the instability. However, the other cores remain unused, and these really need to be in play for Mathematica to remain competitive with other technologies such as R linked with MKL.


There are other settings for OMP and MKL, such as OMP_DYNAMIC, but I have not found a magic recipe to get this to work.


My question: have other users seen such (mis)behavior, and are there settings that can be invoked in order to get around these issues?




Comments

Post a Comment

Popular posts from this blog

functions - Get leading series expansion term?

Given a function f[x] , I would like to have a function leadingSeries that returns just the leading term in the series around x=0 . For example: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x)] x and leadingSeries[(1/x + 2 + (1 - 1/x^3)/4)/(4 + x)] -(1/(16 x^3)) Is there such a function in Mathematica? Or maybe one can implement it efficiently? EDIT I finally went with the following implementation, based on Carl Woll 's answer: lds[ex_,x_]:=( (ex/.x->(x+O[x]^2))/.SeriesData[U_,Z_,L_List,Mi_,Ma_,De_]:>SeriesData[U,Z,{L[[1]]},Mi,Mi+1,De]//Quiet//Normal) The advantage is, that this one also properly works with functions whose leading term is a constant: lds[Exp[x],x] 1 Answer Update 1 Updated to eliminate SeriesData and to not return additional terms Perhaps you could use: leadingSeries[expr_, x_] := Normal[expr /. x->(x+O[x]^2) /. a_List :> Take[a, 1]] Then for your examples: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x), x] leadingSeries[Exp[x], x] leadingSeries[(1/x + 2 + (1 - 1/x...
Post a Comment
Read more

mathematical optimization - Minimizing using indices, error: Part::pkspec1: The expression cannot be used as a part specification

I want to use Minimize where the variables to minimize are indices pointing into an array. Here a MWE that hopefully shows what my problem is. vars = u@# & /@ Range[3]; cons = Flatten@ { Table[(u[j] != #) & /@ vars[[j + 1 ;; -1]], {j, 1, 3 - 1}], 1 vec1 = {1, 2, 3}; vec2 = {1, 2, 3}; Minimize[{Total@((vec1[[#]] - vec2[[u[#]]])^2 & /@ Range[1, 3]), cons}, vars, Integers] The error I get: Part::pkspec1: The expression u[1] cannot be used as a part specification. >> Answer Ok, it seems that one can get around Mathematica trying to evaluate vec2[[u[1]]] too early by using the function Indexed[vec2,u[1]] . The working MWE would then look like the following: vars = u@# & /@ Range[3]; cons = Flatten@{ Table[(u[j] != #) & /@ vars[[j + 1 ;; -1]], {j, 1, 3 - 1}], 1 vec1 = {1, 2, 3}; vec2 = {1, 2, 3}; NMinimize[ {Total@((vec1[[#]] - Indexed[vec2, u[#]])^2 & /@ R...
Post a Comment
Read more

How to remap graph properties?

Graph objects support both custom properties, which do not have special meanings, and standard properties, which may be used by some functions. When importing from formats such as GraphML, we usually get a result with custom properties. What is the simplest way to remap one property to another, e.g. to remap a custom property to a standard one so it can be used with various functions? Example: Let's get Zachary's karate club network with edge weights and vertex names from here: http://nexus.igraph.org/api/dataset_info?id=1&format=html g = Import[ "http://nexus.igraph.org/api/dataset?id=1&format=GraphML", {"ZIP", "karate.GraphML"}] I can remap "name" to VertexLabels and "weights" to EdgeWeight like this: sp[prop_][g_] := SetProperty[g, prop] g2 = g // sp[EdgeWeight -> (PropertyValue[{g, #}, "weight"] & /@ EdgeList[g])] // sp[VertexLabels -> (# -> PropertyValue[{g, #}, "name"]...
Post a Comment
Read more