Skip to main content

performance tuning - Launching remote kernels - will it help RAM needs?


Given the exasperating RAM needs of my Mathematica program, I contemplated launching remote kernels to run my Mathematica script. This is a two part question:


Part 1


If I launch a remote kernel(s), is the need for RAM also distributed amongst all the remote computers?


Part 2


I keep getting an error message when I try to launch a remote kernel from the Mathematica GUI.



The kernel c20-0707-23 failed to connect to the front end. (Error = MLECONNECT). You should try running the kernel connection outside the front end.



I have located the Kernel Program correctly on the remote computer. When I try starting a kernel over ssh in a terminal window with:



ssh username@remote.computer /path/to/math

Or


ssh username@remote.computer /path/to/MathKernel

After I enter the password to log in to the remote computer, I see that a kernel has been launched.


However, the same doesn't occur with the front end.


Are there any easy to understand tutorials/examples on launching remote kernels, without needing the GUI as I'd like to run scripts? I tried reading, this, for instance and the only thing that it did was make me panic as it was greek. I also read this but it's not an entirely similar problem


Here's a sample script that I used to launch local kernels and execute a simple NDSolve command:


#!/usr/local/bin/MathematicaScript -script


CloseKernels[]
LaunchKernels[3]
Print["AbsoluteTiming reveals:" NDSolve[{y'[x] == y[x] Cos[x + y[x]],
     y[0] == 1}, y, {x, 0, 30}]; // AbsoluteTiming ]
CloseKernels[]

This question may help answer the other question I linked in the first line.



Answer



Sometimes, yes it is sometimes distributed. I can't say for sure when and why, but I have noticed cases where ParallelMap does not distribute all the memory, but ParallelTable does.



Basically, Mathematica is trying to make things easy. You don't always want to have to spell out each and every variable that you want available on a remote kernel, and so it does its best to figure out just what you need. There are lots of options in Mathematica 8.


If you're trying to "aggregate" memory on other nodes, then you have to be careful to create the large data on remote machines to avoid the network transfer of the contents of the master kernel to the other machines. (Please know that I have run into issues in the past, where there seemed to be a limit as to precisely how much memory could be allocated by a Remote Kernel itself. i.e. couldn't allocate a gigabyte, let alone 16 gigabytes that are found in machines today. I don't know if this is still a problem.)


Comments

Post a Comment

Popular posts from this blog

plotting - How to draw lines between specified dots on ListPlot?

I would like to create a plot where I have unconnected dots and some connected. So far, I have figured out how to draw the dots. My code is the following: ListPlot[{{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4,13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full] I have thought using ListLinePlot command, but I don't know how to specify to the command to draw only selected lines between the dots. Do have any suggestions/hints on how to do that? Thank you. Answer One possibility would be to use Epilog with Line : ListPlot[ {{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4, 13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full, Epilog -> { Line[ ...
Post a Comment
Read more

dynamic - How can I make a clickable ArrayPlot that returns input?

I would like to create a dynamic ArrayPlot so that the rectangles, when clicked, provide the input. Can I use ArrayPlot for this? Or is there something else I should have to use? Answer ArrayPlot is much more than just a simple array like Grid : it represents a ranged 2D dataset, and its visualization can be finetuned by options like DataReversed and DataRange . These features make it quite complicated to reproduce the same layout and order with Grid . Here I offer AnnotatedArrayPlot which comes in handy when your dataset is more than just a flat 2D array. The dynamic interface allows highlighting individual cells and possibly interacting with them. AnnotatedArrayPlot works the same way as ArrayPlot and accepts the same options plus Enabled , HighlightCoordinates , HighlightStyle and HighlightElementFunction . data = {{Missing["HasSomeMoreData"], GrayLevel[ 1], {RGBColor[0, 1, 1], RGBColor[0, 0, 1], GrayLevel[1]}, RGBColor[0, 1, 0]}, {GrayLevel[0], GrayLevel...
Post a Comment
Read more

Is there a way to do conditional matrix loop using 'continue'

I have the following: n = 3; m = 5; ww = RandomReal[{0, 0.1}, {n, n}]; uu = RandomReal[{0, 1}, {m, n}]; pp = RandomReal[{0, 1}, {n, n}]; ss = RandomInteger[{0, 5}, {m, n}]; Grid[{{"ww", "uu", "pp", "ss"}, {ww // TableForm, uu // TableForm, pp // TableForm, ss // TableForm}}, Spacings -> {5, 2}, Dividers -> All] where I would like to look at every element of matrix ss and produce a matrix tt , with zeroes at the locations in ss which have zeroes, and in all other positions do the following: tt = (-1/Subscript[ww, m]) Log[(1 - uu)/(Subscript[pp, m - 1])], where Subscript[ww, m] is the value at index of ww matrix and where Subscript[pp, m - 1] is the value at index-1 of pp matrix. So for example if the first value ever read from matrix ss happens to be 2, then value taken from matrix ww would be from the row 2, but from pp would be from row 1. Also how to tell difference between a 0 as a valid value from within the matrix elemen...
Post a Comment
Read more