Skip to main content

performance tuning - Launching remote kernels - will it help RAM needs?


Given the exasperating RAM needs of my Mathematica program, I contemplated launching remote kernels to run my Mathematica script. This is a two part question:


Part 1


If I launch a remote kernel(s), is the need for RAM also distributed amongst all the remote computers?


Part 2


I keep getting an error message when I try to launch a remote kernel from the Mathematica GUI.



The kernel c20-0707-23 failed to connect to the front end. (Error = MLECONNECT). You should try running the kernel connection outside the front end.



I have located the Kernel Program correctly on the remote computer. When I try starting a kernel over ssh in a terminal window with:



ssh username@remote.computer /path/to/math

Or


ssh username@remote.computer /path/to/MathKernel

After I enter the password to log in to the remote computer, I see that a kernel has been launched.


However, the same doesn't occur with the front end.


Are there any easy to understand tutorials/examples on launching remote kernels, without needing the GUI as I'd like to run scripts? I tried reading, this, for instance and the only thing that it did was make me panic as it was greek. I also read this but it's not an entirely similar problem


Here's a sample script that I used to launch local kernels and execute a simple NDSolve command:


#!/usr/local/bin/MathematicaScript -script


CloseKernels[]
LaunchKernels[3]
Print["AbsoluteTiming reveals:" NDSolve[{y'[x] == y[x] Cos[x + y[x]],
y[0] == 1}, y, {x, 0, 30}]; // AbsoluteTiming ]
CloseKernels[]

This question may help answer the other question I linked in the first line.



Answer



Sometimes, yes it is sometimes distributed. I can't say for sure when and why, but I have noticed cases where ParallelMap does not distribute all the memory, but ParallelTable does.



Basically, Mathematica is trying to make things easy. You don't always want to have to spell out each and every variable that you want available on a remote kernel, and so it does its best to figure out just what you need. There are lots of options in Mathematica 8.


If you're trying to "aggregate" memory on other nodes, then you have to be careful to create the large data on remote machines to avoid the network transfer of the contents of the master kernel to the other machines. (Please know that I have run into issues in the past, where there seemed to be a limit as to precisely how much memory could be allocated by a Remote Kernel itself. i.e. couldn't allocate a gigabyte, let alone 16 gigabytes that are found in machines today. I don't know if this is still a problem.)


Comments

Popular posts from this blog

plotting - How to draw lines between specified dots on ListPlot?

I would like to create a plot where I have unconnected dots and some connected. So far, I have figured out how to draw the dots. My code is the following: ListPlot[{{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4,13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full] I have thought using ListLinePlot command, but I don't know how to specify to the command to draw only selected lines between the dots. Do have any suggestions/hints on how to do that? Thank you. Answer One possibility would be to use Epilog with Line : ListPlot[ {{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4, 13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full, Epilog -> { Line[ ...

equation solving - Invert and fit implicitly defined curve

I need to fit an implicitly defined curve. I thought I could get some data out of Solve , and then using FindFit . Therefore, I would like to find the relation the parametric curve defined by $F(x,y)=0$: Solve[-(1/2) + 1/2 (0.41202 BesselK[0, 0.1 Sqrt[x^2 + y^2]] + (0.101483 x BesselK[1, 0.1 Sqrt[x^2 + y^2]])/Sqrt[x^2 + y^2]) == 0, y] But I can't get an output: Solve was unable to solve the system with inexact coefficients or the system obtained by direct rationalization of inexact numbers present in the system. Since many of the methods used by Solve require exact input, providing Solve with an exact version of the system may help. >> Edit: In particular, I would like to fit the data coming from the curve with the expression of another curve, and not with a function $f(x)$. In particular, since this clearly looks like a cardioid , I would like it to fit to something like it. What other strategies could I try?

dynamic - How can I make a clickable ArrayPlot that returns input?

I would like to create a dynamic ArrayPlot so that the rectangles, when clicked, provide the input. Can I use ArrayPlot for this? Or is there something else I should have to use? Answer ArrayPlot is much more than just a simple array like Grid : it represents a ranged 2D dataset, and its visualization can be finetuned by options like DataReversed and DataRange . These features make it quite complicated to reproduce the same layout and order with Grid . Here I offer AnnotatedArrayPlot which comes in handy when your dataset is more than just a flat 2D array. The dynamic interface allows highlighting individual cells and possibly interacting with them. AnnotatedArrayPlot works the same way as ArrayPlot and accepts the same options plus Enabled , HighlightCoordinates , HighlightStyle and HighlightElementFunction . data = {{Missing["HasSomeMoreData"], GrayLevel[ 1], {RGBColor[0, 1, 1], RGBColor[0, 0, 1], GrayLevel[1]}, RGBColor[0, 1, 0]}, {GrayLevel[0], GrayLevel...