parallelization - Passing Package Dependencies to Subkernels

Say I have a function f[x_,...]:=... in a *.wl-Package form that depends on a bunch of Packages (some other *.wl files), and say I want to call this function f several times in parallel. I am currently using a set-up of the following form

LaunchKernels[2];
ParallelNeeds["Privatef`",".../f_file.wl"];


ParallelTable[
output[i]=f[i];
,{i,1,10}];

where i is just supposed to represent a set of parameters. The function f itself is of the form

BeginPackage["Privatef`"]

f

Begin["fenv`"]

Needs["other Packages"]

some code

End[]

EndPackage[]

Now, my problem is that the function f is distributed into the different subkernels I generated, though, the other Packages don't seem to load in each subkernel. I have tried to also use ParallelNeeds on these additional Packages, but without success. So how do I get Mathematica to load each additional Package that f requires in each subkernel that was launched?

Answer

Are the packages you need in the $Path?

If not, the following may be the simplest solution:

• 
  

  Create a package file (.m or .wl) which simply modifies the $Path.

  
  

  BeginPackage["PathLoader`"]
  $Path = Join[$Path, {"/path/one", "/path/two"}];
  EndPackage[]
  

  
  

  The only purpose of the BeginPackage/EndPackage here is to ensure that (Parallel)Needs won't complain about "context not created".

  
  
  


• 
  

  Put this in a standard location, e.g. $UserBaseDirectory/Applications/PathLoader.m.

  
  


• 
  

  Start your session with

  
  

  Needs["PathLoader`"]
  ParallelNeeds["PathLoader`"]
  

Now $Path will contain all your directories both on the main kernel and on the subkernels.

I use similar (but more complicated) workarounds for Kernel/init.m not being loaded on any subkernels.