mathlink or wstp - Portable makefile for wscc builds

What's the canonical way to write a portable makefile for wscc programs? The makefile typically requires the location of the wscc script and wstp.h header file.

I see in this example they are hardcoded:

WSTPLINKDIR = /usr/local/Wolfram/Mathematica/10.0/SystemFiles/Links/WSTP/DeveloperKit
SYS = Linux  # Set this value with the result of evaluating $SystemID
CADDSDIR = ${WSTPLINKDIR}/${SYS}/CompilerAdditions

Can we do something better? For ordinary linux packages, I'd use e.g.,pkg-config. Do I just have to tell anyone who wants to build my code to search for the correct directories (possibly with locate) and hack the makefile themselves?

Is there a way within a Mathematica shell to get the location of them reliably? Even GetEnvironment[{"MATHEMATICA_HOME"}] isn't enough.

Answer

There is no platform-independent way to detect the location of Mathematica from a terminal shell or Makefile. You also have to decide which version to use when multiple versions are installed (a fairly common situation). For a few platform-dependent attempts to detect locations, take a look at the makefiles we used for MATLink. None of these ways are robust though.

However, if you are creating a Mathematica package with some binary components, and all the C source code is contained within the package (without hard-to-control external dependencies, such as MATLAB was for MATLink), then you can compile from within Mathematica. The installation directory is pointed to by $InstallationDirectory. All the relevant paths (such as the MathLink libraries) are at a fixed location relative to that.

The CCompilerDriver package seems to have much of this automated. It takes care not only of the location of MathLink libraries, but also most platform-specific or compiler-specific details (such as different syntax for gcc and MSVC). See the "WSTP Executables" section in its documentation. Note that you must use MathLink instead of WSTP if you are working with CCompilerDriver.

A nice solution is to set up your package to auto-compile its binary components on first load. If you do this, keep in mind that you can only count on your users having a C compiler installed on Linux. On OS X and Windows it is better to ship pre-compiled executables.

---

Skeleton project that auto-compiles its MathLink component

The following is a simple proof-of-concept of a package that auto-compiles its MathLink dependency using CreateExecutable. For simple projects, it is sufficient to use CreateExecutable, and you do not need a Makefile. The advantage is that it will take care of most OS-specific or compiler-specific details, so you will not need to have separate makefiles for different operating systems / compilers.

To set up the project, create the following directory structure

AddTwo
  |
  |- Kernel
  |    |
  |    \- init.m

  |
  |- AddTwo.m
  |
  |- Sources
  |     |
  |     |- addtwo.tm
  |     |
  |     \- addtwo.c
  |
  \- addtwoprog

        |
        \

Here addtwoprog must be an empty directory. This is where the MathLink executables will be saved.

The parent of the top-level AddTwo directory (not AddTwo itself!) must be in the $Path.

addtwo.tm and addtwo.c must be copied from the location opened by

SystemOpen@
 FileNameJoin[{$InstallationDirectory, "SystemFiles", "Links", 
   "MathLink", "DeveloperKit", $SystemID, "MathLinkExamples"}]

If this location contains addtwo.cxx instead of addtwo.c, simply rename it to addtwo.c after copying.

This is the contents of the other files:

(* init.m *)
Get["AddTwo`AddTwo`"]

(* AddTwo.m *)
BeginPackage["AddTwo`"]

(* CCompilerDriver` is not included in BeginPackage to avoid keeping it in the $ContextPath *)
Needs["CCompilerDriver`"] 


(* Must mention AddTwo symbol from addtwo.tm here 
   to ensure it is created in the correct context. *)
AddTwo::usage = "AddTwo[m, n] returns the sum of integers m and n.";

Begin["`Private`"]

$packageDir = DirectoryName[$InputFileName];
$executable = FileNameJoin[{$packageDir, "addtwoprog"}]; (* See Install[] documentation, Details section, for how it resolves directories into executables *)


(* This is the list of source files to be compiled. There may be multiple .c and .tm files, 
   all of which will be linked together. See the CreateExecutable[] documentation, 
   Details section, on how to include additional object files. See CreateObjectFile[] 
   on how to create an object file without also linking it into an executable. *)
$sourceFiles = FileNameJoin[{$packageDir, "Sources", #}]& /@ {"addtwo.tm", "addtwo.c"};

(* Just a colourful Print alternative *)
print[args___] := Print @@ (Style[#, Red]&) /@ {args}

If[

    Install[$executable] === $Failed
    ,
    print["Compiling AddTwo executable..."];
    CreateExecutable[$sourceFiles, "addtwoprog", 
     "TargetDirectory" -> FileNameJoin[{$executable, $SystemID}],

     (* the following two options are here to ease debugging *)
     "ShellCommandFunction" -> Print, (* see compilation command *)
     "ShellOutputFunction" -> Print (* see compiler messages *)
    ];

    If[
        Install[$executable] === $Failed
        ,
        print["Failed to compile or run AddTwo executable. \[SadSmiley]"],
        print["Compiling successful! \[HappySmiley]"]
    ]
]

End[] (* `Private` *)


EndPackage[]

This package will auto-compile its C component when it is loaded for the first time.

Try it with

<
?AddTwo

AddTwo[3,4]

The compiled binary will be stored in the package directory. On a second load of the package, it won't be re-compiled again. The best approach is probably to distribute these binaries to your users. This is especially important on OS X and Windows where you can't count on a compiler being already set up. Luckily, on these platforms it is easier to ensure that your binaries will work for everyone (e.g. see here for OS X).

This skeleton package was tested on OS X / M11.1 and Linux / Raspberry Pi / M11.0.