Skip to main content

performance tuning - A Simplistic C# (.NET) Reloader


Can one write a very straightforward C# reloader akin to this Java reloader, that enables Mathematica-like usage of functions from existing C# code in string format, in an easy and elegant fashion directly in Mathematica?



Answer



Yes it is possible to write such a code and I will present the code and sample usage below. The motivation for this question/answer is this Java reloader by Leonid Shifrin and I borrowed some code from his answer.



Simplistic C# Reloader


The code below is a C# reloader. It takes a string of C# code and (optionally) a C# compiler path, attempts a compile and if the compilation succeeds it calls on NETLink to load the DLL. It also takes the same Options as LoadNETType, making it easy to choose how you want the loaded functions to appear.


Code


BeginPackage["DotNETReloader`", {"NETLink`"}];

dotNETLoader::usage =
"dotNETLoader[code_, compilerPath_] takes a C# code in string format
and (optionally) a C# compiler path, it then attempts to compile the
string code into a .NET dll suitable for use with NETLink"


dotNETLoader::dirmakeerr = "Can not create directory `1`";
dotNETLoader::invst = "The loader is not in a valid state. Perhaps some temporary
directories do not exist or could not be created";
dotNETLoader::cmperr = "The following compilation errors were encountered: \n`1`";

Options[dotNETLoader] = Options[LoadNETType]

Begin["`Private`"]

$stateValidQ = True;

$csc = If[$SystemID === "Windows-x86-64", "csc.exe", "mcs"];
$err = If[$SystemID === "Windows-x86-64", 2, 3];
$tempDirectory = FileNameJoin[{$UserBaseDirectory, "Temp", "dotNET"}];

(* If a namespace exists, get the name *)
getNamespace[code_String] :=
With[{ns =
StringCases[code, "namespace" ~~ Whitespace ~~ name : (WordCharacter ..) :> name]},
First @ ns /; ns =!= {}]


(* Get the class name *)
getClass[classCode_String] :=
With[{cl =
StringCases[classCode,
"public" ~~ Whitespace | (Whitespace ~~ "static" ~~ Whitespace) ~~
"class" | "struct" | "interface" ~~ Whitespace ~~
name : (WordCharacter ..) :> name]}, First @ cl /; cl =!= {}]

getClass[__] := Throw[$Failed, error[getClass]];


If[!FileExistsQ[#] && CreateDirectory[#] === $Failed,
Message[dotNETLoader::dirmakeerr, #]; $stateValidQ = False;] &@$tempDirectory

dotNETLoader[code_String, compiler_String: $csc,
opts : OptionsPattern[]] /; $stateValidQ :=
Module[{sourcePath, run,
options = FilterRules[{opts}, Options[dotNETLoader]],
namespace = getNamespace[code], className = getClass[code]},
sourcePath = FileNameJoin[{$tempDirectory, className <> ".cs"}];
Export[sourcePath, code, "String"];

run = RunProcess[{compiler, "-t:library", "-optimize+", sourcePath},
ProcessDirectory -> $tempDirectory];
If[run[[1]] =!= 0, Message[dotNETLoader::cmperr,
Style[#, Red]& @ run[[$err]]]; $Failed,
ReinstallNET[];
LoadNETAssembly @ FileNameJoin[{$tempDirectory, className <> ".dll"}];
If[Head @ namespace =!= getNamespace,
LoadNETType[namespace <> "." <> className, options],
LoadNETType[className, options]]
]

]

dotNETLoader[_String] := CompoundExpression[Message[dotNETLoader::invst]; $Failed]

End[]

EndPackage[]

Notes


I have tested this code successfully on both Windows and Linux and would appreciate it if someone that has a Mac can also test it on MacOS (I won't be able to do this but I expect it to work unaltered since the .NET platform and C# compiler are available via Mono). Also, please I will like others to test this code and provide feedback on all platforms. You will notice that I have named the function dotNETLoader instead of cSharpLoader, this is because I'm planning on extending this function to include F# (which I have successfully tested on Windows) and possibly, other .NET languages.



Due to my usage of RunProcess to avoid bringing up the console in Windows, this will only work on Mathematica 10 and above versions.


The package works by saving the C# string code and saving it as a .cs file in a temporary directory and using the New in 10 function RunProcess to invoke your installed C# compiler to compile the class into a .NET library .dll file. It then calls NETLink to load the library and make the functions available for use in Mathematica.


Usage


I will show some examples of how to use this package. Let's start by borrowing some C# codes from the web and saving them as strings.


code1 = 

"public class LCS{
public static string longestCommonSubstring(string s1, string s2)
{
int Start = 0;

int Max = 0;
for (int i = 0; i < s1.Length; i++){
for (int j = 0; j < s2.Length; j++){
int x = 0;
while (s1[i + x] == s2[j + x]){
x++;
if (((i + x) >= s1.Length || ((j + x) >= s2.Length)))
break;
}
if (x > Max) {

Max = x;
Start = i;
}
}
}
return s1.Substring(Start, Max);
}
}";

This code is the C# equivalent of the 2nd code from here (There are only a few differences but it took less than a minute to make the changes to C# code)



First we load the package:


Needs["DotNETReloader`"]

Now we load the function using dotNETLoader:


dotNETLoader[code1, StaticsVisible -> True]


NETType["LCS", 1]



We can see the possible options:



Options[dotNETLoader]


{StaticsVisible -> False, AllowShortContext -> True}



Setting StaticsVisible to True enables us to call the function directly without the class name and makes it feel like a built-in function.


longestCommonSubstring["AAABBBBCCCCC","CCCBBBAAABABA"]


"AAAB"




I'll add the same benchmark Leonid used in his Java reloader just to show that in some cases it may be better to use C#:


s1 = StringJoin@RandomChoice[{"A", "C", "T", "G"}, 10000];
s2 = StringJoin@RandomChoice[{"A", "C", "T", "G"}, 10000];

Here we use the Mathematica's built-in function LongestCommonSubsequence:


LongestCommonSubsequence[s1, s2] // RepeatedTiming    


{0.25, "GTCCCTCACACTG"}




And here is our loaded C# function:


longestCommonSubstring[s1, s2] // RepeatedTiming


{0.35, "GTCCCTCACACTG"}



This collatz optimization question is interesting, so I decided to see how our new C# reloader can help us here. I found this code online a while ago (can't remember where) and made just one simple change to parallelize it.


code2 = 


"using System.Threading.Tasks;

namespace Tester
{
public struct Calc
{
public static int[] ParallelCollatz(int[] list)
{
Parallel.For(0, list.Length, i =>
{

int count;
long k = list[i];
for (count = 1; k != 1; count++)
{
k = ((k & 1) == 0) ? k / 2 : 3 * k + 1;
}
list[i] = count;
});

return list;

}
}

}";

As before (load the package if not already loaded), we load the function using dotNETLoader


dotNETLoader[code2, StaticsVisible -> True]


NETType["Tester.Calc", 2]




I wanted to point out a few things about the package with this example. First, notice that the type is Tester.Calc instead of just Calc, this is becaude this class was placed in a namespace, but the reloader notices this and takes care of it for us. Secondly, the way we use the feature in conjuction with built-in Mathematica functions to solve the problem shows how natural the approach becomes. Finally, this will also work for structs as well as static classes. Back to the example:


int = Range[1*^6];   

(* Here ParallelCollatz is the loaded C# function *)

RepeatedTiming @ Ordering[ParallelCollatz @ int, -1]


{0.0610, {837799}}




I will compare this to the compiled-to-C function provided by DumpsterDoofus.


Ordering[collatzLength @ int, -1] // RepeatedTiming


{0.877, {837799}}



Summary


We see that this C# Reloader can be quite useful and fits naturally into the Mathematica-style usage scenario. In the first example the highly optimized built in function was only slightly faster than our borrowed-from-web code. What's interesting from my testing so far is that there's not much performance difference when using these functions on both Windows and Linux. For the second example where I compared brute force approaches, we see that the loaded function was over a magnitude faster.


Comments

Popular posts from this blog

plotting - Filling between two spheres in SphericalPlot3D

Manipulate[ SphericalPlot3D[{1, 2 - n}, {θ, 0, Pi}, {ϕ, 0, 1.5 Pi}, Mesh -> None, PlotPoints -> 15, PlotRange -> {-2.2, 2.2}], {n, 0, 1}] I cant' seem to be able to make a filling between two spheres. I've already tried the obvious Filling -> {1 -> {2}} but Mathematica doesn't seem to like that option. Is there any easy way around this or ... Answer There is no built-in filling in SphericalPlot3D . One option is to use ParametricPlot3D to draw the surfaces between the two shells: Manipulate[ Show[SphericalPlot3D[{1, 2 - n}, {θ, 0, Pi}, {ϕ, 0, 1.5 Pi}, PlotPoints -> 15, PlotRange -> {-2.2, 2.2}], ParametricPlot3D[{ r {Sin[t] Cos[1.5 Pi], Sin[t] Sin[1.5 Pi], Cos[t]}, r {Sin[t] Cos[0 Pi], Sin[t] Sin[0 Pi], Cos[t]}}, {r, 1, 2 - n}, {t, 0, Pi}, PlotStyle -> Yellow, Mesh -> {2, 15}]], {n, 0, 1}]

plotting - Plot 4D data with color as 4th dimension

I have a list of 4D data (x position, y position, amplitude, wavelength). I want to plot x, y, and amplitude on a 3D plot and have the color of the points correspond to the wavelength. I have seen many examples using functions to define color but my wavelength cannot be expressed by an analytic function. Is there a simple way to do this? Answer Here a another possible way to visualize 4D data: data = Flatten[Table[{x, y, x^2 + y^2, Sin[x - y]}, {x, -Pi, Pi,Pi/10}, {y,-Pi,Pi, Pi/10}], 1]; You can use the function Point along with VertexColors . Now the points are places using the first three elements and the color is determined by the fourth. In this case I used Hue, but you can use whatever you prefer. Graphics3D[ Point[data[[All, 1 ;; 3]], VertexColors -> Hue /@ data[[All, 4]]], Axes -> True, BoxRatios -> {1, 1, 1/GoldenRatio}]

plotting - Mathematica: 3D plot based on combined 2D graphs

I have several sigmoidal fits to 3 different datasets, with mean fit predictions plus the 95% confidence limits (not symmetrical around the mean) and the actual data. I would now like to show these different 2D plots projected in 3D as in but then using proper perspective. In the link here they give some solutions to combine the plots using isometric perspective, but I would like to use proper 3 point perspective. Any thoughts? Also any way to show the mean points per time point for each series plus or minus the standard error on the mean would be cool too, either using points+vertical bars, or using spheres plus tubes. Below are some test data and the fit function I am using. Note that I am working on a logit(proportion) scale and that the final vertical scale is Log10(percentage). (* some test data *) data = Table[Null, {i, 4}]; data[[1]] = {{1, -5.8}, {2, -5.4}, {3, -0.8}, {4, -0.2}, {5, 4.6}, {1, -6.4}, {2, -5.6}, {3, -0.7}, {4, 0.04}, {5, 1.0}, {1, -6.8}, {2, -4.7}, {3, -1.