Skip to main content

Automating Testing of Mathematica Code


I'm getting started with unit testing of Mathematica code and looking for information on fitting this into an automated testing environment.


For example with Node.js I write some code and tests and then hook it up with a testing pipeline. So when I check in my changes, Jenkins gets the changes, runs the tests and publishes the results (and possibly other things like build artifacts or build documentation, but this is the simplest case).


From the Mathematica documentation I'm having a hard time figuring out how to do two important steps here.


First, everything I see about testing in Mathematica talks about testing and seeing the results in a notebook. For automated testing by a build server the tests would have to be run without a notebook with just a command line interface.



Second, the test report would need to be in some type of format that could be consumed by Jenkins (or some other build server). This could be one of various different formats but I don't see how to export the test results.



Answer



I am not aware of something ready to use for fully automated testing of Mathematica code, but the building blocks for that are available in newer Mathematica versions.


The following is just a rough description of the steps necessary for such a setup. As I think it would be valuable to have a more detailed recipe for such a setup I made this a community wiki, so everybody is welcomed to fill in the details...


Here are the steps I think would be necessary:


Create testfiles


You can either convert testing-notebooks to *.wlt files (using the "More" -> "Save as .wlt" menu in the Testing-Notebook docked cells), write .wlt files directly or also write "old-style" .mt files for MUnit (for the latter you would also need to ensure the package is available). Here is a very minimal example of .wlt file content.


BeginTestSection["AutomatedTests"]
VerificationTest[Plus[1, 1],2]
EndTestSection[]


Create a "driver" script


Create a .m file or probably better a .wls script which runs the tests and prints or returns the outcome of the tests. A very simplified first step would be something like:


Print[TestReport["/path/to/testfile.wlt"]@"AllTestsSucceeded"]

Start driver script from command line


Ensure you can run the above file from the command line. Depending on your preferences and OS you might want to write a shell-script to do that. Using a .wls script and Exit[returncode] you should be able to do everything you need from within the wolfram script, though...


Configure a test automation tool


For the automation I would suggest to install the test automation tool of your choice and configure it to run the above script on the given events (e.g. a checkin or push to a version control system). You might need to adopt the driver script so that the tool can understand the outcome of it.


Comments

Post a Comment

Popular posts from this blog

functions - Get leading series expansion term?

Given a function f[x] , I would like to have a function leadingSeries that returns just the leading term in the series around x=0 . For example: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x)] x and leadingSeries[(1/x + 2 + (1 - 1/x^3)/4)/(4 + x)] -(1/(16 x^3)) Is there such a function in Mathematica? Or maybe one can implement it efficiently? EDIT I finally went with the following implementation, based on Carl Woll 's answer: lds[ex_,x_]:=( (ex/.x->(x+O[x]^2))/.SeriesData[U_,Z_,L_List,Mi_,Ma_,De_]:>SeriesData[U,Z,{L[[1]]},Mi,Mi+1,De]//Quiet//Normal) The advantage is, that this one also properly works with functions whose leading term is a constant: lds[Exp[x],x] 1 Answer Update 1 Updated to eliminate SeriesData and to not return additional terms Perhaps you could use: leadingSeries[expr_, x_] := Normal[expr /. x->(x+O[x]^2) /. a_List :> Take[a, 1]] Then for your examples: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x), x] leadingSeries[Exp[x], x] leadingSeries[(1/x + 2 + (1 - 1/x...
Post a Comment
Read more

How to thread a list

I have data in format data = {{a1, a2}, {b1, b2}, {c1, c2}, {d1, d2}} Tableform: I want to thread it to : tdata = {{{a1, b1}, {a2, b2}}, {{a1, c1}, {a2, c2}}, {{a1, d1}, {a2, d2}}} Tableform: And I would like to do better then pseudofunction[n_] := Transpose[{data2[[1]], data2[[n]]}]; SetAttributes[pseudofunction, Listable]; Range[2, 4] // pseudofunction Here is my benchmark data, where data3 is normal sample of real data. data3 = Drop[ExcelWorkBook[[Column1 ;; Column4]], None, 1]; data2 = {a #, b #, c #, d #} & /@ Range[1, 10^5]; data = RandomReal[{0, 1}, {10^6, 4}]; Here is my benchmark code kptnw[list_] := Transpose[{Table[First@#, {Length@# - 1}], Rest@#}, {3, 1, 2}] &@list kptnw2[list_] := Transpose[{ConstantArray[First@#, Length@# - 1], Rest@#}, {3, 1, 2}] &@list OleksandrR[list_] := Flatten[Outer[List, List@First[list], Rest[list], 1], {{2}, {1, 4}}] paradox2[list_] := Partition[Riffle[list[[1]], #], 2] & /@ Drop[list, 1] RM[list_] := FoldList[Transpose[{First@li...
Post a Comment
Read more

front end - keyboard shortcut to invoke Insert new matrix

I frequently need to type in some matrices, and the menu command Insert > Table/Matrix > New... allows matrices with lines drawn between columns and rows, which is very helpful. I would like to make a keyboard shortcut for it, but cannot find the relevant frontend token command (4209405) for it. Since the FullForm[] and InputForm[] of matrices with lines drawn between rows and columns is the same as those without lines, it's hard to do this via 3rd party system-wide text expanders (e.g. autohotkey or atext on mac). How does one assign a keyboard shortcut for the menu item Insert > Table/Matrix > New... , preferably using only mathematica? Thanks! Answer In the MenuSetup.tr (for linux located in the $InstallationDirectory/SystemFiles/FrontEnd/TextResources/X/ directory), I changed the line MenuItem["&New...", "CreateGridBoxDialog"] to read MenuItem["&New...", "CreateGridBoxDialog", MenuKey["m", Modifiers-...
Post a Comment
Read more