Skip to main content

Automating Testing of Mathematica Code


I'm getting started with unit testing of Mathematica code and looking for information on fitting this into an automated testing environment.


For example with Node.js I write some code and tests and then hook it up with a testing pipeline. So when I check in my changes, Jenkins gets the changes, runs the tests and publishes the results (and possibly other things like build artifacts or build documentation, but this is the simplest case).


From the Mathematica documentation I'm having a hard time figuring out how to do two important steps here.


First, everything I see about testing in Mathematica talks about testing and seeing the results in a notebook. For automated testing by a build server the tests would have to be run without a notebook with just a command line interface.



Second, the test report would need to be in some type of format that could be consumed by Jenkins (or some other build server). This could be one of various different formats but I don't see how to export the test results.



Answer



I am not aware of something ready to use for fully automated testing of Mathematica code, but the building blocks for that are available in newer Mathematica versions.


The following is just a rough description of the steps necessary for such a setup. As I think it would be valuable to have a more detailed recipe for such a setup I made this a community wiki, so everybody is welcomed to fill in the details...


Here are the steps I think would be necessary:


Create testfiles


You can either convert testing-notebooks to *.wlt files (using the "More" -> "Save as .wlt" menu in the Testing-Notebook docked cells), write .wlt files directly or also write "old-style" .mt files for MUnit (for the latter you would also need to ensure the package is available). Here is a very minimal example of .wlt file content.


BeginTestSection["AutomatedTests"]
VerificationTest[Plus[1, 1],2]
EndTestSection[]


Create a "driver" script


Create a .m file or probably better a .wls script which runs the tests and prints or returns the outcome of the tests. A very simplified first step would be something like:


Print[TestReport["/path/to/testfile.wlt"]@"AllTestsSucceeded"]

Start driver script from command line


Ensure you can run the above file from the command line. Depending on your preferences and OS you might want to write a shell-script to do that. Using a .wls script and Exit[returncode] you should be able to do everything you need from within the wolfram script, though...


Configure a test automation tool


For the automation I would suggest to install the test automation tool of your choice and configure it to run the above script on the given events (e.g. a checkin or push to a version control system). You might need to adopt the driver script so that the tool can understand the outcome of it.


Comments

Post a Comment

Popular posts from this blog

plotting - How to draw lines between specified dots on ListPlot?

I would like to create a plot where I have unconnected dots and some connected. So far, I have figured out how to draw the dots. My code is the following: ListPlot[{{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4,13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full] I have thought using ListLinePlot command, but I don't know how to specify to the command to draw only selected lines between the dots. Do have any suggestions/hints on how to do that? Thank you. Answer One possibility would be to use Epilog with Line : ListPlot[ {{1, 1}, {2, 2}, {3, 3}, {4, 4}, {1, 4}, {2, 5}, {3, 6}, {4, 7}, {1, 7}, {2, 8}, {3, 9}, {4, 10}, {1, 10}, {2, 11}, {3, 12}, {4, 13}, {2.5, 7}}, Ticks -> {{1, 2, 3, 4}, None}, AxesStyle -> Thin, TicksStyle -> Directive[Black, Bold, 12], Mesh -> Full, Epilog -> { Line[ ...
Post a Comment
Read more

equation solving - Invert and fit implicitly defined curve

I need to fit an implicitly defined curve. I thought I could get some data out of Solve , and then using FindFit . Therefore, I would like to find the relation the parametric curve defined by $F(x,y)=0$: Solve[-(1/2) + 1/2 (0.41202 BesselK[0, 0.1 Sqrt[x^2 + y^2]] + (0.101483 x BesselK[1, 0.1 Sqrt[x^2 + y^2]])/Sqrt[x^2 + y^2]) == 0, y] But I can't get an output: Solve was unable to solve the system with inexact coefficients or the system obtained by direct rationalization of inexact numbers present in the system. Since many of the methods used by Solve require exact input, providing Solve with an exact version of the system may help. >> Edit: In particular, I would like to fit the data coming from the curve with the expression of another curve, and not with a function $f(x)$. In particular, since this clearly looks like a cardioid , I would like it to fit to something like it. What other strategies could I try?
Post a Comment
Read more

dynamic - How can I make a clickable ArrayPlot that returns input?

I would like to create a dynamic ArrayPlot so that the rectangles, when clicked, provide the input. Can I use ArrayPlot for this? Or is there something else I should have to use? Answer ArrayPlot is much more than just a simple array like Grid : it represents a ranged 2D dataset, and its visualization can be finetuned by options like DataReversed and DataRange . These features make it quite complicated to reproduce the same layout and order with Grid . Here I offer AnnotatedArrayPlot which comes in handy when your dataset is more than just a flat 2D array. The dynamic interface allows highlighting individual cells and possibly interacting with them. AnnotatedArrayPlot works the same way as ArrayPlot and accepts the same options plus Enabled , HighlightCoordinates , HighlightStyle and HighlightElementFunction . data = {{Missing["HasSomeMoreData"], GrayLevel[ 1], {RGBColor[0, 1, 1], RGBColor[0, 0, 1], GrayLevel[1]}, RGBColor[0, 1, 0]}, {GrayLevel[0], GrayLevel...
Post a Comment
Read more