Skip to main content

Best practices for using the Testing Framework


I am looking for best practices and recommended workflows for using the Testing Framework. For example, lets say I am developing a Mathematica package for other users, it is hosted on GitHub and I wish to make it easy for other developers to contribute to repository.


The project includes a suite of (unit) tests that I would like to run as often as necessary. I would like to run them within FrontEnd and conveniently inspect the results (timings, which tests failed and why, etc), save them to file and also further automate testing in the future. I can think of few possible workflows but none of them ticks all the boxes.




  • "Testing Notebook", Pro: It gives a clear overview of tests and looks nice. It can be run in FrontEnd and also programmatically with TestReport. Con: Contains even more metadata than the normal notebook (all those buttons) so it is less suitable to put under version control. Somehow the evaluation of tests is slower in testing notebook than in normal notebook.





  • ".wlt file", Pro: This is the documented way to use plain text files with test and run them programmatically from another notebook or command line. Con: I can't open .wlt files in FrontEnd!? So I have to open them in other text editor without nice features of FrontEnd (autocomplete, code coloring, etc).




  • ".wl file", Pro: I can easily edit it in FrontEnd as any other package. Con: This workflow is not documented. Should I put all tests in some specific Context?




So the question is, what is your recommended workflow for testing? While browsing other people's GitHub repositories I have seen some custom made approaches, but I would like rely on something that is documented and easy to explain to other developers who will contribute to the project.


I am also curious what is the purpose of BeginTestSection and EndTestSection in .wlt files?




Answer



I think @Jason B.'s answer is a good workflow. Here is an alternative that I have adopted over the years. Most of my code I write in *.m / *.mt / *.wl / *.wlt files using IntelliJ Idea with the Mathematica Plugin, so version control is straightforward.


My projects are written as paclets (see How to distribute Mathematica packages as paclets? for an introduction). Paclets allow you to specify resource files in the PacletInfo.m file. For every (major) function, I write a test-file .wlt or .mt and add it to the Paclet. Here is an example from my Multiplets paclet:


Paclet[
  Name -> "Multiplets",
  Version -> "0.1",
  MathematicaVersion -> "11.0+",
  Description -> "Defines multiplets and Young tableaux.",
  Thumbnail -> "multiplets.png",
  Creator -> "Johannes E. M. Mosig",

  Extensions -> {
    {"Kernel", Root -> ".", Context -> "Multiplets`"},
    {"Resource", Root -> "Testing", Resources -> {
        "MultipletDimension.mt",
        "MultipletQ.mt",
        "MultipletReduce.mt",
        "Tableau.mt",
        "TableauAppend.mt",
        "TableauClear.mt",
        "TableauDimension.mt",

        "TableauFirst.mt",
        "TableauFromMultiplet.mt",
        "TableauQ.mt",
        "TableauRest.mt",
        "TableauSimplify.mt",
        "TableauToMatrix.mt",
        "TableauToMultiplet.mt"
      }
    }
  }

]

In Mathematica, I can then load my paclet


<< Multiplets`

and check if any given function works as expected


TestReport@PacletResource["Multiplets", "MultipletReduce.mt"]

I can also open the test file within Mathematica, using


NotebookOpen@PacletResource["Multiplets", "MultipletReduce.mt"]


The front end opens *.mt and *.wlt files as plain text files, however, so if the notebook interface is important to you, then you may want to safe them as *.wl files instead.


In addition, one may add a test-script that just calls all other test scripts, for convenience.


Comments

Post a Comment

Popular posts from this blog

functions - Get leading series expansion term?

Given a function f[x] , I would like to have a function leadingSeries that returns just the leading term in the series around x=0 . For example: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x)] x and leadingSeries[(1/x + 2 + (1 - 1/x^3)/4)/(4 + x)] -(1/(16 x^3)) Is there such a function in Mathematica? Or maybe one can implement it efficiently? EDIT I finally went with the following implementation, based on Carl Woll 's answer: lds[ex_,x_]:=( (ex/.x->(x+O[x]^2))/.SeriesData[U_,Z_,L_List,Mi_,Ma_,De_]:>SeriesData[U,Z,{L[[1]]},Mi,Mi+1,De]//Quiet//Normal) The advantage is, that this one also properly works with functions whose leading term is a constant: lds[Exp[x],x] 1 Answer Update 1 Updated to eliminate SeriesData and to not return additional terms Perhaps you could use: leadingSeries[expr_, x_] := Normal[expr /. x->(x+O[x]^2) /. a_List :> Take[a, 1]] Then for your examples: leadingSeries[(1/x + 2)/(4 + 1/x^2 + x), x] leadingSeries[Exp[x], x] leadingSeries[(1/x + 2 + (1 - 1/x...
Post a Comment
Read more

How to thread a list

I have data in format data = {{a1, a2}, {b1, b2}, {c1, c2}, {d1, d2}} Tableform: I want to thread it to : tdata = {{{a1, b1}, {a2, b2}}, {{a1, c1}, {a2, c2}}, {{a1, d1}, {a2, d2}}} Tableform: And I would like to do better then pseudofunction[n_] := Transpose[{data2[[1]], data2[[n]]}]; SetAttributes[pseudofunction, Listable]; Range[2, 4] // pseudofunction Here is my benchmark data, where data3 is normal sample of real data. data3 = Drop[ExcelWorkBook[[Column1 ;; Column4]], None, 1]; data2 = {a #, b #, c #, d #} & /@ Range[1, 10^5]; data = RandomReal[{0, 1}, {10^6, 4}]; Here is my benchmark code kptnw[list_] := Transpose[{Table[First@#, {Length@# - 1}], Rest@#}, {3, 1, 2}] &@list kptnw2[list_] := Transpose[{ConstantArray[First@#, Length@# - 1], Rest@#}, {3, 1, 2}] &@list OleksandrR[list_] := Flatten[Outer[List, List@First[list], Rest[list], 1], {{2}, {1, 4}}] paradox2[list_] := Partition[Riffle[list[[1]], #], 2] & /@ Drop[list, 1] RM[list_] := FoldList[Transpose[{First@li...
Post a Comment
Read more

front end - keyboard shortcut to invoke Insert new matrix

I frequently need to type in some matrices, and the menu command Insert > Table/Matrix > New... allows matrices with lines drawn between columns and rows, which is very helpful. I would like to make a keyboard shortcut for it, but cannot find the relevant frontend token command (4209405) for it. Since the FullForm[] and InputForm[] of matrices with lines drawn between rows and columns is the same as those without lines, it's hard to do this via 3rd party system-wide text expanders (e.g. autohotkey or atext on mac). How does one assign a keyboard shortcut for the menu item Insert > Table/Matrix > New... , preferably using only mathematica? Thanks! Answer In the MenuSetup.tr (for linux located in the $InstallationDirectory/SystemFiles/FrontEnd/TextResources/X/ directory), I changed the line MenuItem["&New...", "CreateGridBoxDialog"] to read MenuItem["&New...", "CreateGridBoxDialog", MenuKey["m", Modifiers-...
Post a Comment
Read more