What are recommended guidelines for developing packages?

This might be of general interest – I have different questions regarding naming conventions, contexts, subcontexts, shadowing, etc., but I do feel that they are closely related, thus I don't really want to split this post into three.

1. 
   

   Naming
   What is the best method to name the package file, the package context, the directory of package files (or a more complex hierarchy of these files)? Which of these names must be the same? I got confused several times before, and though I always manage to solve a situation, I don't feel like I have a good understanding on how these things work.

   
   


2. 
   
   

   Shadowing
   When there are interrelated packages with different contexts where some symbols appear in all of these contexts then - when calling contexts in the same session - usually shadowing messages appear. This is useful, when such symbols have different definitions and are unintentionally named the same way, but not, in the following case. If someone has a newly introduced function option, like Verbose, which doesn't have an OwnValue, then it is totally unnecessary to invoke shadowing messages, as no call of Verbose could do any harm. There still might be difference in the overall description of Verbose in two packages (even when all OwnValues, DownValues, etc. are the same), for example their usage messages might differ, as different functions would utilize the Verbose option in the different packages.

   
   

   What is the best way to deal with these things? Should a Common.m package be introduced, and all the related packages be moved under a common context-name and/or directory? Do they have to be in the same directory?

   
   


3. 
   

   Grouping and sub-contexts
   Following point 2, when is it useful to introduce sub-contexts (e.g. myContext`format` and myContext`content`)? Should these be split into different files? How should these files be named? Is it necessary then to include a Common.m too or is it just for convenience? What should be kept in Common.m?

   
   

Answer

Part of what you are asking is of course a matter of taste and habits, but here are my 2 cents:

1) if you want Mathematica to find your package files with a Needs or Get their context names must agree with the hierarchy of directories and filenames. I don't see any good reasons to diverge from that standard convention. For complex packages with many files you will also typically have a Kernel-subdirectory with an init.m, but I think these things are relatively well documented.

2) My personal opinion is that using symbols for option names is asking for exactly these kind of problems. Obviously at least some of the WRI personnel thinks the same, since in later versions there are more and more options that accept strings as names and the new way to work with options also full supports this. If you are worried about cluttering your code with too many pairs of "", note that this will work alright:

Options[f] = {"Verbose" -> False}

f[OptionsPattern[]] := (If[OptionValue["Verbose"], 
   Print["I'm so verbose!"]]; RandomReal[])

f[Verbose -> True]

or even:

f[someothercontext`Verbose -> True]

What you loose is the possibility to have a usage message bound to the option name, but as you have noticed if there are more than one function using the same option name, the usage message is of limited use anyway and the details must be explained in the documentation of the function, not the option. WRI has the same problem, obviously: At least I don't think that this usage is of very much help:

?Method

Method is an option for various algorithm-intensive functions that specifies what internal methods they should use.

3) Introducing sub-contexts is useful when things get more complex and parts of the whole can be split up in more or less independent parts. Of course giving these parts names that make it easy to recognize what they provide is a good idea, but I think that's so obvious that I doubt I fully understand that part of your question. If you want these parts to be loadable without the other parts, you must split them in different files, otherwise it's up to you from the technical viewpoint. From the code organization point of view I would think that if it makes sense to split your packages in separate contexts, it usually is also a good idea to split them into separate files. That becomes even more important if several people work on the various parts, but I feel there is not much Mathematica code written that way (except within WRI). Of course it's not necessary to include a Common.m, but as you have mentioned it's a good approach to collect all symbols that the various parts share into one common context/file, and Common.m (myPackage`Common`) is a common convention that is also used by WRI, so I'd stick with it. On the other hand I would consider it as a good design of your package when you don't need a Common.m, since then you obviously managed to really split your package in independent parts.