programming - How does Needs["foo`"] find the file that defines context "foo`"?

The post's title says it all, but if more detail is needed:

Suppose I have the following in some file /path/to/foo.m:

Begin["foo`"];
hello = Function[{}, Print["Hello world!"]];
End[];

...and in some other file, say, ./client.m, I put the following

Needs["foo`"];
foo`hello[];

Then, when the Needs expression gets evaluated, I get the error

Needs::nocont: Context foo` was not created when Needs was evaluated.

I'm too surprised by this error, since I see no good way for Mathematica to find where foo` is defined.

(BTW, putting /path/to in $Path does not change the results described above.)

What else must I put in ./client.m, besides Needs["foo`"], so that the expression foo`hello[] evaluates properly?

---

EDIT: I should explain that the reason I'm using Begin instead of BeginPackage is that I want to enforce full qualified names (e.g. foo`hello, rather than plain hello) as the only way to refer to imported functions.

Answer

This post answers specifically the title question:

How does Needs["foo`"] find the file that defines context "foo`"?

---

Get, Needs, Install, OpenRead, etc. all use FindFile. How FindFile resolves file names is discussed in:

---

I don't know the full details (it's complicated), but roughly FindFile translates a context to a file path as follows:

FindFile["foo`"] either

• Looks on $Path for foo.m, foo.wl, foo.mx or directory foo


• If a paclet declares the context foo` in its Kernel extension, it translates to the corresponding Root directory: See PacletInfo.m documentation project, Kernel extension section.

Then if the result was a directory dir, it continues to look for

• dir/init.m (or init.wl)


• dir/Kernel/init.m (or init.wl)

If the result was a directory dir.mx, it continues to look for

• dir.mx/$SystemID/dir.mx (insert the value of $SystemID)

This is useful because .mx files are not compatible across different platforms.

---

FindFile["foo"], where foo is not a context,

• Looks on $Path for foo.

If the result is a directory, then it continues to look for

• foo/$SystemID/foo

This is useful with Install, when we need a separate executable for each $SystemID, but we want to be able to use the same name to refer to them on any platform.

---

Notes

The context of a package, i.e. the context given in BeginPackage that will contain the public package symbols, does not play any role in how FindFile resolves a context name to a file path.

However, Needs is different from Get in that it expects the context passed to it to appear in $Packages after the package has been loaded. This is why one needs to use BeginPackage and not merely Begin when writing a package. BeginPackage will permanently add that context both to $ContextPath and $Packages. Needs uses $Packages to determine if a package has already been loaded and avoid double-loading.

In short, the file names and the context of a package don't strictly need to be the same. But if they aren't, Mathematica will sometimes get confused.

The rules described above explain why the standard application directory structure is as described here. There is nothing strictly enforcing this particular structure, but the system is designed in a way that it expects to find this structure in any package.

---

Some interesting undocumented variables:

• 
  

  Internal`$PackageDependencies is updated by BeginPackage based on its second argument

  
  


• 
  

  $LoadedFiles is updated by Get. To get a more useful list, use Select[$LoadedFiles, Not@StringStartsQ[#, $InstallationDirectory] &]