5
$\begingroup$

The post's title says it all, but if more detail is needed:

Suppose I have the following in some file /path/to/foo.m:

Begin["foo`"]; hello = Function[{}, Print["Hello world!"]]; End[];

...and in some other file, say, ./client.m, I put the following

Needs["foo`"]; foo`hello[];

Then, when the Needs expression gets evaluated, I get the error

Needs::nocont: Context foo` was not created when Needs was evaluated.

I'm too surprised by this error, since I see no good way for Mathematica to find where foo` is defined.

(BTW, putting /path/to in $Path does not change the results described above.)

What else must I put in ./client.m, besides Needs["foo`"], so that the expression foo`hello[] evaluates properly?

EDIT: I should explain that the reason I'm using Begin instead of BeginPackage is that I want to enforce full qualified names (e.g. foo`hello, rather than plain hello) as the only way to refer to imported functions.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Needs calls Get which "by default successively searches for files in the directories specified by the elements of $Path." $\endgroup$ Commented Dec 10, 2016 at 19:53
  • $\begingroup$ @Quantum_Oli: please, see the clarification I added to my post (starting with "(BTW, putting...)"). $\endgroup$ Commented Dec 10, 2016 at 19:56
  • $\begingroup$ @Quantum_Oli: in any case, it stands to reason that putting a path in $Path can't possibly be the answer to my question, since there is no relationship between names of files and the names of contexts defined therein. IOW, a file named foo.m need not define a context foo` , and, conversely, a file named something other than foo.m, can define a context named foo` . Since there's no correspondence whatsoever between context names and file names, I see no way in which the content of $Path could be used to find the where a context is defined. $\endgroup$ Commented Dec 10, 2016 at 20:02
  • 1
    $\begingroup$ I added some extra notes to the answer to explain the message you get. In short: Get just loads a file and doesn't care what is in it. Needs should be used to load a package that follows the standard structure. It uses Get internally, but it also does extra checks and avoids double loading. $\endgroup$ Commented Dec 11, 2016 at 11:09

3 Answers 3

Reset to default
9
$\begingroup$

This post answers specifically the title question:

How does Needs["foo`"] find the file that defines context "foo`"?

Get, Needs, Install, OpenRead, etc. all use FindFile. How FindFile resolves file names is discussed in:

I don't know the full details (it's complicated), but roughly FindFile translates a context to a file path as follows:

FindFile["foo`"] either

  • Looks on $Path for foo.m, foo.wl, foo.mx or directory foo
  • If a paclet declares the context foo` in its Kernel extension, it translates to the corresponding Root directory: See PacletInfo.m documentation project, Kernel extension section.

Then if the result was a directory dir, it continues to look for

  • dir/init.m (or init.wl)
  • dir/Kernel/init.m (or init.wl)

If the result was a directory dir.mx, it continues to look for

  • dir.mx/$SystemID/dir.mx (insert the value of $SystemID)

This is useful because .mx files are not compatible across different platforms.

FindFile["foo"], where foo is not a context,

  • Looks on $Path for foo.

If the result is a directory, then it continues to look for

  • foo/$SystemID/foo

This is useful with Install, when we need a separate executable for each $SystemID, but we want to be able to use the same name to refer to them on any platform.

Notes

The context of a package, i.e. the context given in BeginPackage that will contain the public package symbols, does not play any role in how FindFile resolves a context name to a file path.

However, Needs is different from Get in that it expects the context passed to it to appear in $Packages after the package has been loaded. This is why one needs to use BeginPackage and not merely Begin when writing a package. BeginPackage will permanently add that context both to $ContextPath and $Packages. Needs uses $Packages to determine if a package has already been loaded and avoid double-loading.

In short, the file names and the context of a package don't strictly need to be the same. But if they aren't, Mathematica will sometimes get confused.

The rules described above explain why the standard application directory structure is as described here. There is nothing strictly enforcing this particular structure, but the system is designed in a way that it expects to find this structure in any package.

Some interesting undocumented variables:

  • Internal`$PackageDependencies is updated by BeginPackage based on its second argument

  • $LoadedFiles is updated by Get. To get a more useful list, use Select[$LoadedFiles, Not@StringStartsQ[#, $InstallationDirectory] &]

Improve this answer
$\endgroup$
1
  • $\begingroup$ Feel free to edit this post to add or correct details. $\endgroup$ Commented Dec 11, 2016 at 10:42
7
$\begingroup$

The problem is not that Mathematica can not find the file, which it would indicate with another message. The problem is that Needs insists in your package to create the context it loads and put it onto $Packages.

To achieve that you need BeginPackage["foo`"] and the corresponding EndPackage in that file. Using that standard idiom will also make it possible to use the function name without the full context. That will of course also become a more minimal solution for the OPs problem.

From your comment I understand that you want to load the package but not have the package context in $ContextPath. The answer of Simoon Rochester shows you several ways to achieve that. Here are some more:

use Needs as usual, but remove the package name from $ContextPath afterwards:

Needs["foo`"]; $ContextPath = DeleteCases[$ContextPath,"foo`"]

the following will do the same and is somewhat more elegant and in some senses more reliable:

Block[{$ContextPath=$ContextPath},Needs["foo`"]]

but using Block in that way might have additional subtleties in specific situations.

Another thing you can do is to use two contexts, one for loading, the other one as the namespace of your symbols. If you put the following into file loadfoo.m:

BeginPackage["loadfoo`"]; EndPackage[]; Begin["foo`"]; hello = Function[{}, Print["Hello world!"]]; End[];

then 

Needs["loadfoo`"]

will load with no error or warning messages and put "loadfoo`" onto $ContextPath but with no symbols in it. That file will also define foo`hello which you can use as you seem to intend.

EDIT jkuczm has silently changed a note given above which originally read that Needs expect the context to be added to $ContextPath to the more correct statement that it wants the context to be added to $Packages. Actually that is all that Needs really checks, a fact that I was not aware of as the pair of BeginPackage and EndPackage do more than that and the message that Needs just states that it wants the context to be created. Acutally you don't need to create any symbols in that context nor put it onto $ContextPath as I always assumed. So a file "test.m" anywhere in $Path will be loaded with no error messages by Needs if it contains just this:

Unprotect[$Packages]; AppendTo[$Packages,"test`"]; Protect[$Packages];

Using that idea, putting the following into a file "foo.m" should also solve the OPs problem without any special care by the user who loads and without adding an empty context to $ContextPath:

Unprotect[$Packages]; AppendTo[$Packages,"foo`"]; Protect[$Packages]; Begin["foo`"]; hello = Function[{}, Print["Hello world!"]]; End[];

which has the advantage that it can be loaded with just Needs["foo`"] and avoids any unnecessary side effects.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ "Using that standard idiom will also make it possible to use the function name without the full context...": This is precisely what I want to disallow. I want to make the use of full context mandatory when invoking imported functions. Is there no way to do this? $\endgroup$ Commented Dec 10, 2016 at 21:04
  • $\begingroup$ @kjo this comment really needs to be included in the question itself as it serves to frame it. $\endgroup$ Commented Dec 11, 2016 at 7:46
  • $\begingroup$ @jkuczm: I just added another solution based on your edit. I have never realized that in all these years :-). I have also made this a community wiki so feel free to add or correct more... $\endgroup$ Commented Jan 5, 2017 at 16:46
4
$\begingroup$

You've got more than one question here, but I'm not sure which problem you're really trying to solve.

  • From the title: "How does Needs["foo`"] find the file that defines context foo`?"

Needs looks for a file in the $Path or in the current working directory named foo.m or foo.wl. (As a complication, there is also a list of Paclet directories that it looks in, but you can ignore that if you're not using PacletInfo.m files. The paclet files can be used to register packages where the file names don't correspond to the contexts, however, which may address one of your other questions.)

  • What else must I put in ./client.m, besides Needs["foo`"], so that the expression foo`hello[] evaluates properly?

You don't need to put anything else. foo`hello[] should already be evaluating properly with what you have (assuming foo.m has been found on the $Path). The warning message that you get when you evaluate Needs doesn't affect that.

Likewise for the question from your comment about making the the use of full context manditory -- you have already done that, by using Begin instead of BeginPackage, so that the context foo` is not added to the $Contexts variable.

If it is the warning message that is primarily concerning you, the best approach is to use Get["foo`"] rather than Needs, since Needs is set up to expect the context to be registered, while Get doesn't care.

If you want to use Needs because you only want to load foo.m once in the session, you could instead use the Once function (new in version 11): Once[Get["foo`"]]. Or you could just use Quiet[Needs["foo`"]] or Off[Needs::nocont] to suppress the warning message.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.