Determining how to redefine this policy is a tricky subject. Not only because of the varying and contradictory opinions about it, but also because we seem to be tackling the issue from the middle. Admittedly, in football, that is a sound strategy, but in politics (and that's just what this is), it's often best to start from the beginning.
Before we get ahead of ourselves, we should step back and officially answer some questions that I'm sure many of you will consider already answered. But until we know everyone agrees on them and knows our stance, we can't properly move forward.
This first question we need to ask ourselves is simply "What are the goals of this site?" What is it we want this site to represent? What are the ideals to which we should hold all of the content on this site? Should it be by physicists and for physicists? Should it be a place where people can learn physics concepts? Should it be a homework help site? Again, many of you may think this an unnecessary step, but when there exists so much disagreement concerning a policy, it makes sense to retreat to a position where we can all agree on something and build from there.
The second question we should ask ourselves is "How much freedom are we willing to grant users in deviating from the goals and ideals of this site?" If the site's goals are to make it a place for professional physicists to ask/answer about research-level physics, how much leeway should we give for users to ask more basic questions? This answer needs to be weighed with how much any amount of deviation will detract from the ability of our site to live up to the goals we agree on. It also serves to provide necessary background and understanding so that everyone can be on the same logical page moving forward.
Now we get into some of the more contentious questions. Most close reasons are easy to determine; the ones we need to decide on are the off-topic sub-reasons. So the next thing we must ask ourselves is "What types/topics/formats of questions would be actively harmful towards meeting the goals/ideals of the site?" This should be applied to questions at all levels. By determining what is actively harmful towards the proposed site goals, we can more easily determine what is off-topic. We should also take this opportunity to realize that the freedoms we established in question 2 may indicate sub-goals of the site and that it may be the case that some questions, which do not directly contribute to the primary goals, may be beneficial or detrimental to the sub-goals.
The last question we should be asking is "What commonalities in these harmful questions can be isolated as reasons to close a question as off-topic?" Here is the important step. This is a place to identify the important patterns in the questions we found to harm the site's objectives. We also should use this step to recognize any biases we have. For example, if we find that everyone is more lenient towards interesting questions, then we need to decide if policy should reflect this or not; allowing exceptions based on general interest or requiring that decisions be made with no regard to the popularity of the question. By approaching this from a standpoint of "Why are these questions harmful to the goals of the site?", we can more easily divine the set of rules that would prevent such questions. But, again, in order to do that properly, we need to have the answers to all the previous questions agreed on and made explicit. I would expect that reasons like the non-mainstream one will remain, but we may find that our homework-like policy never enters into it. We may also find that much of the misuse of some close reasons spawn from a general disinterest of users, and not from any rational source. That would, of course, be a worst case scenario; I'm just giving examples. At any rate, this step also needs to keep in mind how enforceable any close-reason can be. As we once discovered, closing a question purely because it is taken from a homework assignment is not very enforceable. All one needs to do is slightly reword it and claim it is a genuine curiosity. Furthermore, something like closing because a question is too tedious is too subjective to be adequate. Obviously, reasons should be clear, defined, and effective.
If I may jump back to the beginning of my post, we've been approaching this starting from the third question (David Z's thorough research). While I definitely see the appeal in this strategy and advocate it being a good first attempt, once this fails (as we noticed it did), we should go back and truly start from the beginning. It's a longer process that should help to create a more permanent solution.
Now, some of you may point out that I haven't actually done anything towards saying how we could answer these questions, nor have I suggested any way the home-work policy should be changed or researched. You're right. At the moment, I don't see much point in trying to directly address the homework policy, nor can I imagine a situation where we could conduct research a find a clear solution. And if I had included my own answers to the questions I presented, you'd all have voted on whether or not you agree with my answers to those questions as opposed to agreeing with the issue of whether or not the questions need to be addressed at all. Additionally, as demonstrated, people don't seem to have any one thing in mind when they use the homework reason, so the only way I know to fix that is to establish a basis that gets everyone on the same page.
We seem to be building a house on sand and every time the tide comes in, we question why part of this house gets washed away and every time, we try to rebuild that section newer and better. Instead, I merely suggest we tear it all down and lay a proper foundation before attempting to build it back up. The tide may eventually erode the foundation, but at least it'll last a good while longer.