I can work out the answer more thoroughly if you'd like but here's a write-up of the steps you want to take to find a solution.
First, the size of the shadow is a continuous function of (I think) seven variables. We have already defined the length, width and height, though, so I'll just treat those as constants for now. You probably want to set them to 1, though, to make the calculations easier. You can make them the real numbers later.
The first three unknown variables that affect the shadow of the box, then, are the yaw, pitch and roll of the box. Using those, we can fully represent the rotation of the box by the 3D rotation matrix. These are super clean to work with, so if you haven't learned about them yet then don't worry -- things will cancel out.
Rather than working with the box itself, though, I'd recommend working with the vectors representing the box. It is really just three vectors, after all, where each is parallel to one of the $X$, $Y$, or $Z$ axes. So what does the rotation matrix do to those vectors?
Next, I believe a fourth necessary variable would be the angle of the sun. Let's just say that the sun is directly above the box, though. That's the easiest place to start and then you can consider what changes when you remove that assumption.
Okay, we know we are representing our rotation in terms of the operations $\{\text{rotate in x}, \text{rotate in y}, \text{rotate in z}\}$. We now want a function that relates this rotation to the area of the shadow.
The size of the shadow can be represented as flattening the box onto some plane that is flat in $X$ and $Y$ (assuming the sun is above us). If we assume the box is centered at the origin (which we probably want to), this means that we take all of the $Z$ values for every point in the box and set them to 0. This operation is called a projection and can also be described by a matrix. Now what is the area of the following 2D shape? I trust you to work that part out. It will come out being some sum of trigonometric functions of the yaw, pitch, and roll and should, likely, have a lot of things cancel.
Our second to last step is to take the derivative of this function and set it to 0 to find the maximum. I am sure there are both mins and maxes (most likely an even number of both), so you only need to solve for the cases where the derivative is 0 at a local maximum of the shape.
Then my last step would be to double-check the work. Does the resulting answer make sense? Is it a function of all three variables as we hoped? Does it maintain the symmetries that we'd expect of such an operation? Does our area for the shadow work for the easy 90 degree rotation cases? etc. etc.
Have fun :)