I had to transfer my project over to Microsoft to look into a LiveUnitTesting bug, and the agent I got said there's too many errors to look at and he doesn't want to bother.
Okay so clearly my csproj has ..\..\..\packages which really is the root of C where nuget is pointing.
Is there no variable I can place in my csproj that would tell VS to "just use whatever location the local nuget uses"
Like {NugetPath}\Castle.Core.4.2.1\lib\net45\Castle.Core.dll or something?
TL;DR Consider migrating from packages.config to PackageReference.
Long version:
"Normally" packages are restored automatically on build, and the only ways I know for this to be disabled is either a nuget.config setting, or a Visual Studio setting. It doesn't make sense to me why an agent whose responsibility is to investigate issues using other people's code would disable automatic package restore, so I can only assume that either your repo has disabled it, or there's something non-standard with your project that caused it to fail on the agent's machine. New projects from templates do not have this problem.
Anyway, this is not what your question asked. The answer to your question is no, there's no built-in MSBuild property that points to the NuGet restore folder.
However, projects using PackageReference for NuGet dependencies, instead of packages.config, do not modify the csproj to add paths to assembly files. It's calculated automatically at build time, so there is less risk of build failures when people clone your repo at different locations to where you cloned yours, or if you move code around your repo, it will not require modifications to the csproj, only run a nuget restore.
I assume the reason you're using a packages folder outside the repo is to share the folder with multiple repos and therefore save disk space. PackageReference does this automatically. It no longer creates a package folder per repo/solution, and builds use the assemblies directly from the global packages folder.
<Reference /> elements to assembles in the csproj, so if you tried to migrate manually (for example deleting the packages.config file and adding PackageReference>, it's an incomplete migration. not only do the correct Reference elements need to be deleted, there's possibly props and targets Imports that need to be deleted, and maybe other things I don't know about.