I have referenced directly a NuGet package x with v5.0.
Another NuGet package that I reference has this x package referenced but with version v4.0.
What determines which assembly version is built into the bin folder?
In your case v5.0 of package x is restored and the matching target framework lib copied to your application's output path, due to the Nearest Wins rule, which resolves the version of the package closest to your app in the dependency graph. When referencing different package versions having the same distance to your application, then the Cousin Dependencies rule applies.
To "see" which version is resolved using Visual Studio, go to your project in the Solution Explorer and expand Dependencies | Packages.
Alternatively, you may use dotnet list package to list the package references for a project, or use dotnet list package --include-transitive to additionally get a list of packages that the top-level packages depend on.
PackageReferences are put into the package graph, both direct and via ProjectReferences. The scenario you have described in your comment differs from the one of your original question: Now v4.0 is closer to your app than v5.0 and therefore v4.0 is resolved. But this now results in a "detected package downgrade" warning: NuGet Warning NU1605.The answer of FlashOver is correct for newer PackageReference csproj format(aka .Net Core format).A full algorithm (including the Nearest Wins and the Cousin Dependencies rules) described in https://learn.microsoft.com/en-us/nuget/concepts/dependency-resolution#dependency-resolution-with-packagereference.
The algorithm is less predictable with older packages.config format (aka .Net Framework format).
With packages.config, NuGet attempts to resolve dependency conflicts during the installation of each individual package…
Each new package installation requires a traversal of the whole graph and raises the chance for version conflicts…
When a conflict occurs, installation is stopped, leaving the project in an indeterminate state, especially with potential modifications to the project file itself.