1

In most build systems and development environments, there tends to be exactly one dependency tree per project - each module depends on a constant set of other modules to do its job, with the only leniency being different backwards-compatible versions of the same module. I'm currently going through several circles of hell trying to update such a dependency tree, where many non-backwards-compatible elements depend on a library that has been deprecated due to security vulnerabilities.

What I would be interested to know is if there are viable build systems and languages that allow multiple alternate modules to be selected automatically - the idea is that if you have multiple dependencies that can all solve the same problem, but not necessarily the same way or within the same amount of generality, the build system can choose the best dependency from that list and inform the compilation or runtime of which dependency was used, allowing it to choose the implementation that uses that dependency.

In this way, dependencies first in the preference list can be jettisoned in favor of later ones, either per platform to choose the one that works best on each, per compilation to create a sort of A/B test for different binaries, or permanently when a security vulnerability is discovered. This would also help the dependency tree to be less rigid, as alternatives would be available if the dev team ever wanted to cut off a dependency for other reasons (such as expense).

Does something like this exist, or are there better ways to solve the problem? I imagine that the biggest limiter is language support - anything with a set of static import statements at the top of the file, where you can't modify it with macros or check import errors at runtime, is not going to support this, so Java, Python, and modern JavaScript are right out. There's also the issue of having to write alternate implementations for each dependency anyway, which likely not enough software houses want to do. If a solution like this does exist, what kinds of limitations does it face?

EDIT: To highlight more directly why this might be useful:

  • Permanent removal of dependencies due to security or expense. The tradeoff here is that the change occurs earlier than when the need arises - it might not ever actually be needed, but you have more time to do it and are more likely to do it right.
  • Optimal dependency satisfaction among multiple libraries. For instance, if you have three separate dependencies, one of which uses dependency alternates (A, B, C), one of which uses (A, B, D), and one of which uses (B, C, E), the build system can just load B to satisfy all of them. Such a benefit would be more pronounced if you're writing a software library used by multiple external clients.
  • A/B testing. Each dependency in an alternate set has different performance and usability characteristics, and it would be easy to release unique combinations to different users.
  • OS build differentiation. A certain dependency might perform better on Windows x64 than a different dependency with the same purpose on Linux ARM64, even though a version for both does exist. (I am aware that most build systems let you accomplish this using different OS versions of an intermediate library).
Improve this question
5
  • What happens when your backup dependency has a breaking change? How do you know? Commented Nov 18, 2021 at 11:03
  • @Caleth Presumably the same way I'd find out that my frontline dependency has a breaking change - SemVer, compilation errors, test failures, etc. I recognize that the code depending on a backup might get stale if it doesn't get used for months, so you'd probably have to test that code deliberately in CI every so often. Commented Nov 18, 2021 at 15:39
  • And then what? you add to the speculative work fixing that dependancy? jettison it as a backup? Good luck getting a stakeholder to accept that as a sensible use of your time Commented Nov 18, 2021 at 15:48
  • @Caleth Fair enough. That is a possibility I considered, that it would never be seen as worth the time investment since most of the backup code is never used, though I wasn't sure it would be the absolute limiting factor. Could you submit that as an answer? Commented Nov 18, 2021 at 16:23
  • Isn't this the problem that autoconf, cmake, and similar tools try to solve? Commented Dec 19, 2021 at 15:55

4 Answers 4

Reset to default
1
+100

Allowing dependencies to be satisfied by different module or modules variants, where the decision about which module to pick is not fixed at development time, but made at build time, or even later at run time, is a well-known and frequently used concept. I am sure you know it already, this is called dependency injection. The mentioned Java, Python or Javascript all allow this, there have been books written about how DI can be implemented in these languages, and for each of those languages components like DI containers provide the necessary environment to make this work smooth.

However, I don't think it would be usually a good idea to let something like the build system pick the satisfying dependency fully automatically. Any software system, with a specific combination of dependencies, needs to be tested by it's vendor, and any software vendor who wants to give some warranties about the functionality of their software would be hesitant to provide any guarantees when a bot-like system like the build process can swap out components by other versions or variants of those components arbitrarily.

Software vendors themselves can (and do) make use of DI if they want to provide different product variants just by specifying a different set of modules (so they have a chance to test the specific combination before deploying the software). But several systems also allow this for users or clients in a restricted manner, by providing plugin interfaces.

For example, I have installed an adblocker here for my Firefox browser, but when this plugin does not please me any more, I can decide to deinstall it or use a different one (and since the Mozilla foundation does not provide any warranties to me, this part isn't really problematic for them). I am sure you know lots of other programs which provide a comparable plugin functionality, too.

So in short, keeping dependencies exchangable is a pretty common idea, but a system must be specifially designed for this, and the decision about if certain dependencies fit together must always accompanied by tests.

Improve this answer
3

I don't think this is idea is ever going to work. The biggest problem here is that you'd have to spend time creating tons of alternate implementations that may never get used. You are making them in an effort to future-proof your application in case of something that may never happen.

Ask yourself this: for a personal project would you want to spend time re-implementing working functionality to combat the chance that maybe, someday someone will stop maintaining a given library? I doubt it, I'd rather work on other features to implement. For a professional project, do you think you can convince anyone that it's a good use of your time to build multiple, unused implementations of the same code just in case? Any business I've seen will tell you no and explain that if this thing you are worried about happens, you can deal with it then. Spend the time when there's actually a problem to solve, not when there might be at sometime in the future.

Your best defense against these problems is to apply good design principles to your code so that a risky library doesn't become so pervasive that removing becomes a nightmare. Create an interface and implementation that provides reasonable wrappers around libraries you are worried about. Then if the library needs to be removed you can re-implement and everything else should just work (in theory). Just make sure your wrappers don't conform so much to a certain library's way of doing things that you could never get another library to work within that contract. And of course, only do this if there is a reasonable risk that a library will become problematic. Doing this for every dependency will be a never ending sinkhole of time and energy for little to no benefit (in all likelihood).

Improve this answer
2
  • Yes, it's up-front work to create the insulating interfaces but it pays off. The cost (and tediousness) of doing it, however, means it most often isn't done, and that's when you get problems like this where you can't swap out dependencies. Mostly because #1) lots of scattered code changes and #2) use of convenient features provided by the dependency that are more than you really need, but you used them because they were convenient, and it turns out other packages that could do what you mostly need don't have these extra features ... Commented Dec 15, 2021 at 18:42
  • 1
    It certainly can pay off. It's just something that, in my experience, is something to do carefully. I've seen times where it has paid off. But I've seen plenty of times where it hasn't or has been done so poorly that it can't ever be reused. I'm all for doing it where it makes sense. Figuring out where it makes sense is the difficult part. Commented Dec 15, 2021 at 19:26
1

Why don't you build for all the binary dependencies you want to Target and test/release the one that passes testing performance etc?

Assuming your problem code/app is using interface it's a matter of these dlls/libraries implementing the interface.

Let's say you have problem X that can be solved by 5 diff dlls/libraries. Build your project 5 times using each dependency...test and remove the ones you don't want.

Notice that if you have 10 problems like X in your app and each can be solved using 5 diff libraries - you are looking at a large number of build configurations to test and validate..so it's not worth doing it...unless you know the exact combination you want to select (in that case you might as well just built it with the combination you wanted once)

Anyways with my suggestion It's the same nightmare you are in right now but all the possible builds are done for you. Testing and selecting will become the nightmare.

Improve this answer
1

What I would be interested to know is if there are viable build systems and languages that allow multiple alternate modules to be selected automatically - the idea is that if you have multiple dependencies that can all solve the same problem, but not necessarily the same way or within the same amount of generality, the build system can choose the best dependency from that list and inform the compilation or runtime of which dependency was used, allowing it to choose the implementation that uses that dependency.

The fundamental missing ingredient is a language that can express a dependency as a requirement, not a needed product. "I need a function that can add two integers" rather than, "I need version 4 of company XYZ's math library".

Figure out a formal way to do that and we can make this happen.

Improve this answer

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.