Assume that we have two separate tree regressions. I'm interested in understanding whether the product of tree regressions can be represented by a single tree. Would this be possible?
2
- $\begingroup$ By "product of tree regressions" do you just mean the pointwise product of the resulting prediction functions? The two regressions have the same features but different targets, or...? $\endgroup$Ben Reiniger– Ben Reiniger ♦2022-12-15 14:39:29 +00:00Commented Dec 15, 2022 at 14:39
- $\begingroup$ @BenReiniger Yes, in the case the two tree regressions have the same feature but different targets. $\endgroup$TFT– TFT2022-12-15 16:56:08 +00:00Commented Dec 15, 2022 at 16:56
Add a comment |
1 Answer
$\begingroup$ $\endgroup$
1 Yes, it is possible to find a tree that represents the product. An easy way to do so is to extend the first tree in each leaf with the second tree.
Example
Assume there are theses two tree:
A regression product tree can then be given by:
Notes
Is there a smaller product tree?
Not necessarily (but in some cases there might be). Look at the example (which I chose to make this point): Each leaf has a different output, so a tree with less leaf nodes would not work.
Is it feasable?
In most cases, a representation by two trees is more compact and better suited to many tasks. When it comes to training regression trees - as long as the targets of the two trees are given, training them independently is faster and requires less training data.
Note that the depth of regression product tree is the sum of the depth of both trees and the number of leafs is the product of the number of leafs of single trees.
General Case
Your case is just a special case of a more general property of regression trees: Assume you have $K$ regression trees that take the same features as input. Let's say $f_k(x),\,(k=1,\ldots, K)$.
Given a function $y=g(f_1(x),\ldots,f_K(x))$ that combines these trees (the product in your case), this function $g$ can be represented by a single regression tree.
The construction is done similar to above example.
- $\begingroup$ I know, I am late to the party. I hope the answer is still of interest to anyone $\endgroup$Broele– Broele2023-06-22 11:06:57 +00:00Commented Jun 22, 2023 at 11:06