You had missed to apply the column ordering the in the test data (X_test). When you do the same on the test data, you will get the same score.

from sklearn.datasets import load_iris from sklearn.tree import DecisionTreeClassifier from sklearn.model_selection import train_test_split import numpy as np iris = load_iris() X = iris['data'] y = iris['target'] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.90, random_state=0) def shuffle_data(data, n): return np.hstack((data[:,n:], data[:,:n])) clf = DecisionTreeClassifier(random_state=0) clf.fit(X_train, y_train) print(clf.score(X_test, y_test)) # 0.9407407407407408 clf = DecisionTreeClassifier(random_state=0) clf.fit(shuffle_data(X_train,1), y_train) print(clf.score(shuffle_data(X_test,1), y_test)) # 0.9407407407407408 clf = DecisionTreeClassifier(random_state=0) clf.fit(shuffle_data(X_train,2), y_train) print(clf.score(shuffle_data(X_test,2), y_test)) # 0.9407407407407408 clf = DecisionTreeClassifier(random_state=0) clf.fit(shuffle_data(X_train,3), y_train) print(clf.score(shuffle_data(X_test,3), y_test)) # 0.9407407407407408 

Update:

In your second example, you set the test_size equal to 0.95, which has left you with only 7 data point and their classes are array([0, 0, 0, 2, 1, 2, 0]).

If you measure the training score of decision tree in both cases, it is 1.0. This tells us that the model has found the optimal seperation in both the scenarios.

Simple answer is yes, the results would vary when the column order changes, when different combination of rules (different splitting condition) could lead to perfect separation of datapoints (100% accuracy).

Using plot_tree we can visualise the tree. Here we need to understand the implementation of DecisionTree. This answer quotes the important point from the documentation:

The problem of learning an optimal decision tree is known to be NP-complete under several aspects of optimality and even for simple concepts. Consequently, practical decision-tree learning algorithms are based on heuristic algorithms such as the greedy algorithm where locally optimal decisions are made at each node. Such algorithms cannot guarantee to return the globally optimal decision tree. This can be mitigated by training multiple trees in an ensemble learner, where the features and samples are randomly sampled with replacement.

The point that we need to concentrate here is that practical decision-tree learning algorithms are based on heuristic algorithms such as the greedy algorithm where locally optimal decisions are made at each node when take a greedy algorithm, the change of column order can impact its results.

At the same, when there are more datapoints in your dataset (when is not in your example), it is highly unlikely to get different results when you change order of the columns.

Even in this example, when set test_size=0.90, we can get the same score as 0.9407407407407408.