机器学习-随机森林回归(Random Forest Regression)

Section I: Brief Introduction on Random Forest Regression

The random forest algorithm is an ensemble technique that combines multiple decision trees. A random forest usually has a better generalization performance than an individual tree due to randomness, which helps to decrease the model’s varaiance. Other advantages of random forests are that they are less sensitive to outliers in the dataset and don’t require much parameter tuning. The only parameter in random forests that we typically need to experiment with is the number of trees in the ensemble. The only difference is that we use the MSE criterion to grow the individual decision trees, and the predicted target variable is calculated as the average prediction over all decision trees.

FROM
Sebastian Raschka, Vahid Mirjalili. Python机器学习第二版. 南京：东南大学出版社，2018.

代码

from sklearn import datasets
import matplotlib.pyplot as plt
import numpy as np
import warnings
warnings.filterwarnings("ignore")

plt.rcParams['figure.dpi']=200
plt.rcParams['savefig.dpi']=200
font = {'weight': 'light'}
plt.rc("font", **font)

#Section 1: Load data and split it into train/test datasets
price=datasets.load_boston()
X=price.data[:,-1]
y=price.target

from sklearn.model_selection import train_test_split

X_train,X_test,y_train,y_test=train_test_split(X.reshape(-1,1),y,
                                               test_size=0.4,
                                               random_state=1)

#Section 2: Train RandomForestRegressor by using train dataset
from sklearn.ensemble import RandomForestRegressor

forest=RandomForestRegressor(n_estimators=1000,
                             criterion='mse',
                             random_state=1,
                             n_jobs=1)

forest.fit(X_train,y_train)
y_train_pred=forest.predict(X_train)
y_test_pred=forest.predict(X_test)

#Section 3: Evaluate model performance via MSE and R2_Score
from sklearn.metrics import mean_squared_error,r2_score

print("MSE Train: %.3f, Test: %.3f" % (mean_squared_error(y_train,y_train_pred),
                                       mean_squared_error(y_test,y_test_pred)))
print("R2_Score Train: %.3f, Test: %.3f" % (r2_score(y_train,y_train_pred),
                                            r2_score(y_test,y_test_pred)))

#Section 4: Visualize the residuals of the prediction
plt.scatter(y_train_pred,y_train_pred-y_train,
            c='steelblue',
            edgecolor='white',
            marker='o',
            s=35,
            alpha=0.9,
            label='Training Data')
plt.scatter(y_test_pred,y_test_pred-y_test,
            c='limegreen',
            edgecolor='white',
            marker='s',
            s=35,
            alpha=0.9,
            label='Test Data')
plt.xlabel("Predicted Values")
plt.ylabel("Residuals")
plt.legend(loc='upper left')
plt.hlines(y=0,xmin=-10,xmax=50,lw=2,color='black')
plt.xlim([-10,50])
plt.savefig('./fig1.png')
plt.show()

结果

MSE Train: 5.630, Test: 45.144
R2_Score Train: 0.930, Test: 0.500

由上图运行结果可以得知，模型的泛化能力较差，可以后续调整模型深度参数。

参考文献
Sebastian Raschka, Vahid Mirjalili. Python机器学习第二版. 南京：东南大学出版社，2018.