Scikit-learn : Input contains NaN, infinity or a value too large for dtype ('float64')

The problem with your regression is that somehow NaN's have sneaked into your data. This could be easily checked with the following code snippet:

import pandas as pd import numpy as np from sklearn import linear_model from sklearn.cross_validation import train_test_split reader = pd.io.parsers.read_csv("./data/all-stocks-cleaned.csv") stock = np.array(reader) openingPrice = stock[:, 1] closingPrice = stock[:, 5] openingPriceTrain, openingPriceTest, closingPriceTrain, closingPriceTest = \ train_test_split(openingPrice, closingPrice, test_size=0.25, random_state=42) openingPriceTrain = openingPriceTrain.reshape(openingPriceTrain.size,1) openingPriceTrain = openingPriceTrain.astype(np.float64, copy=False) closingPriceTrain = closingPriceTrain.reshape(closingPriceTrain.size,1) closingPriceTrain = closingPriceTrain.astype(np.float64, copy=False) openingPriceTest = openingPriceTest.reshape(openingPriceTest.size,1) openingPriceTest = openingPriceTest.astype(np.float64, copy=False) np.isnan(openingPriceTrain).any(), np.isnan(closingPriceTrain).any(), np.isnan(openingPriceTest).any() (True, True, True) 

If you try imputing missing values like below:

openingPriceTrain[np.isnan(openingPriceTrain)] = np.median(openingPriceTrain[~np.isnan(openingPriceTrain)]) closingPriceTrain[np.isnan(closingPriceTrain)] = np.median(closingPriceTrain[~np.isnan(closingPriceTrain)]) openingPriceTest[np.isnan(openingPriceTest)] = np.median(openingPriceTest[~np.isnan(openingPriceTest)]) 

your regression will run smoothly without a problem:

regression = linear_model.LinearRegression() regression.fit(openingPriceTrain, closingPriceTrain) predicted = regression.predict(openingPriceTest) predicted[:5] array([[ 13598.74748173], [ 53281.04442146], [ 18305.4272186 ], [ 50753.50958453], [ 14937.65782778]]) 

In short: you have missing values in your data, as the error message said.

EDIT::

perhaps an easier and more straightforward approach would be to check if you have any missing data right after you read the data with pandas:

data = pd.read_csv('./data/all-stocks-cleaned.csv') data.isnull().any() Date False Open True High True Low True Last True Close True Total Trade Quantity True Turnover (Lacs) True 

and then impute the data with any of the two lines below:

data = data.fillna(lambda x: x.median()) 

or

data = data.fillna(method='ffill')