1

I have a dataset of 15 variables (1 under examination, and 14 its regressors) all numeric. What I do is that i run an algorithm that is a recursive forecasting technique. This algorithm cuts the data in an in-sample and an out-sample. Here I want to figure out how to store the results produced for each value of a and t, which are parameters of the cv.hqreg function (hqreg package).

  • Note: That for each value of t and a we get 1 value (the one in the code as predicedQ. For each of those t and a we run the cv.hqreg 648 times. And then again 648 times for the next value of t and a. Thus the ending result will be a matrix/dataset of 648 rows and 231 columns.

For each cv.hqreg I get 100 fitted models from which I select the one with the smallest error via this LMQ$fit$beta[,which(LMQ$lambda.min==LMQ$lambda)] command line. 

 dataR<-TRAINSET fittedvaluesQRidge<-NULL for(i in 1:(nrow(TESTSET)-1)){ #adding a new row and repeat for(a in seq(0,1,0.1)){ #for each penalty of selection for(t in seq(0,1,0.05)){ #for each quantile print(i) #to see it works/or where stops dataR<-rbind(dataR,TESTSET[i,]) #update dataset LMQ<-cv.hqreg(as.matrix(dataR[,-15]),dataR$LHS,method = "quantile",tau=t,alpha = a) #FIT THE Lasso Quantile-MODEL predictdQR<-LMQ$fit$beta[1,which(LMQ$lambda.min==LMQ$lambda)]+LMQ$fit$beta[2,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,1]+LMQ$fit$beta[3,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,2]+LMQ$fit$beta[4,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,3]+LMQ$fit$beta[5,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,4]+LMQ$fit$beta[6,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,5]+LMQ$fit$beta[7,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,6]+LMQ$fit$beta[8,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,7]+LMQ$fit$beta[9,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,8]+LMQ$fit$beta[10,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,9]+LMQ$fit$beta[11,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,10]+LMQ$fit$beta[12,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,11]+LMQ$fit$beta[13,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,12]+LMQ$fit$beta[14,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,13]+LMQ$fit$beta[15,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,14] #find the forecasts fittedvaluesQRidge<-c(fittedvaluesQRidge,predictdQR) #then put them in a vector } } }

The commands I have used to get the predicted value are quite extensive using each one variable at a time. However I have tried to use matrix algebra (matrix of the covariates %*% data with no results but an error: non-numeric argument to binary operator. It works, in an ugly yes way, but if there is a shorter way I would like all the assistance.

Improve this question
6
  • So what is your question? Your title says saving results of a nested loop, but you are saving data per last vector, fittedvaluesQRidge. Does code not work? Undesired results? Commented Dec 2, 2016 at 3:51
  • Yes i get the data in a vector. But it is like '11*21*658' long. While i want to make it come out as a matrix. For example: for t=0.1 and a=0 i get their 468 results. Then for t=0.2 and a=0 i get their next 648 and so on. I really can visualise how to set up the results to come out as a matrix. Commented Dec 2, 2016 at 10:19
  • I thought you were familiar from your last question! I see you do not use any apply functions. Sapply can bind the predictedQR vectors in a matrix. Where is a and t loop variables being used? Commented Dec 2, 2016 at 14:33
  • I really tried but i failed miserably. Well the 't' and 'a' are used inside the 'cv.hqreg' function. Commented Dec 2, 2016 at 14:54
  • Do you need a matrix or an array? You are showing more than two dimensions: 11 X 21 X 658. Commented Dec 2, 2016 at 22:10

1 Answer 1

Reset to default
1

Consider sapply() with expand.grid() as sapply can take multiple input lists or vectors, similar to the nested for loops but returns a matrix. And with expand.grid which cross joins two lists in a data.frame object, you can capture every combination between a and t:

at_combns <- expand.grid(a=seq(0,1,0.05), t=seq(0,1,0.1)) matpredictdQR <- sapply(seq(nrow(at_combns)), function(j, i){ # UPDATE dataset dataR <- rbind(TRAINSET, TESTSET[1:i,]) # FIT THE Lasso Quantile-MODEL LMQ <- cv.hqreg(as.matrix(dataR[,-15]),dataR$LHS,method = "quantile", tau=at_combns$t[j], alpha=at_combns$a[j]) predictdQR <-LMQ$fit$beta[1,which(LMQ$lambda.min==LMQ$lambda)]+ LMQ$fit$beta[2,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,1]+ LMQ$fit$beta[3,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,2]+ LMQ$fit$beta[4,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,3]+ LMQ$fit$beta[5,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,4]+ LMQ$fit$beta[6,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,5]+ LMQ$fit$beta[7,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,6]+ LMQ$fit$beta[8,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,7]+ LMQ$fit$beta[9,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,8]+ LMQ$fit$beta[10,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,9]+ LMQ$fit$beta[11,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,10]+ LMQ$fit$beta[12,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,11]+ LMQ$fit$beta[13,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,12]+ LMQ$fit$beta[14,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,13]+ LMQ$fit$beta[15,which(LMQ$lambda.min==LMQ$lambda)]*TESTSET[i+1,14] return(predictdQR) }, seq(nrow(TESTSET)-1))
Improve this answer
Sign up to request clarification or add additional context in comments.

18 Comments

I dont think this updates each run the matrix. Like run 1: 361 rows, then run 2 362 rows, run 3 363 rows and so on. I used the print(dim(dataR)) and i got that dataR was now a matrix 1x15. Using rbind(dataR,TESTSET[i,] i think will work. Also I think you need to add a ) at the end of the whole code to close the sapply function.
Also, everytime the code goes to run the cv.hqreg function R just crashes. I know it does crash cause when cv.hqreg runs the function prints at which cross validation you are.
Ok after some pc change the code run. The only issue is that it doesn't update the dataR. Both rbind(dataR,TESTSET[1:i,] and rbind(dataR,TESTSET[i,]) don't update it. The first gives a dim = 1008x15 and the latter a dim = 362x15 in each run.
Function never returns dataR, but temporarily updates it in sapply to calculate the predictdQR which I though was the main focus here. Did you not retreive desired results of predicted values matrix? Have dataR updated in a separate for loop. Now dataR has TRAINSET to a rbind call. See edit.
Sadly rbind(dataR,TESTSET[1:i,] for each run gives that dataR is 362 rows. It doesn;t seem to update the matrix which is weird given that we rbind. Same results for rbind(dataR,TESTSET[i,] but here we have straight 1008 rows. I don't understand why.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.