Training CNN for Regression

Question

Background: I am using CNN to predict forces acting on a circular particle in a granular medium. Based on the magnitude of the forces, particle exhibits different patterns on its surface. The images are greyscaled 64-by-64 pixels. You can see different pictures with the magnitude of the corresponding force on the x-axis attached below.

My attempt at a solution: I am relatively new to deep learning and data science and decided to use a simple conv net to run a regression. My code is provided below. I tried to fit the model using adam optimizer and MSE as a loss function, but it takes forever and sometimes aborts execution by itself. What could be the problem? I am running it on a PC with 8GB RAM, 1TB SSD, Intel i7 CPU, and GTX 1080 GPU.

def build_model(): model = Sequential() model.add(Conv2D(64, kernel_size=(3, 3), strides = (1,1), padding = 'valid', activation='relu', input_shape=input_shape)) model.add(Conv2D(64, kernel_size = (3, 3), strides = (1,1), padding = 'valid', activation='relu')) model.add(MaxPooling2D(pool_size=(2, 2), strides = (2,2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(53824, activation='relu')) model.add(Dense(53824, activation='relu')) model.add(Dense(53824, activation='relu')) model.add(Dropout(0.5)) model.add(Dense(1, activation='linear')) return model

Images of 9 particles with different force labels on x-axis:

You sure need 3 dense layers with 53,824 (!) nodes each?? No wonder your code takes forever. You don't provide any details of your problem, but this sure sounds like overkill (and "simple cnn" it is not, despite the simple structure)... — desertnaut
– desertnaut, Commented Sep 28, 2019 at 21:19
I agree that was an overkill :) I did not notice that until I posted it here. Thank you for pointing that out. — Renat Sergazinov
– Renat Sergazinov, Commented Sep 28, 2019 at 22:08

desertnaut · Accepted Answer · 2019-09-28 21:44:42Z

Although building neural network models is admittedly still an art rather than a science, there are some (unwritten) rules, at least for initial approaches to a problem, such as yours here (I guess).

One of them is that dense layers with 50,000 nodes are too large, and AFAIK I have never seen such large layers in practice; multiply this x3 (layers), and no wonder your code takes forever.

I would certainly suggest to experiment with a dense layer size between 100 - 1000, and even start with less than 3 dense layers. Reducing your 1st CNN layer to 32 is also certainly an option.

Stack Exchange Network

Training CNN for Regression

1 Answer 1

Hot Network Questions

Training CNN for Regression

1 Answer 1

Related

Hot Network Questions