1

I have encountered on designing program to allow capturing images every second from video files (avi, mp4, etc...).

First, I was able to capture images frame by frame from video file. Second, I was able to analyze pixel color values from images in the same folder at the same time and saved pixel value in the txt file.

And here I have some problem. I am now trying to combine these two codes at once, but I have strange results. I refer the code below.

int main(){ VideoCapture cap("D:\\data\\extra\\video200ul.avi"); if (!cap.isOpened()) return -1; Ptr<BackgroundSubtractor> pMOG2 = createBackgroundSubtractorMOG2(20, 16, true); Mat fg_mask; Mat frame; int count = 0; String name, folder; for (;;) { // Get frame cap >> frame; // get a new frame from video ++count; // Update counter // Background subtraction if (count % 2 == 0) { pMOG2->apply(frame, fg_mask, 0.001); cout << count << endl; if (!frame.empty()) { imshow("frame", frame); // imshow("fg_mask", fg_mask); } // Save foreground mask name = "mask" + std::to_string(count) + ".png"; // string name = "mask_" + std::to_string(static_cast<long long>(count) + ".png"; folder = imwrite("D:\\data\\extra\\" + name, frame); } anal(folder); } waitKey(0); return 0;

}

First, The code above I wrote is for capturing images frame by frame from video file. However, if I got the images per frame, I will have so many pictures on my folder, so I would like to capture an image per second from the video file. I have tried to use CV_CAP_PROP_POS_MSEC instead using cap << frame, but it did not work for me.

Second, when I merge this code to another code what I wrote below, it showed some error messages like, "libpng warning image width, length, data are zero in ihdr."

int anal(String folder) { folder = "D:\\data\\extra\\*.png"; vector<String> filenames; glob(folder, filenames); cv::Mat ori_image; for (size_t i = 0; i < filenames.size(); ++i) { ori_image = imread(filenames[i], IMREAD_COLOR); if (ori_image.empty()) { cout << "Check your file again." << std::endl; return -1; } rectangle(ori_image, Point(215, 98), Point(245, 110), Scalar(0, 255, 255), 1); imshow("Original Image", ori_image); cv::Scalar sums; sums = cv::sum(ori_image); double totalSum = sums[0] + sums[1] + sums[2]; if (totalSum <= 0) { cout << "$$ RGB percentage $$" << " \n\n"; cout << "R: " << 100.0 / 3 << " % \n"; cout << "G: " << 100.0 / 3 << " % \n"; cout << "B: " << 100.0 / 3 << " % \n\n"; } else { cout << "$$ RGB percentage $$" << " \n\n"; // red value cout << "R: " << sums[2] / totalSum * 100 << " % \n"; // red value cout << "G: " << sums[1] / totalSum * 100 << " % \n"; // green value cout << "B: " << sums[0] / totalSum * 100 << " % \n\n"; // blue value }

}

as I prepared the code above, I tried to calculate red, blue, green percentages of all the captured images from the video. However, when I separate these two code and run them, they worked fine, but if I merge them together, It showed error messages.

I would like to combine these two code for analysis for color values from the captured images at video every second.

Please help me out this problem.

Thank you in advance.

-----------Edited part----------------------

I used your revised version and applied to my updated code, 

void imageAnalysis(std::string folder, cv::Mat frame){ cv::Mat ori_image = frame.clone(); std::string path = folder; cv::rectangle(ori_image, Point(215, 105), Point(245, 120), Scalar(0, 255, 255), 1); cv::imshow("Original Image", ori_image); cv::waitKey(1); String folder = "D:\\data\\dfdf\\*.png"; vector<String> filenames; cv::glob(path, filenames); for (size_t t = 0; t < filenames.size(); t++) { ori_image = imread(filenames[t], IMREAD_COLOR); // ori_image if (ori_image.empty()) { //ori_image cout << "Check your file again." << "\n"; break; //return -1; } rectangle(ori_image, Point(215, 105), Point(245, 120), Scalar(0, 255, 255), 1); imshow("Original Image", ori_image); cv::waitKey(1); Mat image_HSV; cvtColor(ori_image, image_HSV, CV_BGR2HSV); double h = 0.0; double s = 0.0; double v = 0.0; int col = image_HSV.cols; // 480 int row = image_HSV.rows; // 272 int corow = ((col - 235) - 215) * ((row - 152) - 108); Mat mask; inRange(image_HSV, Scalar(100, 0, 0), Scalar(100, 255, 255), mask); // convert binary image_HSV.setTo(Scalar(0, 0, 0), mask); for (int i = 108; i < row - 152; i++) { for (int j = 215; j < col - 235; j++) { Vec3b hsv = image_HSV.at<cv::Vec3b>(i, j); h += (int)(hsv.val[0]); s += (int)(hsv.val[1]); v += (int)(hsv.val[2]); if (hsv[0] != 100) { hsv[0] = 0; hsv[1] = 0; hsv[2] = 0; } } } cout << "$$ Hue(H), Saturation(S), Brightness(V) $$" << filenames[t] << " !! \n\n"; cout << "H: " << h / corow * 360 / 180 << " % \n"; // cout << "S: " << s / corow * 100 / 255 << " % \n"; cout << "V: " << v / corow * 100 / 255 << " % \n\n"; std::ofstream file("D:\\data\\dfdf\\result_4.txt", std::ios_base::app); file << v / corow * 100 / 255 << " \n"; // v value file.close(); }

}

As you can see the imageAnalysis() function, I added std::string folder for the path of extracted images from video clip. However, when I applied this code, I have really weird results like below..

enter image description here

I thought I am supposed to get color value from every 24th image but as you see the results above, I got color values from all images in random order.

Thank you in advance.

It was really nice to learn how to code in efficient way!!

Improve this question
6
  • You should write the complete error, with the stacktrace. It is hard to read huge amount of code and foresee possible problems. In this case I guess you are saving some empty images? You should have in your first code something like, if(frame.empty()) break; to avoid having problems with empty images. Also, what did not work with CV_CAP_PROP_POS_MSEC ? this is not a substitution to cap >> frame. One more thing, is there really a need to the images and re load them, isn't it better to just pass the image to the other function? Commented Jan 21, 2019 at 15:22
  • Hi api55, I really appreciate your comment. I need to save images per second because I need to present some images for the color degradation. So, that's why I saved images from the video file. Then, I tried to calculate the RGB percentage of each image. So, that's why I need to save images every second. Actually, I was able to operate these two codes separately, but I want to save time to get color data from the video file directly. Commented Jan 22, 2019 at 0:45
  • And I appreciate that I totally forgot to add "if(frame.empty()) break;" statement as you said. Commented Jan 22, 2019 at 0:45
  • Further, when I apply CV_CAP_PROP_POS_MSEC to my code, I found some error messages like "CV_CAP_PROP_POS_MSEC is not defined." So, I tried to figure how to solve this problem with similar questions at stackoverflow, but I could not find proper answers. Commented Jan 22, 2019 at 0:48
  • And I worked on this code yesterday, and I have no error message and it was operated well, but I still have some problems. So, I use "if (count % 24 == 0)" statement for saving 24 frames per second. In other word, I was able to save images per second from the video file. So, I solved my first question. However, when I transfer this data with function "anal(folder);", the function anal() read RGB colors of all the frames. In this case, I want to extract color values of every 24th frame (24, 48, 96, .....). This is my problem. Commented Jan 22, 2019 at 0:54

1 Answer 1

Reset to default
1

Just to clear the error you mentioned about CV_CAP_PROP_POS_MSEC in your comments:

when I apply CV_CAP_PROP_POS_MSEC to my code, I found some error messages like "CV_CAP_PROP_POS_MSEC is not defined."

A lot of the constant values are scoped in OpenCV. That means, CV_CAP_PROP_POS_MSEC is not defined, but cv::CV_CAP_PROP_POS_MSEC is. You can also obtain the FPS with cv::CAP_PROP_FPS.

Now to your code, I would actually do something that does not require to save and load the image, but rather pass the images to be processed, like this:

#include "opencv2/opencv.hpp" #include <iostream> int main(){ cv::VideoCapture cap("D:\\data\\extra\\video200ul.avi"); if (!cap.isOpened()) { std::cout << "Could not open video" << std::endl; return -1; } cv::Ptr<cv::BackgroundSubtractor> pMOG2 = cv::createBackgroundSubtractorMOG2(20, 16, true); cv::Mat fg_mask, frame; int count = 0; const int fps = 24; // you may set here the fps or get them from the video std::string name, folder; // with cap.read you can check already if the video ended while (cap.read(frame)) { // Background subtraction if (count % fps == 0) { pMOG2->apply(frame, fg_mask, 0.001); // Save foreground mask name = "mask" + std::to_string(count) + ".png"; bool result = cv::imwrite("D:\\data\\extra\\" + name, frame); imageAnalysis(frame, count); } // at the end of the loop so that the first image is used ++count; } cv::waitKey(0); return 0; }

And the imageAnalysis function is defined as:

// You can pass cv::Mat as value, it is almost like a smart pointer void imageAnalysis(cv::Mat frame, int count) { cv::Mat ori_image = frame.clone(); cv::rectangle(ori_image, Point(215, 98), Point(245, 110), Scalar(0, 255, 255), 1); // each imshow needs a waitKey to update the window in which it is being shown cv::imshow("Original Image", ori_image); cv::waitKey(1); cv::Scalar sums; sums = cv::sum(ori_image); double totalSum = sums[0] + sums[1] + sums[2]; std::ofstream output("D:\\data\\extra\\mask" + std::to_string(count) + ".txt"); if (totalSum <= 0) { std::cout << "$$ RGB percentage $$" << std::endl << std::endl; std::cout << "R: " << 100.0 / 3 << std::endl; std::cout << "G: " << 100.0 / 3 << std::endl; std::cout << "B: " << 100.0 / 3 << std::endl << std::endl; output << "$$ RGB percentage $$" << std::endl << std::endl; output << "R: " << 100.0 / 3 << std::endl; output << "G: " << 100.0 / 3 << std::endl; output << "B: " << 100.0 / 3 << std::endl << std::endl; } else { std::cout << "$$ RGB percentage $$" << std::endl << std::endl; std::cout << "R: " << sums[2] / totalSum * 100 << std::endl; // red value std::cout << "G: " << sums[1] / totalSum * 100 << std::endl; // green value std::cout << "B: " << sums[0] / totalSum * 100 << std::endl << std::endl; // blue value output << "$$ RGB percentage $$" << std::endl << std::endl; output << "R: " << sums[2] / totalSum * 100 << std::endl; // red value output << "G: " << sums[1] / totalSum * 100 << std::endl; // green value output << "B: " << sums[0] / totalSum * 100 << std::endl << std::endl; // blue value } }

Some comments of the code above, I replaced the cap >> frame to cap.read(frame). It is the same functionality, but the later gives a bool result that is false if it could not grab the image, like if the video is over. I change the count add at the end, yo you get the frames 0,23,... this way the first one will be use as well. Finally, you should use the namespaces cv::, std:: etc. This is just best practice, it avoids ambiguities and problems that may arise with certain libraries.

If you do not need the image in disk, but only the analysis, then remove the saving part and pass every frame to the imageAnalysis function, this way you may have more data for your statistics. Also, consider returning the cv:Scalar of sums in the function and then you can do some statistics of the whole second or the whole video.

If you have any question, feel free to ask in the comments.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

I really appreciate your comment. It works as a charm!! But, the reason that I stated "save and load" code in this program is I tried to save RGB data to the txt file. So, I updated the code for saving data in txt file. And, if I don't save the file and load, I could not write color data in the txt file. So, when I save data in txt file, color data from all images are saved in random order. That's what I want to fix. So, I tried to use 'save and load' function to save images, extracted from video.
@DavidS do you mean with the same name as image but with different extension? That is easy, pass the count variable to the imageAnalysis function and create the txt file there. It should not be random order, since it goes progressively and there is no parallel threads. I will update the answer to also save the data and not just print it
@DavidS btw, this line : folder = imwrite("D:\\data\\extra\\" + name, frame); from your code is wrong. imwrite gives a boolean, so folder would be the implicit cast of the boolean to string.
From your updated code, I have an error on this line std::ofstream output("D:\\data\\extra\\mask" + std::to_string(count) + ".txt", frame"); It says "There is no instance for this structure."
@DavidS it is a mistake, frame does not go there
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.