I have a long table of data (the number of rows 795,000) and I need to create a code in Mathematica that can calculate the minimum value for every 5000 rows and then I need to know the positions of these minima.Is there any way to do this?
1
- $\begingroup$ Do you need something like this (76121), i.e the sequential running minima? Or must it be by chunks? $\endgroup$rhermans– rhermans2017-05-15 09:28:23 +00:00Commented May 15, 2017 at 9:28
Add a comment |
2 Answers
$\begingroup$ $\endgroup$
6 First, you split yuor big data into chunks of size 5000 rows you will have a matrix like this:
chunkOfMatrix = Table[RandomReal[], {i, 1, 5000}, {j, 1, 100}]; Then applying this will give the Min and its position in each row:
{Min[#], Position[#, Min[#]]} & /@ chunkOfMatrix Hope this will help
- $\begingroup$ Thank you for your answer, but please can you explain more if my data is already as a matrix and its name is data, how can I use your command? $\endgroup$Ghady– Ghady2017-05-12 14:27:16 +00:00Commented May 12, 2017 at 14:27
- $\begingroup$ Welcome, so if your data is smth. like this :
data = Table[RandomReal[], {i, 1, 150000}, {j, 1, 100}];and you can partition this data into chunks of 5000 rows using Partition function:dataChunks = Partition[data, 5000];and then use the code above for every chunk. For example for the first one would be{Min[#], Position[#, Min[#]]} & /@ dataChunks[[1]]. Did I get you right? $\endgroup$David Baghdasaryan– David Baghdasaryan2017-05-12 15:07:16 +00:00Commented May 12, 2017 at 15:07 - $\begingroup$ I tried your way, but I didn't get what I need. For more details my data is below $\endgroup$Ghady– Ghady2017-05-12 17:56:39 +00:00Commented May 12, 2017 at 17:56
- $\begingroup$ Sorry, my data is huge! $\endgroup$Ghady– Ghady2017-05-12 18:05:40 +00:00Commented May 12, 2017 at 18:05
- $\begingroup$ Ohh, in this way you maybe need to write your data to a file and read it by portions. $\endgroup$David Baghdasaryan– David Baghdasaryan2017-05-12 19:54:27 +00:00Commented May 12, 2017 at 19:54
$\begingroup$ $\endgroup$
2 If you are looking for a running minima, then the answer already exists here. If you need it by chunks, then
Some data as an example (you use your own)
data = RandomReal[1, 795000]; The function
MinPositionAndValueByChunks[data_List, chunks_Integer] := MapIndexed[ {First@FirstPosition[#1, Min[#1]] + chunks (First[#2] - 1), Min[#1]} &, Partition[data, chunks] ] Example
Short@MinPositionAndValueByChunks[data, 5000] {{2042,0.0000435445},{6324,0.000265437},<<155>>,{787796,0.000635954},{793848,0.000139004}}
- $\begingroup$ I tried to use your command, but my data has specific numbers. It is the result of another code that computes the differential cross section, so do you think I can use RandomReal.? I should get stability minima. $\endgroup$Ghady– Ghady2017-05-16 00:27:40 +00:00Commented May 16, 2017 at 0:27
- 1$\begingroup$ @Ghady You must be kidding, the random data I offer is just for testing. For the benefit of the people that do not have access to your data, as you did not share it. $\endgroup$rhermans– rhermans2017-05-18 08:12:39 +00:00Commented May 18, 2017 at 8:12