I make an application that must be able to process and store a lot of measurements.
There can be up to 129600 measurements and every measurement can have 1499 values.
All data is stored in an array, each column is a measurement.
After 10 measurements I save the data from the array to a file. Because if the application crashes, then i haven’t lost all data.
In the beginning of the measurements, saving to the file only takes a few millieseconds. But when i am at measurement 200, it already takes 13 seconds. The time to save to the file keeps increasing. This makes sense, of course, because the array is getting bigger.
I can choose to only save every 50 or 100 measurements. But saving to the file will still take a long time.
Below you can see what the data in the file looks like:
Frequency(Hz);S21(dB)_1;S21(dB)_2 10000000;-42.87726786;-35.66746585 79950000;-60.5887682;-63.55421833 149900000;-84.56555597;-74.36793049 219850000;-76.71335093;-80.68063652 289800000;-68.96360042;-68.41861962 359750000;-71.24272337;-74.90287556 429700000;-78.65528362;-75.50614099 ... ... 13860100000;-85.80839142;-84.03051407 13930050000;-79.31238317;-82.87468675 14000000000;-88.9003575;-75.48071161
Is there a possibility to work in a different way, In such way that it takes less time to save?
Source: Windows Questions