I have a project which outputs data into many files. These files look something like this as interpreted by readmatrix:
ans =
8×8 table
Date_Time EZOO2Con___ EZOCO2Con_ppm_ SGPCO2Con_ppm_ SGPTVOC_ppb_ BMEHumidity___ BMEPressure_Pa_ BMETemp_DegC_
___________________ ___________ ______________ ______________ ____________ ______________ _______________ _____________
09/06/2022 11:55:17 19.16 419 400 0 48.5 95948 22.57
09/06/2022 11:55:18 19.16 419 400 0 48.89 99577 22.58
09/06/2022 11:55:19 19.16 419 400 0 48.89 99578 22.58
09/06/2022 11:55:20 19.15 420 400 0 48.84 99584 22.57
09/06/2022 11:55:21 19.15 420 400 0 48.95 99574 22.58
09/06/2022 11:55:22 19.15 421 400 0 49.15 99578 22.57
09/06/2022 11:55:23 19.15 421 400 0 48.9 99577 22.56
09/06/2022 11:55:24 19.15 422 400 0 48.9 99573 22.57
For my previous test, I have approx. 289 separate files of this format which I'd like to combine together in 8 arrays for plotting.
The Date_Time column is a string with MM/DD/YYYY and HH:MM:SS separated by a space. When using the table2array command, I am able to convert the date&time column of each file's data into a datetime array. However, I am unable to use the cat or vertcat functions to append the date&time column to my "combined" array. Below is the code that's giving me trouble:
for k = 1:length(fileList)
baseFileName = fileList(k).name;
fullFileName = fullfile(fileList(k).folder, baseFileName);
fprintf(1, 'Now reading %s\n', fullFileName);
data = readtable(fullFileName);
timecol = data(:,1);
EZOCO2col = data(:,2);
EZOO2col = data(:, 3);
SGP30CO2col = data(:, 4);
SGP30TVOCcol = data(:, 5);
BME280Humcol = data(:, 6);
BME280Presscol = data(:, 7);
BME280Tempcol = data(:, 8);
timecol_array = table2array(timecol);
EZOCO2col_array = table2array(EZOCO2col);
EZOO2col_array = table2array(EZOO2col);
SGP30CO2col_array = table2array(SGP30CO2col);
SGP30TVOCcol_array = table2array(SGP30TVOCcol);
BME280Humcol_array = table2array(BME280Humcol);
BME280Presscol_array = table2array(BME280Presscol);
BME280Tempcol_array = table2array(BME280Tempcol);
timecol_tot = cat(1, timecol_tot, timecol_array);
EZOCO2col_tot = cat(1, EZOCO2col_tot, EZOCO2col_array);
EZOO2col_tot = cat(1, EZOO2col_tot, EZOO2col_array);
SGP30CO2col_tot = cat(1, SGP30CO2col_tot, SGP30CO2col_array);
SGP30TVOCcol_tot = cat(1, SGP30TVOCcol_tot, SGP30TVOCcol_array);
BME280Humcol_tot = cat(1, BME280Humcol_tot, BME280Humcol_array);
BME280Presscol_tot = cat(1, BME280Presscol_tot, BME280Presscol_array);
BME280Tempcol_tot = cat(1, BME280Tempcol_tot, BME280Tempcol_array);
end
I receive this error each time:
Error using datetime/cat (line 1376)
All inputs must be datetimes or date/time character vectors or date/time strings.
Error in Plot_attempt_9_6_22_1 (line 66)
timecol_tot = cat(1, timecol_tot, timecol_array);
As per How to preallocate a datetime array in matlab, I have tried:
timecol_tot = [];,
timecol_tot = datetime([],[],[],[],[],[]);, and
timecol_tot = NaT(1,1); to no avail.
Because the length of each of these files may vary, I didn't try pre-allocating the datetime array in the size of the incoming data, since that may not work across different datasets. However, it does work if I have only one file.
Is there a way to do this that would allow me to just initialize an empty array and concatenate datetime arrays to it without defining the size of the first datetime set to add?
CodePudding user response：
You are asking 2 questions, I am new here but some moderators may tell you break it down into 2 separate questions.
I am here answering both anyway :
- The error doesn't seem to be coming from
cat
With the data you have provided I have reproduced cat output and cat works fine; it concatenates DATATIME when required and floats told to.
The problem has to be somewhere else ; as long as you feed cat with same format there shouldn't be any error.
The error comes from a function called Plot_attempt_9_6_22_1 right?
Either
- you are sending to
plotrows instead of columns thereforeplotinput vectors contain mixed formats, a transpose or more than one missing somewhere,
or
- somewhere your are mixing date/time formats with something that is not date/time type.
2.- Do you really need to preallocate memory?
In any case MATLAB suggests to preallocate tables with the following expression :
T = table('Size',sz,'VariableTypes',varTypes)
not with the C/C style mentioned in question
If you find this answer useful would you please be so kind to consider marking it as good answer? Thanks anyway for reading my answer.
CodePudding user response：
I ended up fixing this problem by adding an if statement, and now, it looks something like this:
if k == 1
timecol_tot = timecol_array;
EZOCO2col_tot = cat(1, EZOCO2col_tot, EZOCO2col_double);
EZOO2col_tot = cat(1, EZOO2col_tot, EZOO2col_double);
SGP30CO2col_tot = cat(1, SGP30CO2col_tot, SGP30CO2col_array);
SGP30TVOCcol_tot = cat(1, SGP30TVOCcol_tot, SGP30TVOCcol_array);
BME280Humcol_tot = cat(1, BME280Humcol_tot, BME280Humcol_array);
BME280Presscol_tot = cat(1, BME280Presscol_tot, BME280Presscol_array);
BME280Tempcol_tot = cat(1, BME280Tempcol_tot, BME280Tempcol_array);
else
timecol_tot = cat(1, timecol_tot, timecol_array);
EZOCO2col_tot = cat(1, EZOCO2col_tot, EZOCO2col_double);
EZOO2col_tot = cat(1, EZOO2col_tot, EZOO2col_double);
SGP30CO2col_tot = cat(1, SGP30CO2col_tot, SGP30CO2col_array);
SGP30TVOCcol_tot = cat(1, SGP30TVOCcol_tot, SGP30TVOCcol_array);
BME280Humcol_tot = cat(1, BME280Humcol_tot, BME280Humcol_array);
BME280Presscol_tot = cat(1, BME280Presscol_tot, BME280Presscol_array);
BME280Tempcol_tot = cat(1, BME280Tempcol_tot, BME280Tempcol_array);
end
Another reason that I was getting a type error was because some of my files were empty or corrupted. If you're having similar issues, check to make sure that you're not working with empty or corrupted csv's!