Michael Ian Lapsley

The Old Way to Load DataSets

Lots of my time consists of analyzing huge sets of files recorded by and oscilloscope or some other device. Typically, I would try to use the same name for each file with a number at the end that increase each time. Then I can write a loop to load all of the files, then do analysis and create plots. This causes its own problems for several reasons such as: several loops must be used for different regions of the data (tek0000n for n = 1-9; tek000nn for nn = 10-99; tek00nnn for nnn = 100-999; etc) and broken loops when a single file in the data set is a different length or mission, and many other issues. Such a sustem looks like this where many conditional loops are used to fix the problem:

if m <10 & n < 10

   for i = 1:n

       Temp = dlmread([File,'/',File,'-0000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

end

if m <10 & n < 100 & n > 10

   for i = 1:9

       Temp = dlmread([File,'/',File,'-0000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

           

   for i = 10:n

       Temp = dlmread([File,'/',File,'-000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

end

       

if m <10 & n < 1000 & n > 100

   for i = 1:9

       Temp = dlmread([File,'/',File,'-0000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

           

   for i = 10:99

       Temp = dlmread([File,'/',File,'-000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

           

   for i = 100:n

       Temp = dlmread([File,'/',File,'-00',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

end

       

if m > 10 & n < 1000 & n > 100

   for i = m:99

       Temp = dlmread([File,'/',File,'-000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

           

   for i = 100:n

        Temp = dlmread([File,'/',File,'-00',num2str(i),'.txt'],'\t',[17 0 3664 1]);

        Frequency(:,i) = Temp(:,1);

        Absorption(:,i) = Temp(:,2);

   end

end

       

if m < 10 & n > 1000

   for i = 1:9

       Temp = dlmread([File,'/',File,'-0000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

          

   for i = 10:99

       Temp = dlmread([File,'/',File,'-000',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end

           

   for i = 100:999

       Temp = dlmread([File,'/',File,'-00',num2str(i),'.txt'],'\t',[17 0 3664 1]);

       Frequency(:,i) = Temp(:,1);

       Absorption(:,i) = Temp(:,2);

   end  
end

Read Directory and Load

Recently, google helped me discover a better method where you can actually tell matlab to read and record the files names in a specific folder. The, so long as all the data is the same length, a single loop can be used to import all of the data in a single folder. Here is an example:

clear;clc;close all

cd ..
Temp = dir('Data');
Temp2 = {Temp.name};
FileName = Temp2(3:end);

for i = 1:length(FileName)
    Temp = dlmread(['Data/',char(FileName(i))],',',[14 0 10013 2]);
    time = Temp(:,1);
    ch1 = Temp(:,2);
    ch2 = Temp(:,3);
    save(['[Matlab]/MData/DataSet',num2str(i),'.mat'],'time','ch1','ch2')

end

return

The dir command reads the folder \Data, and a structure cell is created with the information for each file. Since I only care about the name, that cell can be extracted and trimmed. Then a single loop can be used to load all of the data into a useful format. To prevent errors if the files are not the same length, each data set is save in its own .mat file. These files can be quickly loaded in Matlab for later use.

Loading several .mat files

Once all of the data is stored in separate .mat files, this data can be easily accessed and plotted. It one were to plot the first 5 data sets that were loaded above, this code would perform the operation:

for i = [1:5]
    Str = ['MData\DataSet',num2str(i),'.mat'];
    load(Str)
    eval(['time_',num2str(i),' = time;'])
    eval(['ch1_',num2str(i),' = ch1;'])
    eval(['ch2_',num2str(i),' = ch2;'])

end

figure

hold on
        

for i = 1:5
    eval(['plot(time_',num2str(i),',ch1_',num2str(i),')'])

end

hold off

I realize that there are many non standard matlab functions here, but I encourage people to learn these functions. They have been very useful for me in performing some more advanced actions.