looping over a list of file names in Matlab - matlab

Probably a really simple question - but I can't seem to figure it out for Matlab. I would like to import data from a list of files, and save the resulting matrices with a name derived from the original file name. There's quite a few files, so I would like to use a for loop.
In BASH I would write something like:
For sample in apple orange guava jackfruit;
do
"$sample"_matrix = someimportfunction("$sample".txt);
done
I can import the files one at a time with dlmread, I just can't figure out how to loop over the names, sort of the matlab equivelent of $.
Many thanks for any suggestions!

I think the code snippet below may do exactly what you want.
Of course, you need the files apple.txt etcetera, with numbers in them
for sample = {'apple', 'orange', 'guava', 'jackfruit'}
matrix.(sample{1}) = load([sample{1},'.txt']);
end
matrix = matrix
I get the following output:
matrix =
scalar structure containing the fields:
apple = 1 2 3
orange =
1 2
4 5
guava =
1 1 1
0 0 0
jackfruit = 17

Let's say your files are in the folder "sfolder".
Files = dir('sfolder');
num_files = length(Files);
for i=1:num_files
fid = fopen(Files(i).name); %do whatever you want now
end
This will help you go through each file in a particular directory.

Related

Concatenating 219 single entry CSV files into one Matlab Variable

--A bit of an update, managed to find a way around this on python. --
I am attempting to concatenate 219 single entry CSV files from a previous export into one Matlab variable file (MAT, CSV - I know how to interchange file formats well enough I'll take what I get on this for the process). The files were written in '10' 'precision' in 'dlmwrite' in a 'for' loop writing one value per file. I need to be able to combine the files, in an indexed order (from 1:73 per 1:3 variable blocks) to produce 3 master variables for statistics/plotting (e.g., usefulness).
Input try 1
for k = 1:73
FileName0 = sprintf('D:/xxxx/yyy%dy.csv',k);
xx = dlmread(FileName0);
end
Input try 2
for k = 1:73
FileName0 = sprintf('D:/xxxx/yyy%dy.csv',k);
xx = dlmread(FileName0);
if (rem(k,1)==0)
xx2 = [xx2,x];
end
Prospective Output 1 (for each variable - fyi, just need to know how to run once, have code setup otherwise).
size(xx(k,1)) = [73 1] %just informative about the desired size
dlmwrite('D:/xxxx/yyy.csv',xx,'precision',10)
Prospective Output 2 (for each variable - fyi, just need to know how to run once, have code setup otherwise).
size(xx2(k,1)) = [73 1] %just informative about the desired size
dlmwrite('D:/xxxx/yyy.csv',xx2,'precision',10)

Matlab-reading large netcdf files

I have a 17G netcdf file that I am trying to use for analysis. Each variable in the netcdf file is set up like: variable(x,y,z,time). I would like to read in and analyze the variables one 'time' at a time for analysis in Matlab. In other words, I want to use all x, y, and z points at one time. In the past I have had smaller files so reading in a variable has been set up like
fid=netcdf.open('filename/location','NC_NOWRITE');
var_id=netcdf.inqVarID(fid,'varname');
var=netcdf.getVar(fid,var_id);
Is it possible to read in the variables using one time step when the variable is read in? (Incorrect syntax) It'd essentially look like
var=netcdf.getVar(fid,var_id,[:,:,:,time_index]);
Yes, the matlab netcdf command supports this, almost the way you wrote it:
data = netcdf.getVar(fid,var_id,var_index,var_length)
See the matlab documentation for more information. You can also use high-level matlab commands instead of the netCDF library functions.
For example, if varname is a 100x4 array, you could get row 7 by using:
% read 4 columns from 1 row of data starting at row 7, column 1
v = ncread('filename/location','varname',[7 1],[1 4]);
or a four-dimensional array, as in the question:
% read all data from dim. 1-3 at dim 4 = 27
v = ncread('filename/location','varname',[1 1 1 27],[Inf Inf Inf 1]);

Loading multiple .mat files in the workspace

I want to load multiple .mat files (around 500 in number) into my workspace.
The files are named as
omni_AP1_trial_1_loc_1.mat
omni_AP1_trial_1_loc_2.mat
omni_AP1_trial_1_loc_3.mat
.
.
omni_AP1_trial_1_loc_57.mat
.
.
omni_AP1_trial_10_loc_1.mat
omni_AP1_trial_10_loc_2.mat
omni_AP1_trial_10_loc_3.mat
.
.
omni_AP1_trial_10_loc_57.mat
I am using the given code :
files_1 = dir('omni_AP1_trial_*_loc_1.mat');
NumberOfDataset = length(files_1);
for i = 1:NumberOfDataset
%get allfiles matching the pattern 'dataset(i)_*'
files = dir(sprintf('omni_AP1_trial_%d_loc_*.mat',i));
for j = 1:length(files)
fprintf('Current file : %s\n',files(j).name)
a= load(files(j).name);
end
end
During execution even though the fprintf statement shows consecutive files being picked, but the structure a holds only the last file that was picked up and the previous file is being overwritten when the loop iterates.
How can I have all the files loaded together in the workspace ? Please help.
You can create an array of structs for your results in the iteration:
a = []; % create empty array
files_1 = dir('omni_AP1_trial_*_loc_1.mat');
NumberOfDataset = length(files_1);
for i = 1:NumberOfDataset
%get allfiles matching the pattern 'dataset(i)_*'
files = dir(sprintf('omni_AP1_trial_%d_loc_*.mat',i));
for j = 1:length(files)
fprintf('Current file : %s\n',files(j).name)
a(end+1)= load(files(j).name); % store data in struct array
end
end
Your a value is being overwritten each time, but you can have an array of structures:
a(j) = load(files(j).name);
The next question is usually how to index the same subelement in multiple struct array elements. If subelement is a scalar field of the variable that's stored in the files (and if that variable has the same name in each file) you can do
[a(:).variablename.subelement]
In order to make it simpler I tried using
for jj=1:57
for ii = 1: 10
a(ii,jj) =load(['omni_AP1_trial_' num2str(ii) '_loc_' num2str(jj)'.mat']);
end
end
I have a total of 57 locations which are mentioned as loc_1 to loc_57 and for each location I am taking 10 trials and these are mentioned like trial_1_loc_1, trial_2_loc_1 upto trial_10_loc_1.
In the above code when I don't use the first loop (ie, jj) and when I am replacing num2str(jj) by 1 (ie location_1), then in that case I am getting 10 trials for location 1. When I change that to 2, then I am getting 10 trials for location 2 and so on.
So when I used one loop only, it worked and the code is:
for ii = 1: 10
a(ii) =load(['omni_AP1_trial_' num2str(ii) '_loc_1.mat']);
end
The above code worked fine. So now I am thinking to use 2 loops one for the location (jj loop) and another for the number of trials (ii loop). I want to use the loop in such a way that for my first location,it will load 10 trials and for the next it will load another 10 trials and so on. When I am using both the loops, I am getting an error (Unexpected matlab operator). Don't know what is wrong. I am not sure the logic I am using is correct or not. If the logic is correct then what is the problem on the above code when I am using 2 loops to load all the files?

creating a structure within a structure from imported text files in matlab

I'm not sure if anyone can help with this question but here we go. I have 4 folders where each folder contains data for different locations, within the folders I have 8 .txt files which represent the measured variables at each location (i.e. same variables measured in each location). I'm trying to import these into matlab and list the measured variables in astructure so they can be compared and plotted against one another afterwards (without doing this they will over write one another).
I've written a script for importing these into matlab, the script works but not exactly in the way I want it to, the script is as follows:
clear all
pathName = 'E:\University\CEH Lancaster\Project\LA practice\final files';
FolderListing = dir(pathName);
FolderListing = FolderListing(3:end);
%lists the folder in the directory specified by pathName
for i = 1:length(FolderListing);
LName{i} = (FolderListing(i).name);
%obtains the name of each folder
end
for i = 1:length(LName)
TopFolder{i} = fullfile(pathName,LName{i});
%path for each individual folder
dirListing{i} = dir(fullfile(TopFolder{i},'*.txt'));
%list of the .txt files
for ii = 1:length(dirListing{1,1});
fileToRead1{1,i}{ii,1} = (dirListing{1,i}(ii,1).name);
%name of the .txt files in the TopFolder
end
end
for i = 1:length(fileToRead1);
for ii = 1:length(fileToRead1{1});
fid{1,i}{ii,1} = fopen((fullfile(TopFolder{1,i},fileToRead1{1,i}{ii,1})));
%open the files specified by fileToRead prior to importing the data
%into matlab
data{1,i}{ii,1} = textscan(fid{1,i}{ii,1},'%f');
%import the data into matlab
[~,name{1,i}{ii,1}] = fileparts(fileToRead1{1,i}{ii,1});
%obtain the name of each of the variables
Location.(LName{i}).(genvarname(name{1,i}{ii,1})) = data{1,i}{ii,1};
%create a strucutre for the individual locations and the
%variables.
end
end
The problem lies in the final outcome where instead of having Location.Name and then the list of variables, I have Location.Name.variables, which doesn't seem necessary. I realise that its due to the way I've written the last line of the script but I can't seem to change it without it producing an error. Any advice you could give on the problem or on the script in general would be much appreciated.
I think cell2mat is the function you want for this purpose. Here's my usage, see if it fits your needs:
tt = {ones(1,100)};
tt
tt =
[1x100 double]
cell2mat(tt)
ans =
Columns 1 through 15
1 1 1 1 1 1 1 1 1 1 1 1...

Reading multiple .csv files into Matlab while organizing them into new matrices

I have lots of .csv files that I want to read into MATLAB, but doing some organization along the way
My first problem is my data looks like this:
[...
file1
ex1
6;0
8;0
9;1
file1
ex2
7;0
8;1
3;2
file1
ex3
7;0
8;1
3;2
The import wizard on MATLAB for some reason only takes the first header text then the data set below that and throws away everything when it reaches the next text header. So how can I organize the file so that it looks like this instead?
[...
file1......file1.....file1
ex1.......ex2.......ex3
6;0.......7;0.......7;0
8;0.......8;1.......8;1
9;1.......3;2.......3;2
NOTE: the number of rows for the different ex's is always different, so you can't just spilt the file into regular chunks.
My second problem is then to compare the same experiments from different files. So I want to take the columns below "ex1" from all the different files and line then up horizontally against each other in a new matrix. So that it looks like this:
file1.....file2.....file3.....
ex1.......ex1.......ex1.......
6;0.......6;0.......6;0.......
8;0.......8;0.......8;0.......
9;1.......9;1.......9;1.......
NOTE: the ex's in the different files are in different orders. I need to match up ex's within files based on matching one of the lines of header (e.g. whenever it is called 'track1').
EDIT:
This is how the actual data looks like.
Since the number of rows in each ex is different, you must use a cell matrix.
file = 'file1.csv';
h = 2; % # header lines
num_ex = 3;
r = h; % track the current row in the file
data = cell(1,num_ex);
for i=1:num_ex
s = importdata(file, ';', r);
x = s.data;
data{i} = x;
r = r + size(x,1) + h;
end
Then you can access your data using the curly bracket cell matrix notation
ex = 2;
x = data{ex};
So you get
x = [ 7 0
8 1
3 2 ]
For your second problem, you can add a loop to go through each file
filenames = {'file1.csv', 'file2.csv', 'file3.csv'};
h = 2; % # header lines
num_ex = 3;
r = h; % track the current row in the file
data = cell(1,num_ex);
for i=1:num_ex
for f=1:length(filenames)
file = filenames{f};
s = importdata(file, ';', r);
x = s.data;
data{i} = [data{i} x];
end
r = r + size(x,1) + h;
end
So that data{1} has all the data for ex 1, etc.
While I recognise this is not a full solution to your problem, an alternate solution I often use to overcome text editing (and the horrific speed penalty you get when parsing in MATLAB) is to load your data in through the MATLAB connectors to Java or C#.
It's fairly easy to call C# and Java from MATLAB, and I do a lot of my text stuff that was.

Resources