9 views (last 30 days)
Show older comments
Daphne PARLIARI on 24 Jan 2020
-
-
Link
Direct link to this question
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table
Commented: Guillaume on 27 Jan 2020
Accepted Answer: Guillaume
- Stations coordinates.txt
- THESS_Temp.csv
Open in MATLAB Online
Hi all!
I have a list of stations (see .txt) and a .csv (see attached) with several data. What I need to do is scan the entire .txt and for every station, eg. Airport, to go and scan column 14 of the .csv. If the station is the same, I want to do several things, eg. extract the date (column 2) and Temperature (column 15) every time Airport appears.
I tried with eq but it doesn't work.
list1 = rdir ('C:\Projects\LIFE ASTI\C.3\C.3\Weather station data\from desktop\Obs. data 4 Keppas\');
wrfdir = 'C:\Projects\LIFE ASTI\C.3\C.3\Weather station data\from desktop\2019 run\Thess\*.csv';
stations = readtable('C:\Projects\LIFE ASTI\C.3\C.3\Weather station data\from desktop\Stations coordinates.txt');
output_path='C:\Projects\LIFE ASTI\C.3\C.3\Weather station data\from desktop\OutputSimulations';
WRF_Data = readtable('C:\Projects\LIFE ASTI\C.3\C.3\Weather station data\from desktop\2019 run\Thess\THESS_Temp.csv');
% Variables
vars={'Temperature'; 'Relative humidity'};
years = {'2015'; '2019'};
varunits={'Celsius'; '%' };
%Starting loop for all files
for i=1:size(stations,1)
for k = 1:size(WRF_Data.Station,1)
if eq(stations(i,1) , WRF_Data.Station(k,1))
KEEP TEMPERATURE OR WHATEVER...
end
end
end
0 Comments Show -2 older commentsHide -2 older comments
Show -2 older commentsHide -2 older comments
Sign in to comment.
Sign in to answer this question.
Accepted Answer
Guillaume on 24 Jan 2020
Open in MATLAB Online
I'm not entirely clear what the ultimate goal is but one thing for sure, you don't need a loop at all.
To keep only the rows of WRF_Data that have a station in stations AND append the rest of the station variables to these rows, it's simply:
merged = innerjoin(WRF_Data, stations, 'Keys', 'Station'); %And you don't even need the 'Keys', 'stations' although it makes the code clearer.
If you don't want all the variables in the output, you can tell innerjoin which one you want:
merged = innerjoin(WRF_Data, stations, 'LeftVariables', {'Station', 'Var1', 'Temp'}, 'RightVariables', {'Lat', 'Lon'}) %for example
8 Comments Show 6 older commentsHide 6 older comments
Show 6 older commentsHide 6 older comments
Daphne PARLIARI on 25 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_789849
Thank you for your answer!
What I need to do in the end, is end up with one .csv (or whatever) for every station that contains the respective data for that station only. In Stations Coordinates.txt you can see that I have 16 stations, therefore I must end up with 16 files which will have hourly data (temperature or whatever) plus everything else, for each station.
I tried merged as you suggested but I get
Error using tabular/innerjoin (line 98)
Left and right key variables 'Station' and 'Station' are not comparable because one is a non-cell.
Error in WRF_Simulations (line 17)
merged = innerjoin(WRF_Data, stations, 'Keys', 'Station')
Guillaume on 25 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_789859
Open in MATLAB Online
I have no idea how you imported your data, the following works fine (R2019b) for me with your input files
stations = readtable('Stations coordinates.txt');
%use detectImportOptions to fix wonky date import of the csv file. Encoding of the dates in that csv file is ... less than ideal!
opts = detectImportOptions('THESS_Temp.csv');
opts = opts.setvaropts('Date', 'InputFormat', 'dd-MM-yy H:mm', 'DatetimeFormat', 'dd-MMM-uuuu HH:mm');
WRF_Data = readtable('THESS_Temp.csv', opts);
%then
merged = innerjoin(WRF_Data, stations, 'Keys', 'Station');
Again, the function you want is innerjoin, not ismember in a loop which just complicates things (and is most likely slower)
Note that your demo files have only one station in common 'Airport'
To then split and save merged per station:
[group, stations] = findgroups(merged.Station);
splitapply(@(rows, statidx) writetable(merged(rows, :), sprintf('%s.csv', stations{statidx(1)})), (1:height(merged))', group, group);
Daphne PARLIARI on 25 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_789861
I am running on 2019a, I don't know if the problem lies there....
Guillaume on 25 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_789863
Edited: Guillaume on 25 Jan 2020
Doesn't the above work exactly as written?
I suspect you're getting your innerjoin error because in one table Station is a string array and in the other a cell array of char vectors. Converting one to the other would fix the problem.
Daphne PARLIARI on 25 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_789871
Edited: Daphne PARLIARI on 25 Jan 2020
I used
WRF_Data.Station = cellstr(WRF_Data.Station);
and indeed it fixed the compatibility problem.
It works like a doll, million thanks!
The only thing remaing now, is that the split files must go to respective folders so I can detect them easier. With the code I have so far, they are saved to the current dir...
Guillaume on 27 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_790350
Open in MATLAB Online
I recommend you use the import code I wrote.
You can modify the anonymous function in the splitapply code to do whatever you want, eg. to save into individual directories named after the station:
rootpath = 'C:\somewhere\somefolder';
[group, stations] = findgroups(merged.Station);
splitapply(@(rows, statidx) writetable(merged(rows, :), fullfile(rootpath, stations{statidx(1)}, 'station.csv')), (1:height(merged))', group, group);
Or if it's too complex, use a loop:
rootpath = 'C:\somewhere\somefolder';
[group, stations] = findgroups(merged.Station);
for gidx = 1:numel(stations) %iterate over each unique station group
writetable(merged(group = gidx, :), fullfile(rootpath, stations{gidx}, 'station.csv'));
end
Daphne PARLIARI on 27 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_790355
Great, thank you!
If you have time, could you please explain to me what each of line
splitapply(@(rows, statidx) writetable(merged(rows, :), sprintf('%s.csv', stations{statidx(1)})), (1:height(merged))', group, group);
does?
Guillaume on 27 Jan 2020
Direct link to this comment
https://support.mathworks.com/matlabcentral/answers/501815-elements-of-table-equal-to-elements-of-another-table#comment_790401
Open in MATLAB Online
splitapply split the input arrays into groups and apply the same function to each group, that is it takes the rows of each input array corresponding to a group and pass that to the function, then it passes the rows corresponding to the next group, etc.
The groups are defined by the last input to splitapply, in this case group. The two arrays that I've decided would be split are (1:height(merged))' which is basically the row indices of the table and group as I actually need the group index to be passed to the processing function. The processing function is the anonymous function: @(rows, statidx) writetable(merged(rows, :), sprintf('%s.csv', stations{statidx(1)})). This is a function with two inputs rows and statidx. rows receives in turn the elements of the first input of splitapply: (1:height(merged))', that is it receives the row indices of the table corresponding to a group (a station), statidx receives in turn the elements of the 2nd input, group corresponding to a group, so it's a vector of identical numbers, hence why I use statidx(1) in the function body.
The function body writes the sekect rows of the table to a file whose name is the station. If we were to rewrite the anonymous function, it would be more or less
function anonfun(rows, statidx)
%rows: row indices of the table corresponding to a single station. Column vector
%statidx: column vector of identical values containing the station/group number
%Unlike normal function, anonymous functions can capture variable from the enclosing workspace
%Here, it also captures
%merged: the table
%stations: list of unique station names, ordered by group number
subtable = merged(rows, :); %extract portion of the table corresponding to station
currentstation = stations{statidx(1)};
filename = sprintf('%s.csv', currentstation);
writetable(subtable, filename);
end
And if you were to write what splitapply does as a loop, it would be (in this case):
input1 = (1:height(merged))'; %row indices of the table
input2 = group;
for g = 1:max(group)
thisgroup_input1 = input1(group == g);
thisgroup_input2 = input2(group == g);
anonfun(thisgroup_input1, thisgroup_input2); %anonfun also captures merged and stations without needing to pass them as arguments
end
Sign in to comment.
More Answers (1)
Philippe Lebel on 24 Jan 2020
Edited: Philippe Lebel on 24 Jan 2020
Open in MATLAB Online
Here you go:
I loop over all station names and check where in the data table i find these names. I generate a boolean index in order to fetch data from the table and store them in a structure.
clear
stations = readtable('\Stations coordinates.txt');
WRF_Data = readtable('\THESS_Temp.csv');
%Starting loop for all files
for i=1:size(stations,1)
WRF_Data_station_mask = ismember(WRF_Data(:,14),stations(i,1));
sorted_values(i).name = char(stations{i,1});
sorted_values(i).temps = WRF_Data(WRF_Data_station_mask,15);
end
0 Comments Show -2 older commentsHide -2 older comments
Show -2 older commentsHide -2 older comments
Sign in to comment.
Sign in to answer this question.
See Also
Categories
MATLABLanguage FundamentalsData TypesTables
Find more on Tables in Help Center and File Exchange
Tags
- table
- equal
- elements
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
An Error Occurred
Unable to complete the action because of changes made to the page. Reload the page to see its updated state.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- Deutsch
- English
- Français
- United Kingdom(English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)
Contact your local office