SGENGCAT Combine data from several Segalider eng files.
Syntax:
[META, DATA] = SGENGCAT(META_LIST, DATA_LIST)
[META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPTIONS)
[META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPT1, VAL1, ...)
Description:
[META, DATA] = SGENGCAT(META_LIST, DATA_LIST) combines metadata in
struct array META_LIST and data in cell or struct array DATA_LIST into
a single data set with metadata in struct META and data in struct DATA.
Respective elements in META_LIST and DATA_LIST should be in the format
returned by function SGENG2MAT, but do not need to have the same parameters
or variables. Outputs META and DATA are in the same format, too, and gather
the input data sorted according to the mission number and the dive number.
META is a struct array combining the information in elements of META_LIST,
ordered according to the mission number and the dive number.
It has the following fields:
HEADERS: struct array with all eng headers.
This is the concatenation of the HEADERS field of all elements in
META_LIST.
START_SECS: number with the reference time for timestamped lines (start
time of first dive as seconds since 1970 January 01 00:00:00 UTC).
COLUMNS: string cell array with the names of the columns in the returned
data array (in the same column order as the data).
SOURCES: string cell array containing FILENAME.
DATA is an array or a struct combining the data in DATA_LIST, ordered
according to the mission number and the dive number, and with the time
fields as seconds since the start time of the first dive.
[META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPTIONS) and
[META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPT1, VAL1, ...)
accept the following options given in key-value pairs OPT1, VAL1...
or in a struct OPTIONS with field names as option keys and field values
as option values:
FORMAT: data output format.
String setting the format of the output DATA. Valid values are:
'array': DATA is a matrix with data readings as columns ordered
as in the COLUMNS metadata field.
'struct': DATA is a struct with column names as field names
and column vectors of data columns as field values.
Default value: 'array'
COLUMNS: data column filtering list.
String cell array with the names of the data columns of interest.
If given, only parameters present in both the input list and this list
will be present in output. The string 'all' may also be given,
in which case column filtering is not performed and all columns
in the input list will be present in output.
Default value: 'all' (do not perform column filtering).
PERIOD: time filtering boundaries.
Two element numeric array with the start and the end of the period
of interest (seconds since 1970-01-01 00:00:00.00 UTC). If given,
only data from dives with start time within this period will be
present in output. The string 'all' may also be given, in which case
time filtering is not performed and data from all dives will be
present in output.
Default value: 'all' (do not perform time filtering).
Examples:
[meta, data] = sgengcat(meta_list, data_list)
See also:
SGENG2MAT
SGLOG2MAT
SGLOGCAT
SGLOGENGMERGE
Authors:
Joan Pau Beltran <joanpau.beltran@socib.cat>0001 function [meta, data] = sgengcat(meta_list, data_list, varargin) 0002 %SGENGCAT Combine data from several Segalider eng files. 0003 % 0004 % Syntax: 0005 % [META, DATA] = SGENGCAT(META_LIST, DATA_LIST) 0006 % [META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPTIONS) 0007 % [META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPT1, VAL1, ...) 0008 % 0009 % Description: 0010 % [META, DATA] = SGENGCAT(META_LIST, DATA_LIST) combines metadata in 0011 % struct array META_LIST and data in cell or struct array DATA_LIST into 0012 % a single data set with metadata in struct META and data in struct DATA. 0013 % Respective elements in META_LIST and DATA_LIST should be in the format 0014 % returned by function SGENG2MAT, but do not need to have the same parameters 0015 % or variables. Outputs META and DATA are in the same format, too, and gather 0016 % the input data sorted according to the mission number and the dive number. 0017 % 0018 % META is a struct array combining the information in elements of META_LIST, 0019 % ordered according to the mission number and the dive number. 0020 % It has the following fields: 0021 % HEADERS: struct array with all eng headers. 0022 % This is the concatenation of the HEADERS field of all elements in 0023 % META_LIST. 0024 % START_SECS: number with the reference time for timestamped lines (start 0025 % time of first dive as seconds since 1970 January 01 00:00:00 UTC). 0026 % COLUMNS: string cell array with the names of the columns in the returned 0027 % data array (in the same column order as the data). 0028 % SOURCES: string cell array containing FILENAME. 0029 % 0030 % DATA is an array or a struct combining the data in DATA_LIST, ordered 0031 % according to the mission number and the dive number, and with the time 0032 % fields as seconds since the start time of the first dive. 0033 % 0034 % [META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPTIONS) and 0035 % [META, DATA] = SGENGCAT(META_LIST, DATA_LIST, OPT1, VAL1, ...) 0036 % accept the following options given in key-value pairs OPT1, VAL1... 0037 % or in a struct OPTIONS with field names as option keys and field values 0038 % as option values: 0039 % FORMAT: data output format. 0040 % String setting the format of the output DATA. Valid values are: 0041 % 'array': DATA is a matrix with data readings as columns ordered 0042 % as in the COLUMNS metadata field. 0043 % 'struct': DATA is a struct with column names as field names 0044 % and column vectors of data columns as field values. 0045 % Default value: 'array' 0046 % COLUMNS: data column filtering list. 0047 % String cell array with the names of the data columns of interest. 0048 % If given, only parameters present in both the input list and this list 0049 % will be present in output. The string 'all' may also be given, 0050 % in which case column filtering is not performed and all columns 0051 % in the input list will be present in output. 0052 % Default value: 'all' (do not perform column filtering). 0053 % PERIOD: time filtering boundaries. 0054 % Two element numeric array with the start and the end of the period 0055 % of interest (seconds since 1970-01-01 00:00:00.00 UTC). If given, 0056 % only data from dives with start time within this period will be 0057 % present in output. The string 'all' may also be given, in which case 0058 % time filtering is not performed and data from all dives will be 0059 % present in output. 0060 % Default value: 'all' (do not perform time filtering). 0061 % 0062 % Examples: 0063 % [meta, data] = sgengcat(meta_list, data_list) 0064 % 0065 % See also: 0066 % SGENG2MAT 0067 % SGLOG2MAT 0068 % SGLOGCAT 0069 % SGLOGENGMERGE 0070 % 0071 % Authors: 0072 % Joan Pau Beltran <joanpau.beltran@socib.cat> 0073 0074 % Copyright (C) 2013-2016 0075 % ICTS SOCIB - Servei d'observacio i prediccio costaner de les Illes Balears 0076 % <http://www.socib.es> 0077 % 0078 % This program is free software: you can redistribute it and/or modify 0079 % it under the terms of the GNU General Public License as published by 0080 % the Free Software Foundation, either version 3 of the License, or 0081 % (at your option) any later version. 0082 % 0083 % This program is distributed in the hope that it will be useful, 0084 % but WITHOUT ANY WARRANTY; without even the implied warranty of 0085 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 0086 % GNU General Public License for more details. 0087 % 0088 % You should have received a copy of the GNU General Public License 0089 % along with this program. If not, see <http://www.gnu.org/licenses/>. 0090 0091 error(nargchk(2, 8, nargin, 'struct')); 0092 0093 0094 %% Set options and default values. 0095 options.format = 'array'; 0096 options.columns = 'all'; 0097 options.period = 'all'; 0098 0099 0100 %% Parse optional arguments. 0101 % Get option key-value pairs in any accepted call signature. 0102 argopts = varargin; 0103 if isscalar(argopts) && isstruct(argopts{1}) 0104 % Options passed as a single option struct argument: 0105 % field names are option keys and field values are option values. 0106 opt_key_list = fieldnames(argopts{1}); 0107 opt_val_list = struct2cell(argopts{1}); 0108 elseif mod(numel(argopts), 2) == 0 0109 % Options passed as key-value argument pairs. 0110 opt_key_list = argopts(1:2:end); 0111 opt_val_list = argopts(2:2:end); 0112 else 0113 error('glider_toolbox:sgengcat:InvalidOptions', ... 0114 'Invalid optional arguments (neither key-value pairs nor struct).'); 0115 end 0116 % Overwrite default options with values given in extra arguments. 0117 for opt_idx = 1:numel(opt_key_list) 0118 opt = lower(opt_key_list{opt_idx}); 0119 val = opt_val_list{opt_idx}; 0120 if isfield(options, opt) 0121 options.(opt) = val; 0122 else 0123 error('glider_toolbox:sgengcat:InvalidOption', ... 0124 'Invalid option: %s.', opt); 0125 end 0126 end 0127 0128 0129 %% Set option flags and values. 0130 output_format = lower(options.format); 0131 column_filter = true; 0132 column_list = cellstr(options.columns); 0133 start_filter = true; 0134 start_range = options.period; 0135 if ischar(options.columns) && strcmp(options.columns, 'all') 0136 column_filter = false; 0137 end 0138 if ischar(options.period) && strcmp(options.period, 'all') 0139 start_filter = false; 0140 end 0141 0142 0143 %% Check input formats. 0144 % Be gentle and accept emtpy inputs and struct or cell arrays. 0145 % Also, ensure column layout. 0146 if isempty(meta_list) 0147 meta_list = struct(); 0148 meta_list.sources = {}; 0149 meta_list.headers = struct('version', {}, 'glider', {}, 'mission', {}, ... 0150 'dive', {}, 'basestation_version', {}, ... 0151 'start', {}, 'columns', {}); 0152 meta_list.columns = {}; 0153 meta_list.start_secs = []; 0154 elseif iscell(meta_list) 0155 meta_list = vertcat(meta_list{:}); 0156 end 0157 if isstruct(data_list) 0158 data_list = num2cell(data_list(:)); % struct array to cell array conversion. 0159 else 0160 data_list = data_list(:); 0161 end 0162 0163 0164 %% Filter and sort input data sets. 0165 % Inputs are filtered according to start time 0166 % and sorted according to mission number and to dive number. 0167 all_sources = vertcat(meta_list.sources); 0168 all_headers = vertcat(meta_list.headers); 0169 all_start_secs = vertcat(meta_list.start_secs); 0170 all_columns = {meta_list.columns}'; 0171 % Filter data columns. 0172 outcol_list = unique(vertcat(all_columns{:})); 0173 column_filtering = true(size(outcol_list)); 0174 if column_filter 0175 column_filtering = ismember(outcol_list, column_list); 0176 end 0177 outcol_list = outcol_list(column_filtering); 0178 % Sort according mission and dive number (using a virtual rank index). 0179 all_dive_nums = vertcat(all_headers.dive); 0180 all_miss_nums = vertcat(all_headers.mission); 0181 [~, miss_dive_sorting] = ... 0182 sort( (all_dive_nums - min(all_dive_nums)) ... 0183 + (all_miss_nums - min(all_miss_nums)) ... 0184 * (max(all_dive_nums) - min(all_dive_nums) + 1)); 0185 % Filter dives out of the period of interest. 0186 time_filtering = true(size(meta_list)); 0187 if start_filter 0188 time_filtering = ... 0189 start_range(1) <= all_start_secs & all_start_secs <= start_range(2); 0190 end 0191 % Sort and filter at once. 0192 time_filtering_and_miss_dive_sorting = ... 0193 miss_dive_sorting(time_filtering(miss_dive_sorting)); 0194 header_list = all_headers(time_filtering_and_miss_dive_sorting, :); 0195 source_list = all_sources(time_filtering_and_miss_dive_sorting, :); 0196 starts_list = all_start_secs(time_filtering_and_miss_dive_sorting, :); 0197 datcol_list = all_columns(time_filtering_and_miss_dive_sorting, :); 0198 values_list = data_list(time_filtering_and_miss_dive_sorting, :); 0199 0200 0201 %% Reorder data columns and correct time offset converting to array if needed. 0202 time_column = 'elaps_t'; 0203 dive_start_offset_list = starts_list - min(starts_list); 0204 outdat_list = cell(size(values_list)); 0205 for data_idx = 1:numel(values_list) 0206 values = values_list{data_idx}; 0207 offset = dive_start_offset_list(data_idx); 0208 datcol = datcol_list{data_idx}; 0209 [col_present, col_indices] = ismember(outcol_list, datcol); 0210 time_select = strcmp(time_column, datcol); 0211 if isstruct(values) 0212 values = cell2mat(struct2cell(values)'); 0213 end 0214 values(:, time_select) = values(:, time_select) + offset; 0215 outdat_list{data_idx} = nan(size(values, 1), numel(outcol_list)); 0216 outdat_list{data_idx}(:, col_present) = values(:, col_indices(col_present)); 0217 end 0218 0219 0220 %% Cat metadata and data. 0221 meta = struct(); 0222 meta.sources = source_list; 0223 meta.headers = header_list; 0224 meta.start_secs = min(starts_list); 0225 meta.columns = outcol_list; 0226 data = vertcat(outdat_list{:}); 0227 0228 0229 %% Convert output data to struct format if needed. 0230 switch output_format 0231 case 'array' 0232 case 'struct' 0233 data = cell2struct(num2cell(data, 1), meta.columns, 2); 0234 otherwise 0235 error('glider_toolbox:sgengcat:InvalidFormat', ... 0236 'Invalid output format: %s.', output_format) 0237 end 0238 0239 end