LOADSEAEXPLORERDATA Load SeaExplorer data from SeaExplorer glider and payload files in directory.
Syntax:
[META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP)
[META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPTIONS)
[META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPT1, VAL1, ...)
Description:
[META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP)
loads data and metadata from SeaExplorer glider or payload files
in ascii text format contained in directory named by string SXDIR and
whose name matches regular expression in string GLIREGEXP (glider files)
or in string PLDREGEXP (payload files). META and DATA contain loaded
metadata and data in the format returned by functions SXCAT and SXMERGE.
[META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPTIONS) and
[META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPT1, VAL1, ...)
accept the following options, given in key-value pairs OPT1, VAL1...
or in a struct OPTIONS with field names as option keys and field values
as option values, allowing to restrict the time range or the set of
variables to load, or to specify the output format:
FORMAT: data output format.
String setting the format of the output DATA. Valid values are:
'array': DATA is a matrix with variable readings in the column order
specified by the VARIABLES metadata field.
'struct': DATA is a struct with variable names as field names
and column vectors of variable readings as field values.
Default value: 'array'
TIMEGLI: glider data timestamp.
String setting the name of the variable to use as timestamp for merging
and sorting data row readings from SeaExplorer glider data set.
Default value: 'Timestamp'
TIMEPLD: payload data timestamp.
String setting the name of the variable to use as timestamp for merging
and sorting data row readings from SeaExplorer payload data set.
Default value: 'PLD_REALTIMECLOCK'
VARIABLES: variable filtering list.
String cell array with the names of the variables of interest.
If given, only variables present in both the input data sets and this
list will be present in output. The string 'all' may also be given,
in which case variable filtering is not performed and all variables
in input data sets will be present in output.
Default value: 'all' (do not perform variable filtering).
PERIOD: time filtering boundaries.
Two element numeric array with the start and the end of the period
of interest (seconds since 1970-01-01 00:0:00.00 UTC). If given,
only row readings with timestamps within this period will be present
in output. The string 'all' may also be given, in which case time
filtering is not performed and all row readings in the input list
will be present in output.
Default value: 'all' (do not perform time filtering).
Notes:
This function is a simple shortcut to load all SeaExplorer data
in a directory belonging to the same deployment or transect.
It just filters the contents of the directory and calls SX2MAT, SXCAT
and SXMERGE, bypassing the given options.
Examples:
[meta, data] = ...
loadSeaExplorerData(ascii_dir, gliregexp, pldregexp)
[meta, data] = ...
loadSeaExplorerData(ascii_dir, '^.*.gli.*$', '^.*.pld.*$', ...
'timegli', 'Timestamp', ...
'timepld', 'PLD_REALTIMECLOCK', ...
'variables', variables_of_interest, ...
'period', period_of_interest, ...
'format', 'struct');
See also:
DIR
REGEXP
SX2MAT
SXCAT
SXMERGE
Authors:
Frederic Cyr <Frederic.Cyr@mio.osupytheas.fr>
Joan Pau Beltran <joanpau.beltran@socib.cat>0001 function [meta, data] = loadSeaExplorerData(sxdir, gliregexp, pldregexp, varargin) 0002 %LOADSEAEXPLORERDATA Load SeaExplorer data from SeaExplorer glider and payload files in directory. 0003 % 0004 % Syntax: 0005 % [META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP) 0006 % [META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPTIONS) 0007 % [META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPT1, VAL1, ...) 0008 % 0009 % Description: 0010 % [META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP) 0011 % loads data and metadata from SeaExplorer glider or payload files 0012 % in ascii text format contained in directory named by string SXDIR and 0013 % whose name matches regular expression in string GLIREGEXP (glider files) 0014 % or in string PLDREGEXP (payload files). META and DATA contain loaded 0015 % metadata and data in the format returned by functions SXCAT and SXMERGE. 0016 % 0017 % [META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPTIONS) and 0018 % [META, DATA] = LOADSEAEXPLORERDATA(SXDIR, GLIREGEXP, PLDREGEXP, OPT1, VAL1, ...) 0019 % accept the following options, given in key-value pairs OPT1, VAL1... 0020 % or in a struct OPTIONS with field names as option keys and field values 0021 % as option values, allowing to restrict the time range or the set of 0022 % variables to load, or to specify the output format: 0023 % FORMAT: data output format. 0024 % String setting the format of the output DATA. Valid values are: 0025 % 'array': DATA is a matrix with variable readings in the column order 0026 % specified by the VARIABLES metadata field. 0027 % 'struct': DATA is a struct with variable names as field names 0028 % and column vectors of variable readings as field values. 0029 % Default value: 'array' 0030 % TIMEGLI: glider data timestamp. 0031 % String setting the name of the variable to use as timestamp for merging 0032 % and sorting data row readings from SeaExplorer glider data set. 0033 % Default value: 'Timestamp' 0034 % TIMEPLD: payload data timestamp. 0035 % String setting the name of the variable to use as timestamp for merging 0036 % and sorting data row readings from SeaExplorer payload data set. 0037 % Default value: 'PLD_REALTIMECLOCK' 0038 % VARIABLES: variable filtering list. 0039 % String cell array with the names of the variables of interest. 0040 % If given, only variables present in both the input data sets and this 0041 % list will be present in output. The string 'all' may also be given, 0042 % in which case variable filtering is not performed and all variables 0043 % in input data sets will be present in output. 0044 % Default value: 'all' (do not perform variable filtering). 0045 % PERIOD: time filtering boundaries. 0046 % Two element numeric array with the start and the end of the period 0047 % of interest (seconds since 1970-01-01 00:0:00.00 UTC). If given, 0048 % only row readings with timestamps within this period will be present 0049 % in output. The string 'all' may also be given, in which case time 0050 % filtering is not performed and all row readings in the input list 0051 % will be present in output. 0052 % Default value: 'all' (do not perform time filtering). 0053 % 0054 % Notes: 0055 % This function is a simple shortcut to load all SeaExplorer data 0056 % in a directory belonging to the same deployment or transect. 0057 % It just filters the contents of the directory and calls SX2MAT, SXCAT 0058 % and SXMERGE, bypassing the given options. 0059 % 0060 % Examples: 0061 % [meta, data] = ... 0062 % loadSeaExplorerData(ascii_dir, gliregexp, pldregexp) 0063 % [meta, data] = ... 0064 % loadSeaExplorerData(ascii_dir, '^.*.gli.*$', '^.*.pld.*$', ... 0065 % 'timegli', 'Timestamp', ... 0066 % 'timepld', 'PLD_REALTIMECLOCK', ... 0067 % 'variables', variables_of_interest, ... 0068 % 'period', period_of_interest, ... 0069 % 'format', 'struct'); 0070 % 0071 % See also: 0072 % DIR 0073 % REGEXP 0074 % SX2MAT 0075 % SXCAT 0076 % SXMERGE 0077 % 0078 % Authors: 0079 % Frederic Cyr <Frederic.Cyr@mio.osupytheas.fr> 0080 % Joan Pau Beltran <joanpau.beltran@socib.cat> 0081 0082 % Copyright (C) 2016 0083 % ICTS SOCIB - Servei d'observacio i prediccio costaner de les Illes Balears 0084 % <http://www.socib.es> 0085 % 0086 % This program is free software: you can redistribute it and/or modify 0087 % it under the terms of the GNU General Public License as published by 0088 % the Free Software Foundation, either version 3 of the License, or 0089 % (at your option) any later version. 0090 % 0091 % This program is distributed in the hope that it will be useful, 0092 % but WITHOUT ANY WARRANTY; without even the implied warranty of 0093 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 0094 % GNU General Public License for more details. 0095 % 0096 % You should have received a copy of the GNU General Public License 0097 % along with this program. If not, see <http://www.gnu.org/licenses/>. 0098 0099 error(nargchk(3, 13, nargin, 'struct')); 0100 0101 0102 %% Set options and default values. 0103 options.format = 'array'; 0104 options.timegli = 'Timestamp'; 0105 options.timepld = 'PLD_REALTIMECLOCK'; 0106 options.variables = 'all'; 0107 options.period = 'all'; 0108 0109 0110 %% Parse optional arguments. 0111 % Get option key-value pairs in any accepted call signature. 0112 argopts = varargin; 0113 if isscalar(argopts) && isstruct(argopts{1}) 0114 % Options passed as a single option struct argument: 0115 % field names are option keys and field values are option values. 0116 opt_key_list = fieldnames(argopts{1}); 0117 opt_val_list = struct2cell(argopts{1}); 0118 elseif mod(numel(argopts), 2) == 0 0119 % Options passed as key-value argument pairs. 0120 opt_key_list = argopts(1:2:end); 0121 opt_val_list = argopts(2:2:end); 0122 else 0123 error('glider_toolbox:loadSeaExplorerData:InvalidOptions', ... 0124 'Invalid optional arguments (neither key-value pairs nor struct).'); 0125 end 0126 % Overwrite default options with values given in extra arguments. 0127 for opt_idx = 1:numel(opt_key_list) 0128 opt = lower(opt_key_list{opt_idx}); 0129 val = opt_val_list{opt_idx}; 0130 if isfield(options, opt) 0131 options.(opt) = val; 0132 else 0133 error('glider_toolbox:loadSeaExplorerData:InvalidOption', ... 0134 'Invalid option: %s.', opt); 0135 end 0136 end 0137 0138 0139 %% Get file names matching the desired patterns. 0140 % Flatten lists to discard unmatched files. 0141 sxdir_contents = dir(sxdir); 0142 gli_sel = ~[sxdir_contents.isdir] ... 0143 & ~cellfun(@isempty, regexp({sxdir_contents.name}, gliregexp)); 0144 pld_sel = ~[sxdir_contents.isdir] ... 0145 & ~cellfun(@isempty, regexp({sxdir_contents.name}, pldregexp)); 0146 gli_names = {sxdir_contents(gli_sel).name}; 0147 gli_sizes = [sxdir_contents(gli_sel).bytes]; 0148 pld_names = {sxdir_contents(pld_sel).name}; 0149 pld_sizes = [sxdir_contents(pld_sel).bytes]; 0150 disp(['SeaExplorer glider files found: ' num2str(numel(gli_names)) ... 0151 ' (' num2str(sum(gli_sizes)*2^-10) ' kB).']); 0152 disp(['SeaExplorer payload files found: ' num2str(numel(pld_names)) ... 0153 ' (' num2str(sum(pld_sizes)*2^-10) ' kB).']); 0154 0155 0156 %% Load glider files. 0157 disp('Loading SeaExplorer glider files...'); 0158 gli_files = cell(size(gli_names)); 0159 gli_success = false(size(gli_names)); 0160 meta_gli = cell(size(gli_names)); 0161 data_gli = cell(size(gli_names)); 0162 for gli_idx = 1:numel(gli_names) 0163 try 0164 gli_files{gli_idx} = fullfile(sxdir, gli_names{gli_idx}); 0165 [meta_gli{gli_idx}, data_gli{gli_idx}] = ... 0166 sx2mat(gli_files{gli_idx}, ... 0167 'time', options.timegli, 'variables', options.variables); 0168 gli_success(gli_idx) = true; 0169 catch exception 0170 disp(['Error loading SeaExplorer glider file ' gli_files{gli_idx} ':']); 0171 disp(getReport(exception, 'extended')); 0172 end 0173 end 0174 meta_gli = meta_gli(gli_success); 0175 data_gli = data_gli(gli_success); 0176 disp(['SeaExplorer glider files loaded: ' ... 0177 num2str(numel(data_gli)) ' of ' num2str(numel(gli_names)) '.']); 0178 0179 0180 %% Load payload files. 0181 disp('Loading SeaExplorer payload files...'); 0182 pld_files = cell(size(pld_names)); 0183 pld_success = false(size(pld_names)); 0184 meta_pld = cell(size(pld_names)); 0185 data_pld = cell(size(pld_names)); 0186 for pld_idx = 1:numel(pld_names) 0187 try 0188 pld_files{pld_idx} = fullfile(sxdir, pld_names{pld_idx}); 0189 [meta_pld{pld_idx}, data_pld{pld_idx}] = ... 0190 sx2mat(pld_files{pld_idx}, ... 0191 'time', options.timepld, 'variables', options.variables); 0192 pld_success(pld_idx) = true; 0193 catch exception 0194 disp(['Error loading SeaExplorer payload file ' pld_files{pld_idx} ':']); 0195 disp(getReport(exception, 'extended')); 0196 end 0197 end 0198 meta_pld = meta_pld(pld_success); 0199 data_pld = data_pld(pld_success); 0200 disp(['SeaExplorer payload files loaded: ' ... 0201 num2str(numel(data_pld)) ' of ' num2str(numel(pld_names)) '.']); 0202 0203 0204 %% Combine data from either glider or payload files. 0205 [meta_gli, data_gli] = sxcat(meta_gli, data_gli, options.timegli); 0206 [meta_pld, data_pld] = sxcat(meta_pld, data_pld, options.timepld); 0207 0208 0209 %% Merge data from glider and payload files. 0210 [meta, data] = ... 0211 sxmerge(meta_gli, data_gli, meta_pld, data_pld, ... 0212 'timegli', options.timegli, 'timepld', options.timepld, ... 0213 'variables', options.variables, 'period', options.period, ... 0214 'format', options.format); 0215 0216 end