LOADSLOCUMDATA Load Slocum data from dba files in directory.
Syntax:
[META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP)
[META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPTIONS)
[META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPT1, VAL1, ...)
Description:
[META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP)
loads data and metadata from Slocum files in ascii text format (dba)
contained in directory named by string DBADIR and whose name matches
regular expression in string NAVREGEXP (navigation files) or in string
SCIREGEXP (science files). META and DATA contain loaded metadata and data
in the format returned by functions DBACAT and DBAMERGE.
[META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPTIONS) and
[META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPT1, VAL1, ...)
accept the following options, given in key-value pairs OPT1, VAL1...
or in a struct OPTIONS with field names as option keys and field values
as option values, allowing to restrict the time range or set of sensors
to load, or to specify the output format:
FORMAT: data output format.
String setting the format of the output DATA. Valid values are:
'array': DATA is a matrix with sensor readings in the column order
specified by the SENSORS metadata field.
'struct': DATA is a struct with sensor names as field names
and column vectors of sensor readings as field values.
Default value: 'array'
TIMENAV: navigation data time stamp.
String setting the navigation data time sensor for merging and sorting
sensor cycles.
Default value: 'm_present_time'
TIMESCI: scientific data time stamp.
String setting the scientific data time sensor for merging and sorting
sensor cycles.
Default value: 'sci_m_present_time'
SENSORS: sensor filtering list.
String cell array with the names of the sensors of interest. If given,
only the sensors present in both the input data sets and this list
will be present in output. The string 'all' may also be given,
in which case sensor filtering is not performed and all sensors
in the input list will be present in output.
Default value: 'all' (do not perform sensor filtering).
PERIOD: time filtering boundaries.
Two element numeric array with the start and the end of the period
of interest (seconds since 1970-01-01 00:0:00.00 UTC). If given,
only sensor cycles with timestamps within this period will be present
in output. The string 'all' may also be given, in which case time
filtering is not performed and all sensors cycles in the input list
will be present in output.
Default value: 'all' (do not perform time filtering).
Notes:
This function is a simple shortcut to load all dba data in a directory
belonging to the same deployment or transect. It just filters the contents
of the directory and calls DBA2MAT, DBACAT and DBAMERGE, bypassing the
given options (with conversions when needed).
Examples:
[meta, data] = ...
loadSlocumData(dbair, navregexp, sciregexp)
[meta, data] = ...
loadSlocumData(dbadir, navregexp, sciregexp, ...
'timenav', 'm_present_time', ...
'timesci', 'sci_m_present_time', ...
'sensors', sensors_of_interest,
'period', period_of_interest, ...
'format', 'struct');
See also:
DIR
REGEXP
DBA2MAT
DBACAT
DBAMERGE
Authors:
Joan Pau Beltran <joanpau.beltran@socib.cat>0001 function [meta, data] = loadSlocumData(dbadir, navregexp, sciregexp, varargin) 0002 %LOADSLOCUMDATA Load Slocum data from dba files in directory. 0003 % 0004 % Syntax: 0005 % [META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP) 0006 % [META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPTIONS) 0007 % [META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPT1, VAL1, ...) 0008 % 0009 % Description: 0010 % [META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP) 0011 % loads data and metadata from Slocum files in ascii text format (dba) 0012 % contained in directory named by string DBADIR and whose name matches 0013 % regular expression in string NAVREGEXP (navigation files) or in string 0014 % SCIREGEXP (science files). META and DATA contain loaded metadata and data 0015 % in the format returned by functions DBACAT and DBAMERGE. 0016 % 0017 % [META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPTIONS) and 0018 % [META, DATA] = LOADSLOCUMDATA(DBADIR, NAVREGEXP, SCIREGEXP, OPT1, VAL1, ...) 0019 % accept the following options, given in key-value pairs OPT1, VAL1... 0020 % or in a struct OPTIONS with field names as option keys and field values 0021 % as option values, allowing to restrict the time range or set of sensors 0022 % to load, or to specify the output format: 0023 % FORMAT: data output format. 0024 % String setting the format of the output DATA. Valid values are: 0025 % 'array': DATA is a matrix with sensor readings in the column order 0026 % specified by the SENSORS metadata field. 0027 % 'struct': DATA is a struct with sensor names as field names 0028 % and column vectors of sensor readings as field values. 0029 % Default value: 'array' 0030 % TIMENAV: navigation data time stamp. 0031 % String setting the navigation data time sensor for merging and sorting 0032 % sensor cycles. 0033 % Default value: 'm_present_time' 0034 % TIMESCI: scientific data time stamp. 0035 % String setting the scientific data time sensor for merging and sorting 0036 % sensor cycles. 0037 % Default value: 'sci_m_present_time' 0038 % SENSORS: sensor filtering list. 0039 % String cell array with the names of the sensors of interest. If given, 0040 % only the sensors present in both the input data sets and this list 0041 % will be present in output. The string 'all' may also be given, 0042 % in which case sensor filtering is not performed and all sensors 0043 % in the input list will be present in output. 0044 % Default value: 'all' (do not perform sensor filtering). 0045 % PERIOD: time filtering boundaries. 0046 % Two element numeric array with the start and the end of the period 0047 % of interest (seconds since 1970-01-01 00:0:00.00 UTC). If given, 0048 % only sensor cycles with timestamps within this period will be present 0049 % in output. The string 'all' may also be given, in which case time 0050 % filtering is not performed and all sensors cycles in the input list 0051 % will be present in output. 0052 % Default value: 'all' (do not perform time filtering). 0053 % 0054 % Notes: 0055 % This function is a simple shortcut to load all dba data in a directory 0056 % belonging to the same deployment or transect. It just filters the contents 0057 % of the directory and calls DBA2MAT, DBACAT and DBAMERGE, bypassing the 0058 % given options (with conversions when needed). 0059 % 0060 % Examples: 0061 % [meta, data] = ... 0062 % loadSlocumData(dbair, navregexp, sciregexp) 0063 % [meta, data] = ... 0064 % loadSlocumData(dbadir, navregexp, sciregexp, ... 0065 % 'timenav', 'm_present_time', ... 0066 % 'timesci', 'sci_m_present_time', ... 0067 % 'sensors', sensors_of_interest, 0068 % 'period', period_of_interest, ... 0069 % 'format', 'struct'); 0070 % 0071 % See also: 0072 % DIR 0073 % REGEXP 0074 % DBA2MAT 0075 % DBACAT 0076 % DBAMERGE 0077 % 0078 % Authors: 0079 % Joan Pau Beltran <joanpau.beltran@socib.cat> 0080 0081 % Copyright (C) 2013-2016 0082 % ICTS SOCIB - Servei d'observacio i prediccio costaner de les Illes Balears 0083 % <http://www.socib.es> 0084 % 0085 % This program is free software: you can redistribute it and/or modify 0086 % it under the terms of the GNU General Public License as published by 0087 % the Free Software Foundation, either version 3 of the License, or 0088 % (at your option) any later version. 0089 % 0090 % This program is distributed in the hope that it will be useful, 0091 % but WITHOUT ANY WARRANTY; without even the implied warranty of 0092 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 0093 % GNU General Public License for more details. 0094 % 0095 % You should have received a copy of the GNU General Public License 0096 % along with this program. If not, see <http://www.gnu.org/licenses/>. 0097 0098 error(nargchk(3, 13, nargin, 'struct')); 0099 0100 0101 %% Set options and default values. 0102 options.format = 'array'; 0103 options.timenav = 'm_present_time'; 0104 options.timesci = 'sci_m_present_time'; 0105 options.sensors = 'all'; 0106 options.period = 'all'; 0107 0108 0109 %% Parse optional arguments. 0110 % Get option key-value pairs in any accepted call signature. 0111 argopts = varargin; 0112 if isscalar(argopts) && isstruct(argopts{1}) 0113 % Options passed as a single option struct argument: 0114 % field names are option keys and field values are option values. 0115 opt_key_list = fieldnames(argopts{1}); 0116 opt_val_list = struct2cell(argopts{1}); 0117 elseif mod(numel(argopts), 2) == 0 0118 % Options passed as key-value argument pairs. 0119 opt_key_list = argopts(1:2:end); 0120 opt_val_list = argopts(2:2:end); 0121 else 0122 error('glider_toolbox:loadSlocumData:InvalidOptions', ... 0123 'Invalid optional arguments (neither key-value pairs nor struct).'); 0124 end 0125 % Overwrite default options with values given in extra arguments. 0126 for opt_idx = 1:numel(opt_key_list) 0127 opt = lower(opt_key_list{opt_idx}); 0128 val = opt_val_list{opt_idx}; 0129 if isfield(options, opt) 0130 options.(opt) = val; 0131 else 0132 error('glider_toolbox:loadSlocumData:InvalidOption', ... 0133 'Invalid option: %s.', opt); 0134 end 0135 end 0136 0137 0138 %% Get file names matching the desired patterns. 0139 % Flatten lists to discard unmatched files. 0140 dbadir_contents = dir(dbadir); 0141 dba_nav_sel = ~[dbadir_contents.isdir] ... 0142 & ~cellfun(@isempty, regexp({dbadir_contents.name}, navregexp)); 0143 dba_sci_sel = ~[dbadir_contents.isdir] ... 0144 & ~cellfun(@isempty, regexp({dbadir_contents.name}, sciregexp)); 0145 dba_nav_names = {dbadir_contents(dba_nav_sel).name}; 0146 dba_nav_sizes = [dbadir_contents(dba_nav_sel).bytes]; 0147 dba_sci_names = {dbadir_contents(dba_sci_sel).name}; 0148 dba_sci_sizes = [dbadir_contents(dba_sci_sel).bytes]; 0149 disp(['Navigation data files found: ' num2str(numel(dba_nav_names)) ... 0150 ' (' num2str(sum(dba_nav_sizes)*2^-10) ' kB).']); 0151 disp(['Scientific data files found: ' num2str(numel(dba_sci_names)) ... 0152 ' (' num2str(sum(dba_sci_sizes)*2^-10) ' kB).']); 0153 0154 0155 %% Load navigation files. 0156 disp('Loading navigation files...'); 0157 dba_nav_files = cell(size(dba_nav_names)); 0158 dba_nav_success = false(size(dba_nav_names)); 0159 meta_nav = cell(size(dba_nav_names)); 0160 data_nav = cell(size(dba_nav_names)); 0161 for dba_nav_idx = 1:numel(dba_nav_names) 0162 try 0163 dba_nav_files{dba_nav_idx} = ... 0164 fullfile(dbadir, dba_nav_names{dba_nav_idx}); 0165 [meta_nav{dba_nav_idx}, data_nav{dba_nav_idx}] = ... 0166 dba2mat(dba_nav_files{dba_nav_idx}, 'sensors', options.sensors); 0167 dba_nav_success(dba_nav_idx) = true; 0168 catch exception 0169 disp(['Error loading dba file ' dba_nav_files{dba_nav_idx} ':']); 0170 disp(getReport(exception, 'extended')); 0171 end 0172 end 0173 meta_nav = meta_nav(dba_nav_success); 0174 data_nav = data_nav(dba_nav_success); 0175 disp(['Navigation files loaded: ' ... 0176 num2str(numel(data_nav)) ' of ' num2str(numel(dba_nav_names)) '.']); 0177 0178 0179 %% Load science files. 0180 disp('Loading science files...'); 0181 dba_sci_files = cell(size(dba_sci_names)); 0182 dba_sci_success = false(size(dba_sci_names)); 0183 meta_sci = cell(size(dba_sci_names)); 0184 data_sci = cell(size(dba_sci_names)); 0185 for dba_sci_idx = 1:numel(dba_sci_names) 0186 try 0187 dba_sci_files{dba_sci_idx} = ... 0188 fullfile(dbadir, dba_sci_names{dba_sci_idx}); 0189 [meta_sci{dba_sci_idx}, data_sci{dba_sci_idx}] = ... 0190 dba2mat(dba_sci_files{dba_sci_idx}, 'sensors', options.sensors); 0191 dba_sci_success(dba_sci_idx) = true; 0192 catch exception 0193 disp(['Error loading dba file ' dba_sci_files{dba_sci_idx} ':']); 0194 disp(getReport(exception, 'extended')); 0195 end 0196 end 0197 meta_sci = meta_sci(dba_sci_success); 0198 data_sci = data_sci(dba_sci_success); 0199 disp(['Science files loaded: ' ... 0200 num2str(numel(data_sci)) ' of ' num2str(numel(dba_sci_names)) '.']); 0201 0202 0203 %% Combine data from each bay. 0204 [meta_nav, data_nav] = dbacat(meta_nav, data_nav, options.timenav); 0205 [meta_sci, data_sci] = dbacat(meta_sci, data_sci, options.timesci); 0206 0207 0208 %% Merge data from both bays. 0209 [meta, data] = ... 0210 dbamerge(meta_nav, data_nav, meta_sci, data_sci, ... 0211 'timenav', options.timenav, 'timesci', options.timesci, ... 0212 'sensors', options.sensors, 'period', options.period, ... 0213 'format', options.format); 0214 0215 end