SX2MAT Load data and metadata from a SeaExplorer data file.
Syntax:
[META, DATA] = SX2MAT(FILENAME)
[META, DATA] = SX2MAT(FILENAME, OPTIONS)
[META, DATA] = SX2MAT(FILENAME, OPT1, VAL1, ...)
Description:
[META, DATA] = SX2MAT(FILENAME) reads the SeaExplorer file named by string
FILENAME, loading its metadata in struct META and its data in array DATA.
[META, DATA] = SX2MAT(FILENAME, OPTIONS) and
[META, DATA] = SX2MAT(FILENAME, OPT1, VAL1, ...) accept the following
options given in key-value pairs OPT1, VAL1... or in a struct OPTIONS with
field names as option keys and field values as option values:
FORMAT: data output format.
String setting the format of the output DATA. Valid values are:
'array': DATA is a matrix with variable readings in the column order
specified by the VARIABLES metadata field.
'struct': DATA is a struct with variable names as field names
and column vectors of variable readings as field values.
Default value: 'array'
VARIABLES: variable filtering list.
String cell array with the names of the variables of interest.
If given, only variables present in both the input file and this list
will be present in output. The string 'all' may also be given,
in which case variable filtering is not performed and all variables
in the input file will be present in output.
Default value: 'all' (do not perform variable filtering).
TIME: timestamp variable list.
String cell array with the names of the time variables.
Variables in this list and also present in the data file are assumed
to be UTC timestamps in the format 'DD/MM/YYYY HH:MM:SS' or
'DD/MM/YYYY HH:MM:SS.FFF', and are converted to numeric values
(seconds since 1970-01-01 00:00:00 UTC) with DATENUM
and UTC2POSIXTIME.
Default value: {'Timestamp' 'PLD_REALTIMECLOCK'}
META has the following fields based on the tags of the ascii header:
VARIABLES: string cell array with the names of the variables present
in the returned data array (in the same column order as the data).
SOURCES: string cell array containing FILENAME.
Examples:
% Retrieve data from all variables as array:
[meta, data] = sx2mat('test.dat.0001')
% Retrieve data from all variables as struct:
[meta, data] = sx2mat('test.gli.0001', 'format', 'struct')
% Retrieve attitude data as struct:
[meta, data] = sx2mat('test.gli.0001', 'format', 'struct', ...
'variables', {'Heading' 'Pitch' 'Roll'});
See also:
SXCAT
SXMERGE
DATENUM
UTC2POSIXTIME
Authors:
Frederic Cyr <Frederic.Cyr@mio.osupytheas.fr>
Joan Pau Beltran <joanpau.beltran@socib.cat>0001 function [meta, data] = sx2mat(filename, varargin) 0002 %SX2MAT Load data and metadata from a SeaExplorer data file. 0003 % 0004 % Syntax: 0005 % [META, DATA] = SX2MAT(FILENAME) 0006 % [META, DATA] = SX2MAT(FILENAME, OPTIONS) 0007 % [META, DATA] = SX2MAT(FILENAME, OPT1, VAL1, ...) 0008 % 0009 % Description: 0010 % [META, DATA] = SX2MAT(FILENAME) reads the SeaExplorer file named by string 0011 % FILENAME, loading its metadata in struct META and its data in array DATA. 0012 % 0013 % [META, DATA] = SX2MAT(FILENAME, OPTIONS) and 0014 % [META, DATA] = SX2MAT(FILENAME, OPT1, VAL1, ...) accept the following 0015 % options given in key-value pairs OPT1, VAL1... or in a struct OPTIONS with 0016 % field names as option keys and field values as option values: 0017 % FORMAT: data output format. 0018 % String setting the format of the output DATA. Valid values are: 0019 % 'array': DATA is a matrix with variable readings in the column order 0020 % specified by the VARIABLES metadata field. 0021 % 'struct': DATA is a struct with variable names as field names 0022 % and column vectors of variable readings as field values. 0023 % Default value: 'array' 0024 % VARIABLES: variable filtering list. 0025 % String cell array with the names of the variables of interest. 0026 % If given, only variables present in both the input file and this list 0027 % will be present in output. The string 'all' may also be given, 0028 % in which case variable filtering is not performed and all variables 0029 % in the input file will be present in output. 0030 % Default value: 'all' (do not perform variable filtering). 0031 % TIME: timestamp variable list. 0032 % String cell array with the names of the time variables. 0033 % Variables in this list and also present in the data file are assumed 0034 % to be UTC timestamps in the format 'DD/MM/YYYY HH:MM:SS' or 0035 % 'DD/MM/YYYY HH:MM:SS.FFF', and are converted to numeric values 0036 % (seconds since 1970-01-01 00:00:00 UTC) with DATENUM 0037 % and UTC2POSIXTIME. 0038 % Default value: {'Timestamp' 'PLD_REALTIMECLOCK'} 0039 % 0040 % META has the following fields based on the tags of the ascii header: 0041 % VARIABLES: string cell array with the names of the variables present 0042 % in the returned data array (in the same column order as the data). 0043 % SOURCES: string cell array containing FILENAME. 0044 % 0045 % Examples: 0046 % % Retrieve data from all variables as array: 0047 % [meta, data] = sx2mat('test.dat.0001') 0048 % % Retrieve data from all variables as struct: 0049 % [meta, data] = sx2mat('test.gli.0001', 'format', 'struct') 0050 % % Retrieve attitude data as struct: 0051 % [meta, data] = sx2mat('test.gli.0001', 'format', 'struct', ... 0052 % 'variables', {'Heading' 'Pitch' 'Roll'}); 0053 % 0054 % See also: 0055 % SXCAT 0056 % SXMERGE 0057 % DATENUM 0058 % UTC2POSIXTIME 0059 % 0060 % Authors: 0061 % Frederic Cyr <Frederic.Cyr@mio.osupytheas.fr> 0062 % Joan Pau Beltran <joanpau.beltran@socib.cat> 0063 0064 % Copyright (C) 2016 0065 % ICTS SOCIB - Servei d'observacio i prediccio costaner de les Illes Balears 0066 % <http://www.socib.es> 0067 % 0068 % This program is free software: you can redistribute it and/or modify 0069 % it under the terms of the GNU General Public License as published by 0070 % the Free Software Foundation, either version 3 of the License, or 0071 % (at your option) any later version. 0072 % 0073 % This program is distributed in the hope that it will be useful, 0074 % but WITHOUT ANY WARRANTY; without even the implied warranty of 0075 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 0076 % GNU General Public License for more details. 0077 % 0078 % You should have received a copy of the GNU General Public License 0079 % along with this program. If not, see <http://www.gnu.org/licenses/>. 0080 0081 error(nargchk(1, 7, nargin, 'struct')); 0082 0083 0084 %% Set options and default values. 0085 options.format = 'array'; 0086 options.variables = 'all'; 0087 options.time = {'Timestamp' 'PLD_REALTIMECLOCK'}; 0088 0089 0090 %% Parse optional arguments. 0091 % Get option key-value pairs in any accepted call signature. 0092 argopts = varargin; 0093 if isscalar(argopts) && isstruct(argopts{1}) 0094 % Options passed as a single option struct argument: 0095 % field names are option keys and field values are option values. 0096 opt_key_list = fieldnames(argopts{1}); 0097 opt_val_list = struct2cell(argopts{1}); 0098 elseif mod(numel(argopts), 2) == 0 0099 % Options passed as key-value argument pairs. 0100 opt_key_list = argopts(1:2:end); 0101 opt_val_list = argopts(2:2:end); 0102 else 0103 error('glider_toolbox:sx2mat:InvalidOptions', ... 0104 'Invalid optional arguments (neither key-value pairs nor struct).'); 0105 end 0106 % Overwrite default options with values given in extra arguments. 0107 for opt_idx = 1:numel(opt_key_list) 0108 opt = lower(opt_key_list{opt_idx}); 0109 val = opt_val_list{opt_idx}; 0110 if isfield(options, opt) 0111 options.(opt) = val; 0112 else 0113 error('glider_toolbox:sx2mat:InvalidOption', ... 0114 'Invalid option: %s.', opt); 0115 end 0116 end 0117 0118 0119 %% Set option flags and values. 0120 output_format = lower(options.format); 0121 variable_filtering = true; 0122 if ischar(options.variables) && strcmp(options.variables, 'all') 0123 variable_filtering = false; 0124 end 0125 variable_list = cellstr(options.variables); 0126 time_variable_list = cellstr(options.time); 0127 0128 0129 %% Open the file. 0130 [fid, fid_msg] = fopen(filename, 'r'); 0131 if fid < 0 0132 error('glider_toolbox:sx2mat:FileError', fid_msg); 0133 end 0134 0135 0136 %% Process the file. 0137 try 0138 % Read variable names in header line. 0139 header = fgetl(fid); 0140 variable_values = ... 0141 textscan(header, '%s', 'Delimiter', ';', 'ReturnOnError', false); 0142 0143 % Build metadata structure. 0144 [~, name, ext] = fileparts(filename); 0145 meta.sources = {[name ext]}; 0146 meta.variables = variable_values{1}; 0147 0148 % Read variable data filtering selected variables if needed. 0149 variable_format = repmat({'%f'}, length(meta.variables), 1); 0150 time_variable_select = ismember(meta.variables, time_variable_list); 0151 variable_format(time_variable_select) = {'%s'}; 0152 if variable_filtering 0153 variable_select = ismember(meta.variables, variable_list); 0154 variable_format( time_variable_select & ~variable_select) = {'%*f'}; 0155 variable_format(~time_variable_select & ~variable_select) = {'%*s'}; 0156 meta.variables = variables(variable_select); 0157 time_variable_select = time_variable_select(variable_select); 0158 end 0159 data_values = textscan(fid, [variable_format{:} '%*s'], ... 0160 'Delimiter', ';', 'ReturnOnError', false); 0161 0162 % Convert timestamp variables to numeric format. 0163 for time_variable_idx = find(time_variable_select) 0164 data_timestamp = data_values{time_variable_idx}; 0165 switch size(char(data_timestamp), 2) 0166 case 19 0167 timestamp_format = 'dd/mm/yyyy HH:MM:SS'; 0168 case 23 0169 timestamp_format = 'dd/mm/yyyy HH:MM:SS.FFF'; 0170 otherwise 0171 error('glider_toolbox:sx2mat:TimestampError', ... 0172 'Unknown timestamp format'); 0173 end 0174 data_datenum = datenum(data_timestamp, timestamp_format); 0175 data_values{time_variable_idx} = utc2posixtime(data_datenum); 0176 end 0177 0178 % Convert data to desired output format: 0179 switch output_format 0180 case 'array' 0181 data = [data_values{:}]; 0182 case 'struct' 0183 data = cell2struct(data_values, meta.variables, 2); 0184 otherwise 0185 error('glider_toolbox:sx2mat:InvalidFormat', ... 0186 'Invalid output format: %s.', format) 0187 end 0188 catch exception 0189 % Close the file after a reading error. 0190 fclose(fid); 0191 rethrow(exception); 0192 end 0193 0194 0195 %% Close the file after successful reading. 0196 fclose(fid); 0197 0198 end