VALIDATEPROFILE Check if profile sequence is a proper profile and if it is well sampled.
Syntax:
VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN)
VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPTIONS)
VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPT1, VAL1, ...)
[VALID, FULL_ROWS]= VALIDATEPROFILE(...)
Description:
VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPTIONS) and
VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPT1, VAL1, ...) check
whether vector DEPTH is a proper profile depth sequence
and if data in vectors or column arrays DATA1, ... , DATAN
is properly sampled over the profile range, according to criteria in
options given in key-value pairs OPT1, VAL1... or in a struct OPTIONS
with field names as option keys and field values as option values.
The profile is required to have a minimum depth range and
contain no significant gaps of invalid data readings or depth inversions.
The number of rows of DEPTH, and DATA1, ... , DATAN should be the same.
[VALID, FULL_ROWS] = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, ...)
also returns a logical column vector FULL_ROWS with the same number of
elements as DEPTH, showing whether respective entries in DEPTH or rows in
DATA1, ... , DATAN contain some invalid value or lie in a depth inversion.
[VALID, FULL_ROWS, DATA1, ... , DATAN] = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, ...)
also returns then same input data DATA1, ... , DATAN but with entries
corresponding to invalid rows in FULL_ROWS replaced according to the mask
value specified in options.
Valid options are:
RANGE: minimum depth range (in the same units as DEPTH).
A profile is invalid if the difference between the maximum and minimum
depth values is smaller than given threshold.
Default value: 0 (only empty profiles are invalid).
GAP: maximum gap ratio (in [0,1]).
A profile is invalid if the ratio of the depth range of the largest gap
to the depth range of the whole profile is larger than given threshold.
A gap is a sequence of consecutive incomplete measurements, either
because of invalid values (NaN) in some column of DATA, or because of
invalid entries in DEPTH.
Default value: 1 (only empty profiles are invalid).
MASK: replacement value for invalid data readings.
When data outputs are requested, respective data inputs are returned
but with entries correponding to invalid rows replaced by the given
value. If empty ([]), the entries are removed instead of replaced.
Default value: nan
New in version v1.1.0:
Identify and discard depth inversions in the profile.
Return input data with invalid rows masked with specified value.
Notes:
This function is based on the previous work by Tomeu Garau, in functions
called FINDPROFILES (not to be confused with the current function with the
same name) and CLEANPROFILE. He is the true glider man.
Examples:
depth = [1 2 3 2 5 nan 7 nan 9 10]'
data1 = [0 1 1 1 4 nan nan 7 8 9]'
data2 = [1 4 4 4 25 nan 49 64 nan 100]'
data = [data1 data2]
% Default options: any profile is valid,
% useful to retrieve valid rows and flag depth inversions.
[valid, full_rows] = validateProfile(depth, data)
depth(full_rows)
data(full_rows, :)
% Invalid profile: range too small.
valid = validateProfile(depth, data1, data2, 'range', 15)
% Invalid profile: gap too large.
valid = validateProfile(depth, data1, data2, 'gap', 0.25)
% Valid profile: large enough range and small enough gap.
valid = validateProfile(depth, data1, data2, 'range', 5, 'gap', 0.75)
% Mask invalid rows in input data with default value (NaN):
[valid, full_rows, data1, data2] = ...
validateProfile(depth, data(:,1), data(:,2))
% Mask invalid rows in input data with a different value (NaN):
[valid, full_rows, data1, data2] = ...
validateProfile(depth, data(:,1), data(:,2), 'mask', 9999)
% Remove invalid rows in input data:
[valid, full_rows, data1, data2] = ...
validateProfile(depth, data(:,1), data(:,2), 'mask', [])
See also:
ISNAN
Authors:
Joan Pau Beltran <joanpau.beltran@socib.cat>0001 function [valid, full_rows, varargout] = validateProfile(depth, varargin) 0002 %VALIDATEPROFILE Check if profile sequence is a proper profile and if it is well sampled. 0003 % 0004 % Syntax: 0005 % VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN) 0006 % VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPTIONS) 0007 % VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPT1, VAL1, ...) 0008 % [VALID, FULL_ROWS]= VALIDATEPROFILE(...) 0009 % 0010 % Description: 0011 % VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPTIONS) and 0012 % VALID = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, OPT1, VAL1, ...) check 0013 % whether vector DEPTH is a proper profile depth sequence 0014 % and if data in vectors or column arrays DATA1, ... , DATAN 0015 % is properly sampled over the profile range, according to criteria in 0016 % options given in key-value pairs OPT1, VAL1... or in a struct OPTIONS 0017 % with field names as option keys and field values as option values. 0018 % The profile is required to have a minimum depth range and 0019 % contain no significant gaps of invalid data readings or depth inversions. 0020 % The number of rows of DEPTH, and DATA1, ... , DATAN should be the same. 0021 % 0022 % [VALID, FULL_ROWS] = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, ...) 0023 % also returns a logical column vector FULL_ROWS with the same number of 0024 % elements as DEPTH, showing whether respective entries in DEPTH or rows in 0025 % DATA1, ... , DATAN contain some invalid value or lie in a depth inversion. 0026 % 0027 % [VALID, FULL_ROWS, DATA1, ... , DATAN] = VALIDATEPROFILE(DEPTH, DATA1, ... , DATAN, ...) 0028 % also returns then same input data DATA1, ... , DATAN but with entries 0029 % corresponding to invalid rows in FULL_ROWS replaced according to the mask 0030 % value specified in options. 0031 % 0032 % Valid options are: 0033 % RANGE: minimum depth range (in the same units as DEPTH). 0034 % A profile is invalid if the difference between the maximum and minimum 0035 % depth values is smaller than given threshold. 0036 % Default value: 0 (only empty profiles are invalid). 0037 % GAP: maximum gap ratio (in [0,1]). 0038 % A profile is invalid if the ratio of the depth range of the largest gap 0039 % to the depth range of the whole profile is larger than given threshold. 0040 % A gap is a sequence of consecutive incomplete measurements, either 0041 % because of invalid values (NaN) in some column of DATA, or because of 0042 % invalid entries in DEPTH. 0043 % Default value: 1 (only empty profiles are invalid). 0044 % MASK: replacement value for invalid data readings. 0045 % When data outputs are requested, respective data inputs are returned 0046 % but with entries correponding to invalid rows replaced by the given 0047 % value. If empty ([]), the entries are removed instead of replaced. 0048 % Default value: nan 0049 % 0050 % New in version v1.1.0: 0051 % Identify and discard depth inversions in the profile. 0052 % Return input data with invalid rows masked with specified value. 0053 % 0054 % Notes: 0055 % This function is based on the previous work by Tomeu Garau, in functions 0056 % called FINDPROFILES (not to be confused with the current function with the 0057 % same name) and CLEANPROFILE. He is the true glider man. 0058 % 0059 % Examples: 0060 % depth = [1 2 3 2 5 nan 7 nan 9 10]' 0061 % data1 = [0 1 1 1 4 nan nan 7 8 9]' 0062 % data2 = [1 4 4 4 25 nan 49 64 nan 100]' 0063 % data = [data1 data2] 0064 % % Default options: any profile is valid, 0065 % % useful to retrieve valid rows and flag depth inversions. 0066 % [valid, full_rows] = validateProfile(depth, data) 0067 % depth(full_rows) 0068 % data(full_rows, :) 0069 % % Invalid profile: range too small. 0070 % valid = validateProfile(depth, data1, data2, 'range', 15) 0071 % % Invalid profile: gap too large. 0072 % valid = validateProfile(depth, data1, data2, 'gap', 0.25) 0073 % % Valid profile: large enough range and small enough gap. 0074 % valid = validateProfile(depth, data1, data2, 'range', 5, 'gap', 0.75) 0075 % % Mask invalid rows in input data with default value (NaN): 0076 % [valid, full_rows, data1, data2] = ... 0077 % validateProfile(depth, data(:,1), data(:,2)) 0078 % % Mask invalid rows in input data with a different value (NaN): 0079 % [valid, full_rows, data1, data2] = ... 0080 % validateProfile(depth, data(:,1), data(:,2), 'mask', 9999) 0081 % % Remove invalid rows in input data: 0082 % [valid, full_rows, data1, data2] = ... 0083 % validateProfile(depth, data(:,1), data(:,2), 'mask', []) 0084 % 0085 % See also: 0086 % ISNAN 0087 % 0088 % Authors: 0089 % Joan Pau Beltran <joanpau.beltran@socib.cat> 0090 0091 % Copyright (C) 2013-2016 0092 % ICTS SOCIB - Servei d'observacio i prediccio costaner de les Illes Balears 0093 % <http://www.socib.es> 0094 % 0095 % This program is free software: you can redistribute it and/or modify 0096 % it under the terms of the GNU General Public License as published by 0097 % the Free Software Foundation, either version 3 of the License, or 0098 % (at your option) any later version. 0099 % 0100 % This program is distributed in the hope that it will be useful, 0101 % but WITHOUT ANY WARRANTY; without even the implied warranty of 0102 % MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 0103 % GNU General Public License for more details. 0104 % 0105 % You should have received a copy of the GNU General Public License 0106 % along with this program. If not, see <http://www.gnu.org/licenses/>. 0107 0108 error(nargchk(1, Inf, nargin, 'struct')); 0109 0110 0111 %% Parse basic input arguments. 0112 % Get numeric (non-option) arguments. 0113 nargnum = find(~cellfun(@isnumeric, varargin), 1, 'first') - 1; 0114 if isempty(nargnum) 0115 nargnum = numel(varargin); 0116 end 0117 data = [varargin{1:nargnum}]; 0118 0119 0120 %% Set options and default values. 0121 options.range = 0; 0122 options.gap = 1; 0123 options.mask = nan; 0124 0125 0126 %% Parse optional arguments. 0127 % Get numeric data arguments and option arguments. 0128 % Get option key-value pairs in any accepted call signature. 0129 argopts = varargin(nargnum+1:end); 0130 if isscalar(argopts) && isstruct(argopts{1}) 0131 % Options passed as a single option struct argument: 0132 % field names are option keys and field values are option values. 0133 opt_key_list = fieldnames(argopts{1}); 0134 opt_val_list = struct2cell(argopts{1}); 0135 elseif mod(numel(argopts), 2) == 0 0136 % Options passed as key-value argument pairs. 0137 opt_key_list = argopts(1:2:end); 0138 opt_val_list = argopts(2:2:end); 0139 else 0140 error('glider_toolbox:validateProfile:InvalidOptions', ... 0141 'Invalid optional arguments (neither key-value pairs nor struct).'); 0142 end 0143 % Overwrite default options with values given in extra arguments. 0144 for opt_idx = 1:numel(opt_key_list) 0145 opt = lower(opt_key_list{opt_idx}); 0146 val = opt_val_list{opt_idx}; 0147 if isfield(options, opt) 0148 options.(opt) = val; 0149 else 0150 error('glider_toolbox:validateProfile:InvalidOption', ... 0151 'Invalid option: %s.', opt); 0152 end 0153 end 0154 0155 0156 %% Validate the profile. 0157 % Flag invalid data and depth inversions. 0158 depth_valid = ~isnan(depth(:)); 0159 data_valid = ~any(isnan(data), 2); 0160 [depth_min_value, depth_min_index] = min(depth); 0161 [depth_max_value, depth_max_index] = max(depth); 0162 % CUMMIN and CUMMAX are available in Octave but not in MATLAB 0163 % (in release 2014b they are in the Statistics Toolbox). 0164 % With them, we could use this one-liners: 0165 %{ 0166 if (depth_min_index < depth_max_index) 0167 monotonic = cummax(depth(:) == flipud(cummin(flipud(depth(:)))); 0168 elseif (depth_min_index > depth_max_index) 0169 monotonic = cummin(depth(:) == flipud(cummax(flipud(depth(:)))); 0170 else 0171 monotonic = true(size(depth(:))); 0172 end 0173 %} 0174 if (depth_min_index < depth_max_index) 0175 omax = find(depth_valid, 1, 'first'); 0176 omin = find(depth_valid, 1, 'last'); 0177 elseif (depth_min_index > depth_max_index) 0178 omax = find(depth_valid, 1, 'last'); 0179 omin = find(depth_valid, 1, 'first'); 0180 else 0181 omax = depth_max_index; 0182 omin = depth_min_index; 0183 end 0184 cmax = depth(:); 0185 cmin = depth(:); 0186 tmax = depth(omax); 0187 tmin = depth(omin); 0188 for k = 0:sign(omin-omax):(omin-omax) 0189 if cmax(omax + k) > tmax 0190 tmax = cmax(omax + k); 0191 else 0192 cmax(omax + k) = tmax; 0193 end 0194 if cmin(omin - k) < tmin 0195 tmin = cmin(omin - k); 0196 else 0197 cmin(omin - k) = tmin; 0198 end 0199 end 0200 if (depth_min_index < depth_max_index) 0201 cmax(omin+1:end) = tmax; 0202 cmin(1:omax-1) = tmin; 0203 elseif (depth_min_index > depth_max_index) 0204 cmax(1:omin-1) = tmax; 0205 cmin(omax+1:end) = tmin; 0206 end 0207 monotonic = (cmax == cmin); 0208 % Initialize output. 0209 full_rows = depth_valid & data_valid & monotonic; 0210 valid = false; 0211 for k = 1:min(nargnum, nargout - 2) 0212 masked = varargin{k}; 0213 if isequal(options.mask, []) 0214 masked(~full_rows, :) = []; 0215 else 0216 masked(~full_rows, :) = options.mask; 0217 end 0218 varargout{k} = masked; 0219 end 0220 % Emptiness check. 0221 if ~any(full_rows) 0222 return 0223 end 0224 % Range check. 0225 depth_range = depth_max_value - depth_min_value; 0226 if depth_range < options.range 0227 return 0228 end 0229 % Gap check. 0230 max_gap = max([min(depth(full_rows))-min(depth) ... 0231 max(abs(diff(depth(full_rows)))) ... 0232 max(depth) - max(depth(full_rows))]); 0233 if max_gap > options.gap * depth_range 0234 return 0235 end 0236 % Checks passed, profile is valid. 0237 valid = true; 0238 0239 end