preprocessGliderData

PREPROCESSGLIDERDATA  Preprocess glider deployment data.

  Syntax:
    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW)
    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW, OPTIONS)
    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW, OPT1, VAL1, ...)

  Description:
    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW, ...)
    preprocesses glider deployment data according to given options,
    performing the following actions:
      - Selection of time sensors: 
        A time sequence is selected.
        Optionally, a unit conversion may be applied, if needed.
        This sequence is mandatory, processing aborts if missing.
      - Selection of horizontal position sensors:
        Latitude and longitude sequences are selected.
        A position flag is also selected, if any, and bad values are masked
        as missing. Optionally, a unit conversion may be applied, if needed.
        Latitude and longitude are mandatory, processing aborts if missing.
      - Selection of optional reference sensors:
        Navigation depth, pitch, roll, and heading sequences are selected,
        if any. Unit conversions may be applied, if needed.
      - Selection of water velocity sensors or estimates.
        Mean segment water eastward and northward velocity sequences are 
        selected, if any. Unit conversions may be applied, if needed.
      - Selection of commanded trajectory sensors:
        Commanded waypoint longitude and latitude sequences are selected,
        if any. Unit conversions may be applied, if needed.
      - Selection of CTD sensor:
        Conductivity, temperature and pressure sequences are selected, if any.
        Optionally the CTD timestamp sequence is selected, too.
        Unit conversion may be applied to pressure readings, if needed; and 
        factory calibrations may be applied to temperature and conductivity.
      - Selection of fluorescence and/or scattering sensor:
        Chlorophyll, turbidity, CDOM and/or scattering sequences are selected, 
        if any. Optionally the sensor timestamp sequence is selected, too.
        Manufacturer calibrations may be applied, if needed.
      - Selection of oxygen sensors:
        Oxygen concentration and saturation sequences are selected, if any.
        Optionally oxygen sensor timestamp and temperature sequences are 
        selected. Unit conversion and manufacturer calibrations may be applied.
      - Selection of other sensors of interest:
        Sequences from extra sensors configured in options are selected.
        Unit conversions and manufacturer calibrations may be applied,
        if needed.

    DATA_RAW should be a struct in the format returned by LOADSLOCUMDATA or
    LOADSEAGLIDERDATA, where each field is a vector sequence from the sensor
    or variable with the same name. META_RAW should be the struct with the
    metadata required for the preprocessing. Currently it is only used for
    the following actions:
      - Seaglider log parameter alignment (see option SG_DIVE_PARAMS below).

    Preprocessed data is returned in struct DATA_PRE, where each field is a
    sequence of readings selected and preprocessed according to given options
    (see below). META_PRE is a struct with the same fields keeping track of
    the source of each selected sequence and how it has been modified.

    Options may be given in key-value pairs OPT1, VAL1... or in a struct
    OPTIONS with field names as option keys and field values as option values.
    Recognized options are:
      SG_DIVE_PARAMS: Seaglider dive parameters to align with column data.
        String cell array with the names of the Seaglider log parameters that
        need to align with the rest of the data collected during the dive as
        required by function ALIGNSGDIVEPARAMS.
        Default value: {}
      TIME_LIST: time sensor choices.
        Struct array with the time sensor choices, in order of preference.
        It should have the following fields:
          TIME: time sequence name.
        It may have the following optional fields (empty or missing):
          CONVERSION: time unit conversion.
            Handle or name of the time unit conversion.
            If present and not empty, the selected sequence is converted 
            through that function.
        Default value: struct('time', {'m_present_time' 'sci_m_present_time'})
      POSITION_LIST: longitude and latitude sensor choices.
        Struct array selecting longitude and latitude sensor sets in order
        of preference, with optional mask of valid position readings.
        It should have the following fields:
          LONGITUDE: longitude sequence name.
          LATITUDE: latitude sequence name.
        It may have the following optional fields (empty or missing):
          POSITION_STATUS: position status sequence name.
          POSITION_GOOD: position status good values or filter.
          POSITION_BAD:  position status bad values or filter.
          CONVERSION: position coordinate conversion.
            Handle or name of the position coordinate conversion function.
            If present and not empty, the selected longitude and latitude 
            sequences are converted through this function.
          TIME: time component of position reading timestamp.
          DATE: date component of position reading timestamp.
          TIME_CONVERSION: position timestamp conversion.
            Handle or name of the position timestamp conversion function.
            If present and not empty, the selected position time stamp and
            position date stamp sequences are converted through this function
            to a sequence of absolute timepstamps.
        Default value: struct('longitude', {'m_gps_lon' 'm_lon'}, ...
                              'latitude', {'m_gps_lat' 'm_lat'}, ...
                              'position_status', {'m_gps_status' []}, ...
                              'position_status_good', {0 []}, ...
                              'position_status_bad', {[] []}, ...
                              'conversion', {@nmea2deg @nmea2deg})
      DEPTH_LIST: depth sensor choices.
        Struct array with the depth sensor choices, in order of preference.
        It should have the following fields:
          DEPTH: depth sequence name.
        It may have the following optional fields (empty or missing):
          CONVERSION: depth unit conversion.
            Handle or name of the depth coordinate conversion function.
            If present and not empty, the selected depth sequence is converted 
            through this function. 
        Default value: struct('depth', {'m_depth'});
      ATTITUDE_LIST: roll and pitch sensor choices.
        Struct array with the roll and pitch sensor choices sequences, 
        in order of preference. It should have the following fields:
          ROLL: roll sensor name.
          PITCH: pitch sensor name.
        It may have the following optional fields (empty or missing):
          CONVERSION: roll and pitch unit conversion.
            Handle or name of the attitude conversion function.
            If present and not empty, each selected roll and pitch sequence
            is converted through this function.
        Default value: struct('roll', {'m_roll'}, 'pitch', {'m_pitch'})
      HEADING_LIST: heading sensor choices.
        Struct array with the heading sensor choices, in order of preference.
        It should have the following fields:
          HEADING: heading sequence name.
        It may have the following optional fields (empty or missing):
          CONVERSION: heading unit conversion.
            Handle or name of the heading unit conversion.
            If present and not empty, the selected sequence is converted through
            that function.
        Default value: struct('heading', {'m_heading'})
      WAYPOINT_LIST: waypoint longitude and latitude choices.
        Struct array selecting waypoint longitude and latitude sequences,
        in order of preference. It should have the following fields:
          LONGITUDE: waypoint longitude sequence name.
          LATITUDE: waypoint latitude sequence name.
        It may have the following optional fields (empty or missing):
          CONVERSION: position coordinate conversion.
            Handle or name of the position coordinate conversion function.
            If present and not empty, the selected waypoint longitude and 
            latitude sequences are converted through this function. 
        Default value: struct('longitude', {'c_wpt_lon'}, ...
                              'latitude',  {'c_wpt_lat'}, ...
                              'conversion', {@nmea2deg})
      WATER_VELOCITY_LIST: water velocity choices.
        Struct array selecting estimates water velocity components,
        in order of preference. It should have the following fields:
          VELOCITY_EASTWARD: water velocity eastward component sensor name.
          VELOCITY_NORTHWARD: water velocity northward component sensor name.
          CONVERSION: water velocity conversion.
            Handle or name of the water velocity conversion function.
            If present and not empty, the selected water velocity component 
            sequences are converted through this function. 
        Default value: struct('velocity_eastward', {'m_final_water_vx'}, ...
                              'velocity_northward', {'m_final_water_vy'})
      CTD_LIST: CTD sensor choices.
        Struct array selecting the CTD sensors, in order of preference.
        It should have the following fields:
          CONDUCTIVITY: conductivity sequence name.
          TEMPERATURE: temperature sequence name.
          PRESSURE: pressure sequence name.
        It may have the following optional fields (empty or missing):
          TIME: CTD timestamp sequence name.
          PRESSURE_CONVERSION: pressure unit conversion.
            Handle or name of the pressure unit conversion function.
            If present and not empty, the selected pressure sequence is 
            converted through this function.
          CALIBRATION: conductivity and temperature factory calibration.
            Handle or name of the temperature and conductivity factory
            calibration function. If present and not empty, the raw 
            conductivity and temperature sequences, and the pressure sequence
            are passed to this function to get the calibrated temperature
            and conductivity.
        Default value:
          struct('conductivity', {'sci_water_cond'        'm_water_cond'}, ...
                 'temperature',  {'sci_water_temp'        'm_water_temp'}, ...
                 'pressure',     {'sci_water_pressure'    'm_water_pressure'}, ...
                 'time',         {'sci_ctd41cp_timestamp' []}
                 'pressure_conversion', {@bar2dbar        @bar2dbar})
      OXYGEN_LIST: oxygen sensor set choices.
        Struct array selecting the oxygen sensor sets, in order of preference.
        It should have any of the following fields:
          OXYGEN_CONCENTRATION: concentration of oxygen sequence name.
          OXYGEN_SATURATION: saturation of oxygen sequence name.
          OXYGEN_FREQUENCY: frequency of oxygen sensor sequence name.
        It may have the following optional fields (empty or missing):
          TEMPERATURE: oxygen temperature sequence name.
          TIME: oxygen timestamp sequence name.
        Default value: struct('oxygen_concentration', {'sci_oxy3835_oxygen'}, ...
                              'oxygen_saturation',    {'sci_oxy3835_saturation'}, ...
                              'temperature',          {'sci_oxy3835_temp'}, ...
                              'time',                 {'sci_oxy3835_timestamp'})
      OPTICS_LIST: fluorescence and scattering sensor set choices.
        Struct array selecting the fluorescence and scattering sensor sets,
        in  order of preference. It should have any of the following fields:
          CHLOROPHYLL: chlorophyl sequence name.
          TURBIDITY: turbidity sequence name.
          CDOM: CDOM sequence name.
          SCATTER_650: 650 nm wavelength scattering sequence name.
          BACKSCATTER_700: 700 nm wavelength backscattering sequence name.
        It may have the following optional fields (empty or missing):
          TEMPERATURE: optic sensor temperature sequence name.
          TIME: optic sensor timestamp sequence name.
          CALIBRATION: fluorescence and scattering factory calibration.
            Handle or name of the optic sensor factory calibration function.
            If present and not empty, the selected raw sequences are passed to
            this function to get the calibrated optic measurements.
        Default value: struct('chlorophyll',     {'sci_flntu_chlor_units'}, ...
                              'turbidity',       {'sci_flntu_turb_units'}, ...
                              'cdom',            {[]}, ...
                              'scatter_650',     {[]}, ...
                              'backscatter_700', {[]}, ...
                              'temperature',     {'sci_flntu_temp'}, ...
                              'time',            {'sci_flntu_timestamp'})
      EXTRA_SENSOR_LIST: other sensor set choices.
        Struct selecting other sensor sets of interest, where each field 
        represents a sensor of interest. The field name is an arbitrary sensor 
        name (e.g. battery_info), and the field value should be a struct array 
        with the sensor choices in order of preference, where field names are
        the final sequence names (fields in struct DATA_PRE, e.g. 
        battery_nominal_capacity and battery_total_consumption) and field 
        values are the original sequence name choices (fields in struct 
        DATA_RAW, e.g. f_coulomb_battery_capacity and m_coulomb_amphr_total).
        Optionally, each sensor choice may include a special field named
        CALIBRATION with the handle or name of the extra sensor factory
        calibration function. If present and not empty, the selected raw
        sequences are passed to this function to get the calibrated
        extra sensor measurements.
        Default value: struct()
      CALIBRATION_PARAMETER_LIST: calibration parameters for each variable.
        Struct with the calibration parameters of each uncalibrated variable.
        For each raw variable with calibration parameters there should be a 
        field with the same name and whose value is a struct with the parameter
        names as field names and its parameter values as field values.
        Default value: struct()

  Examples:
    data_pre = preprocessGliderData(data_raw, meta_raw, options)

  See also:
    ALIGNSGPARAMS
    FILLSGMISSINGGPSDATE
    NMEA2DEG
    BAR2DBAR
    SGDEPTH2PRES
    CALIBRATESBECT
    CALIBRATEWLECOBBFL2

  Authors:
    Joan Pau Beltran  <joanpau.beltran@socib.cat>

0001 function [data_pre, meta_pre] = preprocessGliderData(data_raw, meta_raw, varargin)
0002 %PREPROCESSGLIDERDATA  Preprocess glider deployment data.
0003 %
0004 %  Syntax:
0005 %    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW)
0006 %    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW, OPTIONS)
0007 %    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW, OPT1, VAL1, ...)
0008 %
0009 %  Description:
0010 %    [DATA_PRE, META_PRE] = PREPROCESSGLIDERDATA(DATA_RAW, META_RAW, ...)
0011 %    preprocesses glider deployment data according to given options,
0012 %    performing the following actions:
0013 %      - Selection of time sensors:
0014 %        A time sequence is selected.
0015 %        Optionally, a unit conversion may be applied, if needed.
0016 %        This sequence is mandatory, processing aborts if missing.
0017 %      - Selection of horizontal position sensors:
0018 %        Latitude and longitude sequences are selected.
0019 %        A position flag is also selected, if any, and bad values are masked
0020 %        as missing. Optionally, a unit conversion may be applied, if needed.
0021 %        Latitude and longitude are mandatory, processing aborts if missing.
0022 %      - Selection of optional reference sensors:
0023 %        Navigation depth, pitch, roll, and heading sequences are selected,
0024 %        if any. Unit conversions may be applied, if needed.
0025 %      - Selection of water velocity sensors or estimates.
0026 %        Mean segment water eastward and northward velocity sequences are
0027 %        selected, if any. Unit conversions may be applied, if needed.
0028 %      - Selection of commanded trajectory sensors:
0029 %        Commanded waypoint longitude and latitude sequences are selected,
0030 %        if any. Unit conversions may be applied, if needed.
0031 %      - Selection of CTD sensor:
0032 %        Conductivity, temperature and pressure sequences are selected, if any.
0033 %        Optionally the CTD timestamp sequence is selected, too.
0034 %        Unit conversion may be applied to pressure readings, if needed; and
0035 %        factory calibrations may be applied to temperature and conductivity.
0036 %      - Selection of fluorescence and/or scattering sensor:
0037 %        Chlorophyll, turbidity, CDOM and/or scattering sequences are selected,
0038 %        if any. Optionally the sensor timestamp sequence is selected, too.
0039 %        Manufacturer calibrations may be applied, if needed.
0040 %      - Selection of oxygen sensors:
0041 %        Oxygen concentration and saturation sequences are selected, if any.
0042 %        Optionally oxygen sensor timestamp and temperature sequences are
0043 %        selected. Unit conversion and manufacturer calibrations may be applied.
0044 %      - Selection of other sensors of interest:
0045 %        Sequences from extra sensors configured in options are selected.
0046 %        Unit conversions and manufacturer calibrations may be applied,
0047 %        if needed.
0048 %
0049 %    DATA_RAW should be a struct in the format returned by LOADSLOCUMDATA or
0050 %    LOADSEAGLIDERDATA, where each field is a vector sequence from the sensor
0051 %    or variable with the same name. META_RAW should be the struct with the
0052 %    metadata required for the preprocessing. Currently it is only used for
0053 %    the following actions:
0054 %      - Seaglider log parameter alignment (see option SG_DIVE_PARAMS below).
0055 %
0056 %    Preprocessed data is returned in struct DATA_PRE, where each field is a
0057 %    sequence of readings selected and preprocessed according to given options
0058 %    (see below). META_PRE is a struct with the same fields keeping track of
0059 %    the source of each selected sequence and how it has been modified.
0060 %
0061 %    Options may be given in key-value pairs OPT1, VAL1... or in a struct
0062 %    OPTIONS with field names as option keys and field values as option values.
0063 %    Recognized options are:
0064 %      SG_DIVE_PARAMS: Seaglider dive parameters to align with column data.
0065 %        String cell array with the names of the Seaglider log parameters that
0066 %        need to align with the rest of the data collected during the dive as
0067 %        required by function ALIGNSGDIVEPARAMS.
0068 %        Default value: {}
0069 %      TIME_LIST: time sensor choices.
0070 %        Struct array with the time sensor choices, in order of preference.
0071 %        It should have the following fields:
0072 %          TIME: time sequence name.
0073 %        It may have the following optional fields (empty or missing):
0074 %          CONVERSION: time unit conversion.
0075 %            Handle or name of the time unit conversion.
0076 %            If present and not empty, the selected sequence is converted
0077 %            through that function.
0078 %        Default value: struct('time', {'m_present_time' 'sci_m_present_time'})
0079 %      POSITION_LIST: longitude and latitude sensor choices.
0080 %        Struct array selecting longitude and latitude sensor sets in order
0081 %        of preference, with optional mask of valid position readings.
0082 %        It should have the following fields:
0083 %          LONGITUDE: longitude sequence name.
0084 %          LATITUDE: latitude sequence name.
0085 %        It may have the following optional fields (empty or missing):
0086 %          POSITION_STATUS: position status sequence name.
0087 %          POSITION_GOOD: position status good values or filter.
0088 %          POSITION_BAD:  position status bad values or filter.
0089 %          CONVERSION: position coordinate conversion.
0090 %            Handle or name of the position coordinate conversion function.
0091 %            If present and not empty, the selected longitude and latitude
0092 %            sequences are converted through this function.
0093 %          TIME: time component of position reading timestamp.
0094 %          DATE: date component of position reading timestamp.
0095 %          TIME_CONVERSION: position timestamp conversion.
0096 %            Handle or name of the position timestamp conversion function.
0097 %            If present and not empty, the selected position time stamp and
0098 %            position date stamp sequences are converted through this function
0099 %            to a sequence of absolute timepstamps.
0100 %        Default value: struct('longitude', {'m_gps_lon' 'm_lon'}, ...
0101 %                              'latitude', {'m_gps_lat' 'm_lat'}, ...
0102 %                              'position_status', {'m_gps_status' []}, ...
0103 %                              'position_status_good', {0 []}, ...
0104 %                              'position_status_bad', {[] []}, ...
0105 %                              'conversion', {@nmea2deg @nmea2deg})
0106 %      DEPTH_LIST: depth sensor choices.
0107 %        Struct array with the depth sensor choices, in order of preference.
0108 %        It should have the following fields:
0109 %          DEPTH: depth sequence name.
0110 %        It may have the following optional fields (empty or missing):
0111 %          CONVERSION: depth unit conversion.
0112 %            Handle or name of the depth coordinate conversion function.
0113 %            If present and not empty, the selected depth sequence is converted
0114 %            through this function.
0115 %        Default value: struct('depth', {'m_depth'});
0116 %      ATTITUDE_LIST: roll and pitch sensor choices.
0117 %        Struct array with the roll and pitch sensor choices sequences,
0118 %        in order of preference. It should have the following fields:
0119 %          ROLL: roll sensor name.
0120 %          PITCH: pitch sensor name.
0121 %        It may have the following optional fields (empty or missing):
0122 %          CONVERSION: roll and pitch unit conversion.
0123 %            Handle or name of the attitude conversion function.
0124 %            If present and not empty, each selected roll and pitch sequence
0125 %            is converted through this function.
0126 %        Default value: struct('roll', {'m_roll'}, 'pitch', {'m_pitch'})
0127 %      HEADING_LIST: heading sensor choices.
0128 %        Struct array with the heading sensor choices, in order of preference.
0129 %        It should have the following fields:
0130 %          HEADING: heading sequence name.
0131 %        It may have the following optional fields (empty or missing):
0132 %          CONVERSION: heading unit conversion.
0133 %            Handle or name of the heading unit conversion.
0134 %            If present and not empty, the selected sequence is converted through
0135 %            that function.
0136 %        Default value: struct('heading', {'m_heading'})
0137 %      WAYPOINT_LIST: waypoint longitude and latitude choices.
0138 %        Struct array selecting waypoint longitude and latitude sequences,
0139 %        in order of preference. It should have the following fields:
0140 %          LONGITUDE: waypoint longitude sequence name.
0141 %          LATITUDE: waypoint latitude sequence name.
0142 %        It may have the following optional fields (empty or missing):
0143 %          CONVERSION: position coordinate conversion.
0144 %            Handle or name of the position coordinate conversion function.
0145 %            If present and not empty, the selected waypoint longitude and
0146 %            latitude sequences are converted through this function.
0147 %        Default value: struct('longitude', {'c_wpt_lon'}, ...
0148 %                              'latitude',  {'c_wpt_lat'}, ...
0149 %                              'conversion', {@nmea2deg})
0150 %      WATER_VELOCITY_LIST: water velocity choices.
0151 %        Struct array selecting estimates water velocity components,
0152 %        in order of preference. It should have the following fields:
0153 %          VELOCITY_EASTWARD: water velocity eastward component sensor name.
0154 %          VELOCITY_NORTHWARD: water velocity northward component sensor name.
0155 %          CONVERSION: water velocity conversion.
0156 %            Handle or name of the water velocity conversion function.
0157 %            If present and not empty, the selected water velocity component
0158 %            sequences are converted through this function.
0159 %        Default value: struct('velocity_eastward', {'m_final_water_vx'}, ...
0160 %                              'velocity_northward', {'m_final_water_vy'})
0161 %      CTD_LIST: CTD sensor choices.
0162 %        Struct array selecting the CTD sensors, in order of preference.
0163 %        It should have the following fields:
0164 %          CONDUCTIVITY: conductivity sequence name.
0165 %          TEMPERATURE: temperature sequence name.
0166 %          PRESSURE: pressure sequence name.
0167 %        It may have the following optional fields (empty or missing):
0168 %          TIME: CTD timestamp sequence name.
0169 %          PRESSURE_CONVERSION: pressure unit conversion.
0170 %            Handle or name of the pressure unit conversion function.
0171 %            If present and not empty, the selected pressure sequence is
0172 %            converted through this function.
0173 %          CALIBRATION: conductivity and temperature factory calibration.
0174 %            Handle or name of the temperature and conductivity factory
0175 %            calibration function. If present and not empty, the raw
0176 %            conductivity and temperature sequences, and the pressure sequence
0177 %            are passed to this function to get the calibrated temperature
0178 %            and conductivity.
0179 %        Default value:
0180 %          struct('conductivity', {'sci_water_cond'        'm_water_cond'}, ...
0181 %                 'temperature',  {'sci_water_temp'        'm_water_temp'}, ...
0182 %                 'pressure',     {'sci_water_pressure'    'm_water_pressure'}, ...
0183 %                 'time',         {'sci_ctd41cp_timestamp' []}
0184 %                 'pressure_conversion', {@bar2dbar        @bar2dbar})
0185 %      OXYGEN_LIST: oxygen sensor set choices.
0186 %        Struct array selecting the oxygen sensor sets, in order of preference.
0187 %        It should have any of the following fields:
0188 %          OXYGEN_CONCENTRATION: concentration of oxygen sequence name.
0189 %          OXYGEN_SATURATION: saturation of oxygen sequence name.
0190 %          OXYGEN_FREQUENCY: frequency of oxygen sensor sequence name.
0191 %        It may have the following optional fields (empty or missing):
0192 %          TEMPERATURE: oxygen temperature sequence name.
0193 %          TIME: oxygen timestamp sequence name.
0194 %        Default value: struct('oxygen_concentration', {'sci_oxy3835_oxygen'}, ...
0195 %                              'oxygen_saturation',    {'sci_oxy3835_saturation'}, ...
0196 %                              'temperature',          {'sci_oxy3835_temp'}, ...
0197 %                              'time',                 {'sci_oxy3835_timestamp'})
0198 %      OPTICS_LIST: fluorescence and scattering sensor set choices.
0199 %        Struct array selecting the fluorescence and scattering sensor sets,
0200 %        in  order of preference. It should have any of the following fields:
0201 %          CHLOROPHYLL: chlorophyl sequence name.
0202 %          TURBIDITY: turbidity sequence name.
0203 %          CDOM: CDOM sequence name.
0204 %          SCATTER_650: 650 nm wavelength scattering sequence name.
0205 %          BACKSCATTER_700: 700 nm wavelength backscattering sequence name.
0206 %        It may have the following optional fields (empty or missing):
0207 %          TEMPERATURE: optic sensor temperature sequence name.
0208 %          TIME: optic sensor timestamp sequence name.
0209 %          CALIBRATION: fluorescence and scattering factory calibration.
0210 %            Handle or name of the optic sensor factory calibration function.
0211 %            If present and not empty, the selected raw sequences are passed to
0212 %            this function to get the calibrated optic measurements.
0213 %        Default value: struct('chlorophyll',     {'sci_flntu_chlor_units'}, ...
0214 %                              'turbidity',       {'sci_flntu_turb_units'}, ...
0215 %                              'cdom',            {[]}, ...
0216 %                              'scatter_650',     {[]}, ...
0217 %                              'backscatter_700', {[]}, ...
0218 %                              'temperature',     {'sci_flntu_temp'}, ...
0219 %                              'time',            {'sci_flntu_timestamp'})
0220 %      EXTRA_SENSOR_LIST: other sensor set choices.
0221 %        Struct selecting other sensor sets of interest, where each field
0222 %        represents a sensor of interest. The field name is an arbitrary sensor
0223 %        name (e.g. battery_info), and the field value should be a struct array
0224 %        with the sensor choices in order of preference, where field names are
0225 %        the final sequence names (fields in struct DATA_PRE, e.g.
0226 %        battery_nominal_capacity and battery_total_consumption) and field
0227 %        values are the original sequence name choices (fields in struct
0228 %        DATA_RAW, e.g. f_coulomb_battery_capacity and m_coulomb_amphr_total).
0229 %        Optionally, each sensor choice may include a special field named
0230 %        CALIBRATION with the handle or name of the extra sensor factory
0231 %        calibration function. If present and not empty, the selected raw
0232 %        sequences are passed to this function to get the calibrated
0233 %        extra sensor measurements.
0234 %        Default value: struct()
0235 %      CALIBRATION_PARAMETER_LIST: calibration parameters for each variable.
0236 %        Struct with the calibration parameters of each uncalibrated variable.
0237 %        For each raw variable with calibration parameters there should be a
0238 %        field with the same name and whose value is a struct with the parameter
0239 %        names as field names and its parameter values as field values.
0240 %        Default value: struct()
0241 %
0242 %  Examples:
0243 %    data_pre = preprocessGliderData(data_raw, meta_raw, options)
0244 %
0245 %  See also:
0246 %    ALIGNSGPARAMS
0247 %    FILLSGMISSINGGPSDATE
0248 %    NMEA2DEG
0249 %    BAR2DBAR
0250 %    SGDEPTH2PRES
0251 %    CALIBRATESBECT
0252 %    CALIBRATEWLECOBBFL2
0253 %
0254 %  Authors:
0255 %    Joan Pau Beltran  <joanpau.beltran@socib.cat>
0256 
0257 %  Copyright (C) 2013-2016
0258 %  ICTS SOCIB - Servei d'observacio i prediccio costaner de les Illes Balears
0259 %  <http://www.socib.es>
0260 %
0261 %  This program is free software: you can redistribute it and/or modify
0262 %  it under the terms of the GNU General Public License as published by
0263 %  the Free Software Foundation, either version 3 of the License, or
0264 %  (at your option) any later version.
0265 %
0266 %  This program is distributed in the hope that it will be useful,
0267 %  but WITHOUT ANY WARRANTY; without even the implied warranty of
0268 %  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
0269 %  GNU General Public License for more details.
0270 %
0271 %  You should have received a copy of the GNU General Public License
0272 %  along with this program.  If not, see <http://www.gnu.org/licenses/>.
0273 
0274   error(nargchk(2, 24, nargin, 'struct'));
0275   
0276   
0277   %% Set preprocessing options.
0278   % Set default options values.
0279   options = struct();
0280   options.sg_dive_params = {};
0281   options.time_list = ...
0282     struct('time', {'m_present_time' 'sci_m_present_time'});
0283   options.position_list = ...
0284     struct('longitude',            {'m_gps_lon'    'm_lon'}, ...
0285            'latitude',             {'m_gps_lat'    'm_lat'}, ...
0286            'position_status',      {'m_gps_status' []}, ...
0287            'position_good',        {0              []}, ...
0288            'position_bad',         {[]             []}, ...
0289            'time',                 {[]             []}, ...
0290            'date',                 {[]             []}, ...
0291            'conversion',           {@nmea2deg      @nmea2deg});
0292   options.depth_list = ...
0293     struct('depth', {'m_depth'});
0294   options.attitude_list = ...
0295     struct('roll',  {'m_roll'}, ...
0296            'pitch', {'m_pitch'});
0297   options.heading_list = ...
0298     struct('heading', {'m_heading'});
0299   options.waypoint_list = ...
0300     struct('longitude',  {'c_wpt_lon'}, ...
0301            'latitude',   {'c_wpt_lat'}, ...
0302            'conversion', {@nmea2deg});
0303   options.water_velocity_list = ...
0304     struct('velocity_eastward',  {'m_final_water_vx'}, ...
0305            'velocity_northward', {'m_final_water_vy'});
0306   options.ctd_list = ...
0307     struct('conductivity',        {'sci_water_cond'        'm_water_cond'}, ...
0308            'temperature',         {'sci_water_temp'        'm_water_temp'}, ...
0309            'pressure',            {'sci_water_pressure'    'm_water_pressure'}, ...
0310            'time',                {'sci_ctd41cp_timestamp' []}, ...
0311            'pressure_conversion', {@bar2dbar               @bar2dbar});
0312   options.oxygen_list = ...
0313     struct('oxygen_concentration', {'sci_oxy3835_oxygen'}, ...
0314            'oxygen_saturation',    {'sci_oxy3835_saturation'}, ...
0315            'temperature',          {'sci_oxy3835_temp'}, ...
0316            'time',                 {'sci_oxy3835_timestamp'});
0317   options.optics_list = ...
0318     struct('chlorophyll',     {'sci_flntu_chlor_units'}, ...
0319            'turbidity',       {'sci_flntu_turb_units'}, ...
0320            'cdom',            {[]}, ...
0321            'scatter_650',     {[]}, ...
0322            'backscatter_700', {[]}, ...
0323            'temperature',     {'sci_flntu_temp'}, ...
0324            'time',            {'sci_flntu_timestamp'});
0325   options.extra_sensor_list = ...
0326     struct();
0327   options.calibration_parameter_list = ...
0328     struct();
0329   
0330   
0331   %% Get options from extra arguments.
0332   % Parse option key-value pairs in any accepted call signature.
0333   if isscalar(varargin) && isstruct(varargin{1})
0334     % Options passed as a single option struct argument.
0335     option_key_list = fieldnames(varargin{1});
0336     option_val_list = struct2cell(varargin{1});
0337   elseif mod(numel(varargin), 2) == 0
0338     % Options passed as key-value argument pairs.
0339     option_key_list = varargin(1:2:end);
0340     option_val_list = varargin(2:2:end);
0341   else
0342     error('glider_toolbox:preprocessGliderData:InvalidOption', ...
0343           'Invalid optional arguments (neither key-value pairs nor struct).');
0344   end
0345   % Overwrite default options with values given in extra arguments.
0346   for opt_idx = 1:numel(option_key_list)
0347     opt = lower(option_key_list{opt_idx});
0348     val = option_val_list{opt_idx};
0349     if isfield(options, opt)
0350       options.(opt) = val;
0351     else
0352       error('glider_toolbox:preprocessGliderData:InvalidOption', ...
0353             'Invalid option: %s.', opt);
0354     end
0355   end
0356 
0357 
0358   %% Initialize output data.
0359   meta_pre = struct();
0360   data_pre = struct();
0361   
0362   
0363   %% Get list of available variables and their calibrations.
0364   field_list = fieldnames(data_raw);
0365   calibration_parameters = struct();
0366   if isfield(options, 'calibration_parameter_list')
0367     calibration_parameters = options.calibration_parameter_list;
0368   end
0369   
0370   
0371   %% Realign Seaglider log parameters to the start of the dive data, if needed.
0372   if ~isempty(options.sg_dive_params)
0373     data_raw = alignSGDiveParams(data_raw, meta_raw, options.sg_dive_params);
0374   end
0375   
0376   
0377   %% Select time coordinate sensor.
0378   % Find preferred valid time sensor available in list of sensor fields.
0379   % For Slocum data please be aware of the efects of program dba_merge,
0380   % namely the copy of the sci_m_present_time value to the m_present_time for
0381   % sensor cycles coming from the science board.
0382   % Convert char array to string cell array if needed (safe if cell array).
0383   time_choice_list = options.time_list;
0384   for time_choice_idx = 1:numel(time_choice_list)
0385     time_choice = time_choice_list(time_choice_idx);
0386     time_field = time_choice.time;
0387     time_conversion_func = [];
0388     if isfield(time_choice_list, 'conversion')
0389       time_conversion_func = time_choice.conversion;
0390     end
0391     if ismember(time_field, field_list) && any(data_raw.(time_field) > 0)
0392       data_pre.time = data_raw.(time_field);
0393       meta_pre.time.sources = time_field;
0394       fprintf('Selected time sensor %d:\n', time_choice_idx); ...
0395       fprintf('  time: %s\n', time_field);
0396       if ~isempty(time_conversion_func)
0397         if ischar(time_conversion_func)
0398           time_conversion_func = str2func(time_conversion_func);
0399         end
0400         data_pre.time = time_conversion_func(data_pre.time);
0401         meta_pre.time.conversion = func2str(time_conversion_func);
0402         fprintf('  conversion: %s\n', func2str(time_conversion_func));
0403       end
0404       break
0405     end
0406   end
0407   if ~isfield(data_pre, 'time')
0408     error('glider_toolbox:preprocessGliderData:MissingSensorTime', ...
0409           'No time sensor present in data set.');
0410   end
0411   
0412   
0413   %% Select position coordinate sensors.
0414   % Find preferred valid longitude and latitude sensor available in list of
0415   % sensor fields. Also find optional date and time components of position
0416   % timestamp. Optionally set flagged readings as invalid (NaN),
0417   % if position status field and good or bad value criteria are available.
0418   position_choice_list = options.position_list;
0419   for position_choice_idx = 1:numel(position_choice_list)
0420     position_choice = position_choice_list(position_choice_idx);
0421     lon_field = position_choice.longitude;
0422     lat_field = position_choice.latitude;
0423     position_date_field = [];
0424     position_time_field = [];
0425     position_status_field = [];
0426     position_conversion_func = [];
0427     position_time_conversion_func = [];
0428     position_good = [];
0429     position_bad = [];
0430     if isfield(position_choice_list, 'position_status')
0431       position_status_field = position_choice.position_status;
0432     end
0433     if isfield(position_choice_list, 'position_good')
0434       position_good = position_choice.position_good;
0435     end
0436     if isfield(position_choice_list, 'position_bad')
0437       position_bad = position_choice.position_bad;
0438     end
0439     if isfield(position_choice_list, 'conversion')
0440       position_conversion_func = position_choice.conversion;
0441     end
0442     if isfield(position_choice_list, 'time')
0443       position_time_field = position_choice.time;
0444     end
0445     if isfield(position_choice_list, 'date')
0446       position_date_field = position_choice.date;
0447     end
0448     if isfield(position_choice_list, 'time_conversion')
0449       position_time_conversion_func = position_choice.time_conversion;
0450     end
0451     if all(ismember({lon_field lat_field}, field_list)) ...
0452         && ~all(isnan(data_raw.(lon_field))) ...
0453         && ~all(isnan(data_raw.(lat_field)))
0454       data_pre.longitude = data_raw.(lon_field);
0455       data_pre.latitude = data_raw.(lat_field);
0456       meta_pre.longitude.sources = lon_field;
0457       meta_pre.latitude.sources = lat_field;
0458       fprintf('Selected position sensor %d:\n', position_choice_idx);
0459       fprintf('  longitude: %s\n', lon_field);
0460       fprintf('  latitude : %s\n', lat_field);
0461       if ~isempty(position_status_field) ...
0462           && ismember(position_status_field, field_list) ...
0463           && ~all(isnan(data_raw.(position_status_field)))
0464         data_pre.position_status = data_raw.(position_status_field);
0465         meta_pre.position_status.sources = position_status_field;
0466         fprintf('  position status: %s\n', position_status_field);
0467         position_invalid = false(size(data_pre.position_status));
0468         if ~isempty(position_good)
0469           if ischar(position_good) 
0470             position_invalid = ~feval(position_good, ...
0471                                       data_pre.longitude, data_pre.latitude, ...
0472                                       data_pre.position_status);
0473             meta_pre.position_status.position_good = position_good;
0474             fprintf('  position good  : %s\n', position_good);
0475           elseif isa(position_good, 'function_handle')
0476             position_invalid = ~feval(position_good, ...
0477                                       data_pre.longitude, data_pre.latitude, ...
0478                                       data_pre.position_status);
0479             meta_pre.position_status.position_good = func2str(position_good);
0480             fprintf('  position good  : %s\n', func2str(position_good));
0481           else
0482             position_invalid = ...
0483               ~ismember(data_pre.position_status, position_good);
0484             meta_pre.position_status.position_good = position_good;
0485             fprintf('  position good  : %s\n', num2str(position_good));
0486           end
0487         end
0488         if ~isempty(position_bad)
0489           if ischar(position_bad) 
0490             position_invalid = position_invalid ...
0491                              | feval(position_bad, ...
0492                                      data_pre.longitude, data_pre.latitude, ...
0493                                      data_pre.position_status);
0494             meta_pre.position_status.position_bad = position_bad;
0495             fprintf('  position bad   : %s\n', position_bad);
0496           elseif isa(position_bad, 'function_handle')
0497             position_invalid = position_invalid ...
0498                              | feval(position_bad, ...
0499                                      data_pre.longitude, data_pre.latitude, ...
0500                                      data_pre.position_status);
0501             meta_pre.position_status.position_bad = func2str(position_bad);
0502             fprintf('  position bad   : %s\n', func2str(position_bad));
0503           else
0504             position_invalid = position_invalid ...
0505                              | ismember(data_pre.position_status, position_bad);
0506             meta_pre.position_status.position_bad = position_bad;
0507             fprintf('  position bad   : %s\n', num2str(position_bad));
0508           end
0509         end
0510         data_pre.longitude(position_invalid) = nan;
0511         data_pre.latitude(position_invalid) = nan;
0512         meta_pre.longitude.sources = ...
0513           {lon_field lat_field position_status_field}';
0514         meta_pre.latitude.sources = ...
0515           {lon_field lat_field position_status_field}';
0516         if isfield(meta_pre.position_status, 'position_good')
0517           meta_pre.longitude.position_good = ...
0518             meta_pre.position_status.position_good;
0519           meta_pre.latitude.position_good = ...
0520             meta_pre.position_status.position_good;
0521         end
0522         if isfield(meta_pre.position_status, 'position_bad')
0523           meta_pre.longitude.position_bad = ...
0524             meta_pre.position_status.position_bad;
0525           meta_pre.latitude.position_bad = ...
0526             meta_pre.position_status.position_bad;
0527         end
0528       end
0529       if ~isempty(position_conversion_func)
0530         if ischar(position_conversion_func)
0531           position_conversion_func = str2func(position_conversion_func);
0532         end
0533         [data_pre.longitude, data_pre.latitude] = ...
0534           position_conversion_func(data_pre.longitude, data_pre.latitude);
0535         meta_pre.longitude.sources = ...
0536           union(cellstr(meta_pre.longitude.sources), {lon_field lat_field}');
0537         meta_pre.longitude.conversion = func2str(position_conversion_func);
0538         meta_pre.latitude.sources = ...
0539           union(cellstr(meta_pre.latitude.sources), {lon_field lat_field}');
0540         meta_pre.latitude.conversion = func2str(position_conversion_func);
0541         fprintf('  conversion : %s\n', func2str(position_conversion_func));
0542       end
0543       if ~isempty(position_time_field) ...
0544           && ismember(position_time_field, field_list)
0545         data_pre.position_time = data_raw.(position_time_field);
0546         meta_pre.position_time.sources = position_time_field;
0547         fprintf('  position time  : %s\n', position_time_field);
0548       end
0549       if ~isempty(position_date_field) ...
0550           && ismember(position_date_field, field_list)
0551         data_pre.position_date = data_raw.(position_date_field);
0552         meta_pre.position_date.sources = position_date_field;
0553         fprintf('  position date  : %s\n', position_date_field);
0554       end
0555       if ~isempty(position_time_conversion_func)
0556         if ischar(position_time_conversion_func)
0557           position_time_conversion_func = ...
0558             str2func(position_time_conversion_func);
0559         end
0560         data_pre.time_position = ...
0561           position_time_conversion_func(data_pre.position_time, ...
0562                                         data_pre.position_date);
0563         meta_pre.time_position.sources = ...
0564           {position_time_field position_date_field}';
0565         meta_pre.time_position.conversion = ...
0566           func2str(position_time_conversion_func);
0567         fprintf('  time conversion : %s\n', ...
0568                 func2str(position_time_conversion_func));
0569       end
0570       break
0571     end
0572   end
0573   if ~all(isfield(data_pre, {'longitude' 'latitude'}))
0574     error('glider_toolbox:preprocessGliderData:MissingSensorPosition', ...
0575           'No longitude and latitude sensor present in data set.');
0576   end
0577   
0578   
0579   %% Select depth sensor.
0580   % Find preferred valid depth sensor available in list of sensor fields, if any.
0581   % Convert char array to string cell array if needed (safe if cell array).
0582   depth_choice_list = options.depth_list;
0583   for depth_choice_idx = 1:numel(depth_choice_list)
0584     depth_choice = depth_choice_list(depth_choice_idx);
0585     depth_field = depth_choice.depth;
0586     depth_conversion_func = [];
0587     if isfield(depth_choice_list, 'conversion')
0588       depth_conversion_func = depth_choice.conversion;
0589     end
0590     if ismember(depth_field, field_list) ...
0591         && ~all(isnan(data_raw.(depth_field)))
0592       data_pre.depth = data_raw.(depth_field);
0593       meta_pre.depth.sources = depth_field;
0594       fprintf('Selected depth sensor %d:\n', depth_choice_idx);
0595       fprintf('  depth: %s\n', depth_field);
0596       if ~isempty(depth_conversion_func)
0597         if ischar(depth_conversion_func)
0598           depth_conversion_func = str2func(depth_conversion_func);
0599         end
0600         data_pre.depth = depth_conversion_func(data_pre.depth);
0601         meta_pre.depth.conversion = func2str(depth_conversion_func);
0602         fprintf('  conversion : %s\n', func2str(depth_conversion_func));
0603       end
0604       break
0605     end
0606   end
0607   
0608   
0609   %% Select attitude sensors.
0610   % Find preferred valid roll and pitch sensor available in list of sensor
0611   % fields, if any.
0612   attitude_choice_list = options.attitude_list;
0613   for attitude_choice_idx = 1:numel(attitude_choice_list)
0614     attitude_choice = attitude_choice_list(attitude_choice_idx);
0615     roll_field = attitude_choice_list(attitude_choice_idx).roll;
0616     pitch_field = attitude_choice_list(attitude_choice_idx).pitch;
0617     attitude_conversion = [];
0618     if isfield(attitude_choice_list, 'conversion')
0619       attitude_conversion = attitude_choice.conversion;
0620     end
0621     if all(ismember({roll_field pitch_field}, field_list)) ...
0622         && ~all(isnan(data_raw.(roll_field))) ...
0623         && ~all(isnan(data_raw.(pitch_field))) 
0624       data_pre.roll = data_raw.(roll_field);
0625       data_pre.pitch = data_raw.(pitch_field);
0626       meta_pre.roll.sources = roll_field;
0627       meta_pre.pitch.sources = pitch_field;
0628       fprintf('Selected attitude sensors %d:\n', attitude_choice_idx);
0629       fprintf('  roll : %s\n', roll_field);
0630       fprintf('  pitch: %s\n', pitch_field);
0631       if ~isempty(attitude_conversion)
0632         if ischar(attitude_conversion)
0633           attitude_conversion = str2func(attitude_conversion);
0634         end
0635         data_pre.roll = attitude_conversion(data_pre.roll);
0636         data_pre.pitch = attitude_conversion(data_pre.pitch);
0637         meta_pre.roll.conversion = func2str(attitude_conversion);
0638         meta_pre.pitch.conversion = func2str(attitude_conversion);
0639         fprintf('  conversion : %s\n', func2str(attitude_conversion));
0640       end
0641       break
0642     end
0643   end
0644   
0645   
0646   %% Select heading sensor.
0647   % Find preferred valid heading sensor available in list of sensor fields.
0648   % Convert char array to string cell array if needed (safe if cell array).
0649   heading_choice_list = options.heading_list;
0650   for heading_choice_idx = 1:numel(heading_choice_list)
0651     heading_choice = heading_choice_list(heading_choice_idx);
0652     heading_field = heading_choice.heading;
0653     heading_conversion_func = [];
0654     if isfield(heading_choice_list, 'conversion')
0655       heading_conversion_func = heading_choice.conversion;
0656     end
0657     if ismember(heading_field, field_list) ...
0658         && ~all(isnan(data_raw.(heading_field)))
0659       data_pre.heading = data_raw.(heading_field);
0660       fprintf('Selected heading sensor %d:\n', heading_choice_idx);
0661       fprintf('  heading: %s\n', heading_field);
0662       if ~isempty(heading_conversion_func)
0663         if ischar(heading_conversion_func)
0664           heading_conversion_func = str2func(heading_conversion_func);
0665         end
0666         data_pre.heading = heading_conversion_func(data_pre.heading);
0667         meta_pre.heading.conversion = func2str(heading_conversion_func);
0668         fprintf('  conversion : %s\n', func2str(heading_conversion_func));
0669       end
0670       break
0671     end
0672   end
0673   
0674   
0675   %% Select waypoint coordinate sensors.
0676   % Find preferred valid waypoint longitude and latitude sensor available in
0677   % list sensor fields, if any.
0678   waypoint_choice_list = options.waypoint_list;
0679   for waypoint_choice_idx = 1:numel(waypoint_choice_list)
0680     waypoint_choice = waypoint_choice_list(waypoint_choice_idx);
0681     wpt_lon_field = waypoint_choice.longitude;
0682     wpt_lat_field = waypoint_choice.latitude;
0683     wpt_conversion_func = [];
0684     if isfield(waypoint_choice_list, 'conversion')
0685       wpt_conversion_func = waypoint_choice.conversion;
0686     end
0687     if all(ismember({wpt_lat_field wpt_lon_field}, field_list)) ...
0688         && ~all(isnan(data_raw.(wpt_lon_field))) ...
0689         && ~all(isnan(data_raw.(wpt_lat_field)))
0690       data_pre.waypoint_longitude = data_raw.(wpt_lon_field);
0691       data_pre.waypoint_latitude = data_raw.(wpt_lat_field);
0692       meta_pre.waypoint_longitude.sources = wpt_lon_field;
0693       meta_pre.waypoint_latitude.sources = wpt_lat_field;
0694       fprintf('Selected waypoint position sensors %d:\n', waypoint_choice_idx);
0695       fprintf('  waypoint longitude: %s\n', wpt_lon_field);
0696       fprintf('  waypoint latitude : %s\n', wpt_lat_field);
0697       if ~isempty(wpt_conversion_func)
0698         if ischar(wpt_conversion_func)
0699           wpt_conversion_func = str2func(wpt_conversion_func);
0700         end
0701         [data_pre.waypoint_longitude, data_pre.waypoint_latitude] = ...
0702           wpt_conversion_func(data_pre.waypoint_longitude, ...
0703                               data_pre.waypoint_latitude);
0704         meta_pre.waypoint_longitude.sources = {wpt_lon_field wpt_lat_field}';
0705         meta_pre.waypoint_longitude.conversion = func2str(wpt_conversion_func);
0706         meta_pre.waypoint_latitude.sources = {wpt_lon_field wpt_lat_field}';
0707         meta_pre.waypoint_latitude.conversion = func2str(wpt_conversion_func);
0708         fprintf('  conversion : %s\n', func2str(wpt_conversion_func));
0709       end
0710       break
0711     end
0712   end
0713   
0714   
0715   %% Select averaged water velocity sensor.
0716   % Find preferred valid averaged water velocity sensor available in list of
0717   % sensor fields, if any.
0718   water_velocity_choice_list = options.water_velocity_list;
0719   for water_velocity_choice_idx = 1:numel(water_velocity_choice_list)
0720     water_velocity_choice = ...
0721       water_velocity_choice_list(water_velocity_choice_idx);
0722     wat_vel_east_field = water_velocity_choice.velocity_eastward;
0723     wat_vel_north_field = water_velocity_choice.velocity_northward;
0724     wat_vel_conversion_func = [];
0725     if isfield(water_velocity_choice_list, 'conversion')
0726       wat_vel_conversion_func = water_velocity_choice.conversion;
0727     end
0728     if all(ismember({wat_vel_north_field wat_vel_east_field}, field_list)) ...
0729         && ~all(isnan(data_raw.(wat_vel_east_field))) ...
0730         && ~all(isnan(data_raw.(wat_vel_north_field)))
0731       data_pre.water_velocity_eastward = data_raw.(wat_vel_east_field);
0732       data_pre.water_velocity_northward = data_raw.(wat_vel_north_field);
0733       fprintf('Selected water velocity sensors %d:\n', water_velocity_choice_idx);
0734       fprintf('  water velocity eastward : %s\n', wat_vel_east_field);
0735       fprintf('  water velocity northward: %s\n', wat_vel_north_field);
0736       if ~isempty(wat_vel_conversion_func)
0737         if ischar(wat_vel_conversion_func)
0738           wat_vel_conversion_func = str2func(wat_vel_conversion_func);
0739         end
0740         [data_pre.water_velocity_northward, data_pre.water_velocity_eastward] = ...
0741           wat_vel_conversion_func(data_pre.water_velocity_eastward, ...
0742                                   data_pre.water_velocity_northward);
0743         meta_pre.water_velocity_eastward.sources = ...
0744           {wat_vel_north_field wat_vel_east_field}';
0745         meta_pre.water_velocity_eastward.conversion = ...
0746           func2str(wat_vel_conversion_func);
0747         meta_pre.water_velocity_northward.sources = ...
0748           {wat_vel_east_field wat_vel_north_field}';
0749         meta_pre.water_velocity_northward.conversion = ...
0750           func2str(wat_vel_conversion_func);
0751         fprintf('  conversion : %s\n', func2str(wat_vel_conversion_func));
0752       end
0753       break
0754     end
0755   end
0756   
0757   
0758   %% Select CTD sensor.
0759   % Find preferred valid CTD sensor available in list of sensor fields, if any.
0760   ctd_choice_list = options.ctd_list;
0761   for ctd_choice_idx = 1:numel(ctd_choice_list)
0762     ctd_choice = ctd_choice_list(ctd_choice_idx);
0763     cond_field = ctd_choice.conductivity;
0764     temp_field = ctd_choice.temperature;
0765     pres_field = ctd_choice.pressure;
0766     time_ctd_field = [];
0767     pressure_conversion_func = [];
0768     condtemp_calibration_func = [];
0769     if isfield(ctd_choice_list, 'time')
0770       time_ctd_field = ctd_choice.time;
0771     end
0772     if isfield(ctd_choice_list, 'pressure_conversion')
0773       pressure_conversion_func = ctd_choice.pressure_conversion;
0774     end
0775     if isfield(ctd_choice_list, 'calibration')
0776       condtemp_calibration_func = ctd_choice.calibration;
0777     end
0778     if all(ismember({cond_field temp_field pres_field}, field_list)) ...
0779         && any(data_raw.(cond_field) > 0) ...
0780         && any(data_raw.(temp_field) > 0) ...
0781         && any(data_raw.(pres_field) > 0)
0782       data_pre.conductivity = data_raw.(cond_field);
0783       data_pre.temperature = data_raw.(temp_field);
0784       data_pre.pressure = data_raw.(pres_field);
0785       meta_pre.conductivity.sources = cond_field;
0786       meta_pre.temperature.sources = temp_field;
0787       meta_pre.pressure.sources = pres_field;
0788       fprintf('Selected CTD sensor %d:\n', ctd_choice_idx);
0789       fprintf('  conductivity: %s\n', cond_field);
0790       fprintf('  temperature : %s\n', temp_field);
0791       fprintf('  pressure    : %s\n', pres_field);
0792       if ~isempty(time_ctd_field) ...
0793           && ismember(time_ctd_field, field_list) ...
0794           && any(data_raw.(time_ctd_field) > 0)
0795         data_pre.time_ctd = data_raw.(time_ctd_field);
0796         meta_pre.time_ctd.sources = time_ctd_field;
0797         fprintf('  time CTD    : %s\n', time_ctd_field);
0798       end
0799       if ~isempty(pressure_conversion_func)
0800         if ischar(pressure_conversion_func)
0801           pressure_conversion_func = str2func(pressure_conversion_func);
0802         end
0803         data_pre.pressure = pressure_conversion_func(data_pre.pressure);
0804         meta_pre.pressure.conversion = func2str(pressure_conversion_func);
0805         fprintf('  pressure conversion : %s\n', func2str(pressure_conversion_func));
0806       end
0807       if ~isempty(condtemp_calibration_func)
0808         if ischar(condtemp_calibration_func)
0809           condtemp_calibration_func = str2func(condtemp_calibration_func);
0810         end
0811         temp_calib_params = calibration_parameters.(temp_field);
0812         cond_calib_params = calibration_parameters.(cond_field);
0813         [data_pre.temperature, data_pre.conductivity] = ...
0814           condtemp_calibration_func( ...
0815             data_pre.temperature, data_pre.conductivity, data_pre.pressure, ...
0816             temp_calib_params, cond_calib_params);
0817         if isstruct(temp_calib_params)
0818           temp_calib_param_names = fieldnames(temp_calib_params);
0819           temp_calib_param_values = cell2mat(struct2cell(temp_calib_params));
0820         else
0821           temp_calib_param_names = {};
0822           temp_calib_param_values = temp_calib_params;
0823         end
0824         if isstruct(cond_calib_params)
0825           cond_calib_param_names = fieldnames(cond_calib_params);
0826           cond_calib_param_values = cell2mat(struct2cell(cond_calib_params));
0827         else
0828           cond_calib_param_names = {};
0829           cond_calib_param_values = cond_calib_params;
0830         end
0831         meta_pre.temperature.sources = {cond_field temp_field pres_field}';
0832         meta_pre.temperature.calibration = func2str(condtemp_calibration_func);
0833         meta_pre.temperature.calibration_parameters = temp_calib_param_values;
0834         meta_pre.conductivity.sources = {cond_field temp_field pres_field}';
0835         meta_pre.conductivity.calibration = func2str(condtemp_calibration_func);
0836         meta_pre.conductivity.calibration_parameters = cond_calib_param_values;
0837         if isempty(temp_calib_param_names)
0838           temp_calib_param_names = ...
0839             arrayfun(@num2str, 1:numel(temp_calib_param_values), ...
0840                      'UniformOutput', false);
0841         else
0842           meta_pre.temperature.calibration_parameter_names = ...
0843             temp_calib_param_names;
0844         end
0845         if isempty(cond_calib_param_names)
0846           temp_calib_param_names = ...
0847             arrayfun(@num2str, 1:numel(cond_calib_param_values), ...
0848                      'UniformOutput', false);
0849         else
0850           meta_pre.conductivity.calibration_parameter_names = ...
0851             cond_calib_param_names;
0852         end
0853         fprintf('  calibration : %s\n', func2str(condtemp_calibration_func));
0854         for temp_calib_param_idx = 1:numel(temp_calib_param_names)
0855           fprintf('  temperature  calibration parameter %-8s: %f\n', ...
0856                   temp_calib_param_names{temp_calib_param_idx}, ...
0857                   temp_calib_param_values(temp_calib_param_idx));
0858         end
0859         for cond_calib_param_idx = 1:numel(cond_calib_param_names)
0860           fprintf('  conductivity calibration parameter %-8s: %f\n', ...
0861                   cond_calib_param_names{cond_calib_param_idx}, ...
0862                   cond_calib_param_values(cond_calib_param_idx));
0863         end
0864       end
0865       break
0866     end
0867   end
0868   
0869   
0870   %% Select oxygen sensors.
0871   % Find preferred valid oxygen sensor availbale in list of sensor fields,
0872   % if any.
0873   oxygen_choice_list = options.oxygen_list;
0874   oxygen_variables = ...
0875     {'oxygen_concentration' 'oxygen_saturation' 'oxygen_frequency'};
0876   for oxygen_choice_idx = 1:numel(oxygen_choice_list)
0877     oxygen_choice = oxygen_choice_list(oxygen_choice_idx);
0878     oxygen_variables_select = ...
0879       cellfun(@(v)(isfield(oxygen_choice_list, v) && ~isempty(oxygen_choice.(v))), ...
0880               oxygen_variables);
0881     oxygen_var_list = oxygen_variables(oxygen_variables_select);
0882     oxygen_field_list = cellfun(@(v)(oxygen_choice.(v)), ...
0883                                 oxygen_var_list, 'UniformOutput', false);
0884     time_oxygen_field = [];
0885     temperature_oxygen_field = [];
0886     if isfield(oxygen_choice_list, 'time')
0887       time_oxygen_field = oxygen_choice.time;
0888     end
0889     if isfield(oxygen_choice_list, 'temperature')
0890       temperature_oxygen_field = oxygen_choice.temperature;
0891     end
0892     oxygen_available = ...
0893       all(ismember(oxygen_field_list, field_list)) && ...
0894       all(cellfun(@(f)(any(data_raw.(f)) > 0), oxygen_field_list));
0895     if oxygen_available
0896       fprintf('Selected oxygen sensor %d:\n', oxygen_choice_idx);
0897       for oxygen_var_idx = 1:numel(oxygen_var_list);
0898         oxygen_var = oxygen_var_list{oxygen_var_idx};
0899         oxygen_field = oxygen_field_list{oxygen_var_idx};
0900         data_pre.(oxygen_var) = data_raw.(oxygen_field);
0901         meta_pre.(oxygen_var).sources = oxygen_field;
0902         fprintf('  %-20s: %s\n', oxygen_var, oxygen_field);
0903       end
0904       if ~isempty(time_oxygen_field) ...
0905           && ismember(time_oxygen_field, field_list) ...
0906           && any(data_raw.(time_oxygen_field) > 0)
0907         data_pre.time_oxygen = data_raw.(time_oxygen_field);
0908         meta_pre.time_oxygen.sources = time_oxygen_field;
0909         fprintf('  time oxygen         : %s\n', time_oxygen_field);
0910       end
0911       if ~isempty(temperature_oxygen_field) ...
0912           && ismember(temperature_oxygen_field, field_list) ...
0913           && any(data_raw.(temperature_oxygen_field) > 0)
0914         data_pre.temperature_oxygen = data_raw.(temperature_oxygen_field);
0915         meta_pre.temperature_oxygen.sources = temperature_oxygen_field;
0916         fprintf('  temperature oxygen  : %s\n', temperature_oxygen_field);
0917       end
0918       break
0919     end
0920   end
0921   
0922   
0923   %% Select fluorescence and scatter sensor (chlorophyll, turbidity, cdom...).
0924   % Find preferred valid fluorescence and turbidity sensor available in list of
0925   % sensor fields, if any.
0926   optics_choice_list = options.optics_list;
0927   optics_variables = ...
0928     {'chlorophyll' 'turbidity' 'cdom' 'scatter_650' 'backscatter_700'};
0929   for optics_choice_idx = 1:numel(optics_choice_list)
0930     optics_choice = optics_choice_list(optics_choice_idx);
0931     optics_variables_select = ...
0932       cellfun(@(v)(isfield(optics_choice_list, v) && ~isempty(optics_choice.(v))), ...
0933               optics_variables);
0934     optics_var_list = optics_variables(optics_variables_select);
0935     optics_field_list = cellfun(@(v)(optics_choice.(v)), ...
0936                                 optics_var_list, 'UniformOutput', false);
0937     time_optics_field = [];
0938     temperature_optics_field = [];
0939     optics_calibration_func = [];
0940     if isfield(optics_choice_list, 'time')
0941       time_optics_field = optics_choice.time;
0942     end
0943     if isfield(optics_choice_list, 'temperature')
0944       temperature_optics_field = optics_choice.temperature;
0945     end
0946     if isfield(optics_choice_list, 'calibration')
0947       optics_calibration_func = optics_choice.calibration;
0948     end
0949     optics_available = ...
0950       all(ismember(optics_field_list, field_list)) && ...
0951       all(cellfun(@(f)(any(data_raw.(f)) > 0), optics_field_list));
0952     if optics_available
0953       fprintf('Selected fluorescence and scatter sensor %d:\n', optics_choice_idx);
0954       for optics_var_idx = 1:numel(optics_var_list);
0955         optics_var = optics_var_list{optics_var_idx};
0956         optics_field = optics_field_list{optics_var_idx};
0957         data_pre.(optics_var) = data_raw.(optics_field);
0958         meta_pre.(optics_var).sources = optics_field;
0959         fprintf('  %-12s: %s\n', optics_var, optics_field);
0960       end
0961       if ~isempty(time_optics_field) ...
0962           && ismember(time_optics_field, field_list) ...
0963           && any(data_raw.(time_optics_field) > 0)
0964         data_pre.time_optics = data_raw.(time_optics_field);
0965         meta_pre.time_optics.sources = time_optics_field;
0966         fprintf('  time optics         : %s\n', time_optics_field);
0967       end
0968       if ~isempty(temperature_optics_field) ...
0969           && ismember(temperature_optics_field, field_list) ...
0970           && any(data_raw.(temperature_optics_field) > 0)
0971         data_pre.temperature_optics = data_raw.(temperature_optics_field);
0972         meta_pre.temperature_optics.sources = temperature_optics_field;
0973         fprintf('  temperature optics  : %s\n', temperature_optics_field);
0974       end
0975       if ~isempty(optics_calibration_func)
0976         if ischar(optics_calibration_func)
0977           optics_calibration_func = str2func(optics_calibration_func);
0978         end
0979         optics_calib_data = ...
0980           cellfun(@(v)(data_pre.(v)), optics_var_list, 'UniformOutput', false);
0981         optics_calib_params = ...
0982           cellfun(@(f)(calibration_parameters.(f)), optics_field_list, ...
0983                   'UniformOutput', false);
0984         [optics_calib_data{:}] = ...
0985           optics_calibration_func(optics_calib_data{:}, optics_calib_params{:});
0986         fprintf('  calibration : %s\n', func2str(optics_calibration_func));
0987         for optics_var_idx = 1:numel(optics_var_list);
0988           optics_var = optics_var_list{optics_var_idx};
0989           optics_var_calib_data = optics_calib_data{optics_var_idx};
0990           optics_var_calib_params = optics_calib_params{optics_var_idx};
0991           data_pre.(optics_var) = optics_var_calib_data;
0992           if isstruct(optics_var_calib_params)
0993             optics_var_calib_param_names = fieldnames(optics_var_calib_params);
0994             optics_var_calib_param_values = ...
0995               cell2mat(struct2cell(optics_var_calib_params));
0996           else
0997             optics_var_calib_param_names = {};
0998             optics_var_calib_param_values = optics_var_calib_params;
0999           end
1000           meta_pre.(optics_var).calibration = func2str(optics_calibration_func);
1001           meta_pre.(optics_var).calibration_parameters = ...
1002             optics_var_calib_param_values;
1003           if isempty(optics_var_calib_param_names)
1004             optics_var_calib_param_names = ...
1005               arrayfun(@num2str, 1:numel(optics_var_calib_param_values), ...
1006                        'UniformOutput', false);
1007           else
1008             meta_pre.(optics_var).calibration_parameter_names = ...
1009               optics_var_calib_param_names;
1010           end
1011           for optics_var_calib_param_idx = 1:numel(optics_var_calib_param_names)
1012             fprintf('  %-12s calibration parameter %-8s: %f\n', optics_var, ...
1013                     optics_var_calib_param_names{optics_var_calib_param_idx}, ...
1014                     optics_var_calib_param_values(optics_var_calib_param_idx));
1015           end
1016         end
1017       end
1018       break
1019     end
1020   end
1021   
1022   
1023   %% Select any other extra sensor.
1024   % Add the preferred valid extra sensor available in list of sensor fields,
1025   % for each extra sensor option given.
1026   extra_sensor_name_list = fieldnames(options.extra_sensor_list);
1027   for extra_sensor_name_idx = 1:numel(extra_sensor_name_list)
1028     extra_sensor_name = extra_sensor_name_list{extra_sensor_name_idx};
1029     extra_sensor_choice_list = options.extra_sensor_list.(extra_sensor_name);
1030     extra_sensor_variables = setdiff(fieldnames(extra_sensor_choice_list), ...
1031                                      {'calibration' 'conversion'});
1032     for extra_sensor_choice_idx = 1:numel(extra_sensor_choice_list)
1033       extra_sensor_choice = extra_sensor_choice_list(extra_sensor_choice_idx);
1034       extra_sensor_fields = ...
1035         cellfun(@(v)(extra_sensor_choice.(v)), extra_sensor_variables, ...
1036                 'UniformOutput', false);
1037       extra_sensor_variable_select = ~cellfun(@isempty, extra_sensor_fields);
1038       extra_sensor_var_list = ...
1039         extra_sensor_variables(extra_sensor_variable_select);
1040       extra_sensor_field_list = ...
1041         extra_sensor_fields(extra_sensor_variable_select);
1042       extra_sensor_conversion_func = [];
1043       extra_sensor_calibration_func = [];
1044       if isfield(extra_sensor_choice_list, 'conversion')
1045         extra_sensor_conversion_func = extra_sensor_choice.conversion;
1046       end
1047       if isfield(extra_sensor_choice_list, 'calibration')
1048         extra_sensor_calibration_func = extra_sensor_choice.calibration;
1049       end
1050       extra_sensor_available = ...
1051         all(ismember(extra_sensor_field_list, field_list)) && ...
1052         all(cellfun(@(f)(~all(isnan(data_raw.(f)))), extra_sensor_field_list));
1053       if extra_sensor_available
1054         fprintf('Selected %s sensor %d:\n', ...
1055                 extra_sensor_name, extra_sensor_choice_idx);
1056         extra_sensor_data = cell(size(extra_sensor_var_list));
1057         for extra_sensor_var_idx = 1:numel(extra_sensor_var_list)
1058           extra_sensor_var = ... 
1059             extra_sensor_var_list{extra_sensor_var_idx};
1060           extra_sensor_field = ...
1061             extra_sensor_field_list{extra_sensor_var_idx};
1062           extra_sensor_data{extra_sensor_var_idx} = ...
1063             data_raw.(extra_sensor_field);
1064           meta_pre.(extra_sensor_var).sources = extra_sensor_field;
1065           fprintf('  %-12s: %s\n', extra_sensor_var, extra_sensor_field);
1066         end
1067         if ~isempty(extra_sensor_conversion_func)
1068           if ischar(extra_sensor_conversion_func)
1069             extra_sensor_conversion_func = ...
1070               str2func(extra_sensor_conversion_func);
1071           end
1072           [extra_sensor_data{:}] = ...
1073             extra_sensor_conversion_func(extra_sensor_data{:});
1074           fprintf('  conversion : %s\n', func2str(extra_sensor_conversion_func));
1075           for extra_sensor_var_idx = 1:numel(extra_sensor_var_list);
1076             extra_sensor_var = ...
1077               extra_sensor_var_list{extra_sensor_var_idx};
1078             meta_pre.(extra_sensor_var).sources = extra_sensor_field_list;
1079             meta_pre.(extra_sensor_var).conversion = ...
1080               func2str(extra_sensor_conversion_func);
1081           end
1082         end
1083         if ~isempty(extra_sensor_calibration_func)
1084           if ischar(extra_sensor_calibration_func)
1085             extra_sensor_calibration_func = ...
1086               str2func(extra_sensor_calibration_func);
1087           end
1088           extra_sensor_calib_params = ...
1089             cellfun(@(f)(calibration_parameters.(f)), extra_sensor_field_list, ...
1090                     'UniformOutput', false);
1091           [extra_sensor_data{:}] = ...
1092             extra_sensor_calibration_func(extra_sensor_data{:}, ...
1093                                           extra_sensor_calib_params{:});
1094           fprintf('  calibration : %s\n', func2str(extra_sensor_calibration_func));
1095           for extra_sensor_var_idx = 1:numel(extra_sensor_var_list);
1096             extra_sensor_var = ...
1097               extra_sensor_var_list{extra_sensor_var_idx};
1098             extra_sensor_var_calib_params = ...
1099               extra_sensor_calib_params{extra_sensor_var_idx};
1100             if isstruct(extra_sensor_calib_params)
1101               extra_sensor_var_calib_param_names = ...
1102                 fieldnames(extra_sensor_var_calib_params);
1103               extra_sensor_var_calib_param_values = ...
1104                 cell2mat(struct2cell(extra_sensor_var_calib_params));
1105             else
1106               extra_sensor_var_calib_param_names = {};
1107               extra_sensor_var_calib_param_values = ...
1108                 extra_sensor_var_calib_params;
1109             end
1110             meta_pre.(extra_sensor_var).sources = extra_sensor_field_list;
1111             meta_pre.(extra_sensor_var).calibration = ...
1112               func2str(extra_sensor_calibration_func);
1113             meta_pre.(extra_sensor_var).calibration_parameters = ...
1114               extra_sensor_var_calib_param_values;
1115             if isempty(extra_sensor_var_calib_param_names)
1116               extra_sensor_var_calib_param_names = ...
1117                 arrayfun(@num2str, 1:numel(extra_sensor_var_calib_param_values), ...
1118                          'UniformOutput', false);
1119             else
1120               meta_pre.(extra_sensor_var).calibration_parameter_names = ...
1121                 extra_sensor_var_calib_param_names;
1122             end
1123             for extra_sensor_var_calib_param_idx = 1:numel(extra_sensor_var_calib_param_names)
1124               fprintf('  %-12s calibration parameter %-8s: %f\n', ...
1125                       extra_sensor_var, ...
1126                       extra_sensor_var_calib_param_names{extra_sensor_var_calib_param_idx}, ...
1127                       extra_sensor_var_calib_param_values(extra_sensor_var_calib_param_idx));
1128             end
1129           end
1130         end
1131         for extra_sensor_var_idx = 1:numel(extra_sensor_var_list)
1132           extra_sensor_var = ... 
1133             extra_sensor_var_list{extra_sensor_var_idx};
1134           data_pre.(extra_sensor_var) = ...
1135             extra_sensor_data{extra_sensor_var_idx};
1136         end
1137         break
1138       end
1139     end
1140   end
1141 
1142 end