In Situ TAC is the IN SITU OBSERVATIONAL COMPONENT of The Copernicus Marine Environment Monitoring Service.
In Situ TAC integrates thousand of local data from on-site sensors on board of a wide range of platforms
Distribution Units are INSTAC responsibles for providing data on a defined geographical area.
For each region, the In Situ Distribution Unit distributes all its data and metadata products on a FTP server (regional portal):
Data products available on those FTPs keep exactly the same:
in order to facilitate selection and processing to final users.
For each region, the In Situ Distribution Unit distributes all its data and metadata products in netCDFs, format used by up to 1300 institutions for creating, accessing and sharing array-oriented scientific data.
The data files format is an implementation of NetCDF OceanSITES format.
CMEMS list a number of tools users can use for visualization & manipulation of such files
Our contribution for spreading the word about how-to-handle netCDF format can be found in the material build for training sessions in form of :
NRT quality controls are available here
Quality control flag scale:Code | Meaning | Comment |
---|---|---|
0 | No QC performed | - |
1 | Good data | All real-time QC tests passed |
2 | Probably good data | - |
3 | Bad data that are potentially correctable | These data are not to be used without scientific correction |
Code | Meaning | Comment |
---|---|---|
4 | Bad data | Data have failed one or more of the tests |
5 | Value changed | Data may be recovered after transmission error |
6 | Not used | - |
7 | Nominal value | Data were not observed but reported.Example: an instrument target depth |
8 | Interpolated value | Missing data may be interpolated from neighbouring data in space or time |
9 | Missing value | The value is missing, is not reported, is not applicable |
Each DU distributes all its data and metadata products in NetCDF files on an FTP server sticking with the same structure:
History | Monthly | Latest |
---|---|---|
RR_XX_YY_CODE<_ZZZ>.nc | RR_YYYYMM_XX_YY_CODE.nc | RR_LATEST_XX_YY_CODE_YYYYMMDD.nc |
GL_TS_MO_15001_2010daily.nc | IR_200911_TS_DB_44607.nc | IR_LATEST_TS_DB_44607_20091117.nc |
Distribution Units provides 2 types of products :
Collections of data files (netCDFs) continuously updated on the Distribution Units (DUs) - Automate NRTQC
Delayed mode data that has been subject of a throughout Scientific assesment - visual QC.
Build over historical data and delivered by versions, typically once a year.
NORTH WEST SHELVES | |||||
ARTIC | |||||
BALTIC | |||||
IBERIA-BISCAY-IRELAND | |||||
BLACK SEA | |||||
MED | |||||
(Interactive)
All CMEMS products are available via its Catalogue at: http://marine.copernicus.eu/services-portfolio/access-to-products/
Catalogue
(Interactive)
All INSTAC products are listed at: http://www.marineinsitu.eu/products/ with shorcuts to the CMEMS Catalogue
Latest NRT data - INSTAC Dashboard at
http://www.marineinsitu.eu/dashboard/
With useful KPIs
http://www.marineinsitu.eu/monitoring/
(Programmatically)
CMEMS DESK SERVICE:
Pratical Session 'Python and WebServices'
virtual machine ( ✔ techical requirements)
(Programmatically)
Recap!: Do you remember In Situ DUs? In Situ DUs were the responsible of distributing In Situ data products on ftps keeping exactly the same:
in order to facilitate selection and processing to final users.
Recap!: Do you remember In Situ DUs FTP structure?
Examples... LATEST from MED DU
#Title : in-situ observations catalog
#Description : catalog of available in-situ observations per platform
#Project : Copernicus
#Format Version : 1.0
#Last Update : 20170925110012
#catalog_id,file_name,geospatial_lat_min,geospatial_lat_max,geospatial_lon_min,geospatial_lon_max,time_coverage_start,time_coverage_end,provider,date_update,data_mode,parameters
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_3901853_20170826.nc,34.82809,34.82809,23.86541,23.86541,2017-08-26T11:55:00Z,2017-08-26T11:55:00Z,Institute of Marine Sciences Middle East Technical University (IMS-METU),2017-08-28T18:05:57Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_3901907_20170826.nc,40.68048,40.68048,5.48433,5.48433,2017-08-26T02:48:20Z,2017-08-26T02:48:20Z,Euro-Argo,2017-08-28T18:06:02Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_3901978_20170826.nc,41.16919,41.16919,17.58031,17.58031,2017-08-26T07:58:30Z,2017-08-26T07:58:30Z,Euro-Argo,2017-08-28T19:06:48Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6900636_20170826.nc,38.37700,38.37700,5.40800,5.40800,2017-08-26T00:21:06Z,2017-08-26T00:21:06Z,ICM Institut de Ciencies del Mar - Barcelona,2017-08-27T04:07:20Z,R, PRES TEMP PRES_ADJUSTED PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901249_20170826.nc,39.72242,39.72242,1.69070,1.69070,2017-08-26T06:05:00Z,2017-08-26T06:05:00Z,SOCIB (Spain),2017-08-28T19:16:56Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901250_20170826.nc,39.20740,39.20740,4.77373,4.77373,2017-08-26T06:12:00Z,2017-08-26T06:12:00Z,SOCIB (Spain),2017-08-28T19:16:58Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901773_20170826.nc,33.11633,33.11633,31.46497,31.46497,2017-08-26T09:37:00Z,2017-08-26T09:37:00Z,,2017-08-26T13:05:36Z,R, PRES PSAL TEMP DOX2 DOWNWELLING_PAR DOWN_IRRADIANCE490 DOWN_IRRADIANCE380 DOWN_IRRADIANCE412 CDOM BBP700 CPHL CPHL_ADJUSTED NTAW
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901823_20170826.nc,38.85421,38.85421,10.69752,10.69752,2017-08-26T00:01:00Z,2017-08-26T00:01:00Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-08-27T07:07:27Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901824_20170826.nc,32.67120,32.67120,24.85794,24.85794,2017-08-26T23:58:00Z,2017-08-26T23:58:00Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-08-28T19:17:03Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901839_20170826.nc,40.11784,40.11784,3.33751,3.33751,2017-08-26T00:01:00Z,2017-08-26T00:01:00Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-08-27T07:07:31Z,R, PRES PRES_ADJUSTED TEMP TEMP_ADJUSTED PSAL PSAL_ADJUSTED
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901846_20170826.nc,34.44917,34.44917,18.52256,18.52256,2017-08-26T23:57:00Z,2017-08-26T23:57:00Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-08-28T19:17:08Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6901862_20170826.nc,39.65252,39.65252,17.89844,17.89844,2017-08-26T10:33:00Z,2017-08-26T10:33:00Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-08-26T15:13:31Z,R, PRES PSAL TEMP DOWNWELLING_PAR DOWN_IRRADIANCE490 DOWN_IRRADIANCE380 DOWN_IRRADIANCE412 CDOM BBP700 CPHL CPHL_ADJUSTED
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6902688_20170826.nc,39.73100,39.73100,7.05200,7.05200,2017-08-26T06:05:00Z,2017-08-26T06:05:00Z,,2017-08-27T19:16:37Z,R, PRES TEMP PSAL DOX2
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6902764_20170826.nc,38.01364,38.01364,1.39542,1.39542,2017-08-26T04:58:20Z,2017-08-26T04:58:20Z,SHOM Service Hydrographique et Oceanographique de la Marine,2017-08-28T19:17:13Z,R, PRES PSAL TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6902770_20170826.nc,32.78047,32.78047,28.96149,28.96149,2017-08-26T05:02:20Z,2017-08-26T05:02:20Z,SHOM Service Hydrographique et Oceanographique de la Marine,2017-08-28T19:17:20Z,R, PRES PSAL TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6903203_20170826.nc,35.73067,35.73067,21.86430,21.86430,2017-08-26T05:59:00Z,2017-08-26T05:59:00Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-08-28T19:17:25Z,R, PRES TEMP PSAL DOX2
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6903224_20170826.nc,38.99700,38.99700,20.04400,20.04400,2017-08-26T06:58:00Z,2017-08-26T06:58:00Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-08-27T17:06:44Z,R, PRES TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_PR_PF_6903225_20170826.nc,39.65874,39.65874,15.67174,15.67174,2017-08-26T02:40:16Z,2017-08-26T02:40:16Z,OGS Istituto Nazionale di Oceanografia e di Geofisica Sperimentale,2017-09-02T05:06:27Z,R, PRES TEMP PSAL DOX2
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/latest/20170826/GL_LATEST_TS_DB_6100757_20170826.nc,34.92120,35.05840,19.53440,19.70740,2017-08-26T00:00:00Z,2017-08-26T23:00:00Z,Unknown institution,2017-08-27T04:36:06Z,R, DEPH ATMS TEMP PSAL
Examples... MONTHLY from MED DU
#Title : in-situ observations catalog
#Description : catalog of available in-situ observations per platform
#Project : Copernicus
#Format Version : 1.0
#Last Update : 20170925110014
#catalog_id,file_name,geospatial_lat_min,geospatial_lat_max,geospatial_lon_min,geospatial_lon_max,time_coverage_start,time_coverage_end,provider,date_update,data_mode,parameters
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/GL_201201_TS_DB_61787.nc,37.9933,38.2,6.6971,13.4556,2012-01-01T01:00:00Z,2012-01-15T10:00:00Z,IFREMER,2013-04-06T05:06:17Z,R, TEMP EWCT NSCT DEPH
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/GL_201201_TS_DB_61788.nc,38.55600,39.87450,3.27580,4.76350,2012-01-01T00:00:19Z,2012-01-31T23:30:17Z,,2017-07-29T15:17:47Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/GL_201201_TS_DB_61792.nc,39.1625,39.173,-0.24845,0.1677,2012-01-01T00:02:11Z,2012-01-31T23:30:00Z,IFREMER,2013-04-06T05:49:04Z,R, TEMP EWCT NSCT DEPH
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/GL_201201_TS_DB_61793.nc,36.06870,37.32010,-1.41450,0.18100,2012-01-01T00:01:41Z,2012-01-31T23:30:22Z,,2017-07-29T15:17:49Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/GL_201201_TS_DB_61850C.nc,36.882,36.883,10.767,10.767,2012-01-01T17:30:00Z,2012-01-02T19:30:00Z,IFREMER,2013-04-06T04:50:56Z,R, TEMP DEPH
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/MO_201201_TS_DC_108743.nc,36.83715,37.62300,19.65071,21.08360,2012-01-01T00:00:00Z,2012-01-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/MO_201201_TS_DC_300234010168380.nc,40.96201,42.08307,9.75803,11.64286,2012-01-01T00:00:00Z,2012-01-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/MO_201201_TS_DC_300234010652970.nc,39.11646,39.46365,8.05844,8.35325,2012-01-01T00:00:00Z,2012-01-06T03:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201201/MO_201201_TS_DC_3700572.nc,37.72119,38.24943,2.22189,4.17362,2012-01-01T00:00:00Z,2012-01-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201202/GL_201202_TS_DB_61788.nc,37.89860,38.58310,1.16660,3.30370,2012-02-01T00:00:16Z,2012-02-29T23:30:39Z,,2017-07-29T15:17:47Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201202/GL_201202_TS_DB_61793.nc,36.21110,37.13710,-0.68700,7.18730,2012-02-01T00:00:47Z,2012-02-28T10:00:00Z,,2017-07-29T15:17:50Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201202/MO_201202_TS_DC_108743.nc,36.93371,38.78457,18.91089,20.67823,2012-02-01T00:00:00Z,2012-02-29T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201202/MO_201202_TS_DC_300234010168380.nc,39.79373,41.81726,9.98525,11.24973,2012-02-01T00:00:00Z,2012-02-29T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201202/MO_201202_TS_DC_3700572.nc,36.84541,37.79088,2.30323,6.96606,2012-02-01T00:00:00Z,2012-02-29T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201202/MO_201202_TS_DC_dep0002-scb-idrifter003.nc,39.89922,40.06210,2.61684,2.99964,2012-02-28T12:00:00Z,2012-02-29T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:04:59Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61788.nc,37.62600,38.42090,-0.10480,1.17130,2012-03-01T00:00:15Z,2012-03-31T23:30:16Z,,2017-07-29T15:17:47Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61791.nc,41.10780,41.10800,1.23870,1.23900,2012-03-01T00:00:14Z,2012-03-31T23:30:16Z,,2017-07-29T15:17:48Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61792.nc,39.16711,39.16712,-0.24312,-0.24311,2012-03-01T00:30:00Z,2012-03-31T21:32:10Z,,2017-07-29T15:17:48Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61850.nc,42.73670,44.11690,8.94430,9.85620,2012-03-19T13:30:00Z,2012-03-31T23:30:35Z,,2017-07-29T15:17:50Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61851.nc,42.50940,43.34910,6.13350,7.48510,2012-03-21T05:30:00Z,2012-03-31T23:30:31Z,,2017-07-29T15:21:02Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61852.nc,42.73700,43.90940,9.52010,9.86880,2012-03-19T13:30:00Z,2012-03-31T23:30:18Z,,2017-07-29T15:21:03Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61853.nc,42.70380,44.17870,9.26340,9.76460,2012-03-05T14:34:14Z,2012-03-31T23:30:12Z,,2017-07-29T15:21:04Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61854.nc,42.66900,43.89540,9.31450,10.25670,2012-03-05T13:30:00Z,2012-03-31T23:30:14Z,,2017-07-29T15:21:05Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61855.nc,42.69090,43.67260,9.48490,9.89610,2012-03-05T13:30:00Z,2012-03-31T23:30:18Z,,2017-07-29T15:21:05Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61856.nc,42.86750,43.61500,5.11750,7.62400,2012-03-21T11:00:00Z,2012-03-31T23:30:10Z,,2017-07-29T15:21:06Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61857.nc,42.85830,43.60160,6.41980,7.74130,2012-03-21T12:04:14Z,2012-03-31T23:30:29Z,,2017-07-29T15:21:07Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61858.nc,43.59090,43.59090,7.23210,7.23210,2012-03-28T16:04:14Z,2012-03-31T08:34:14Z,,2017-07-29T15:21:07Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61859.nc,43.20030,43.55190,6.72160,7.27450,2012-03-21T05:30:00Z,2012-03-31T23:31:00Z,,2017-07-29T15:21:06Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61860.nc,42.53490,43.48000,4.32500,7.33710,2012-03-21T05:30:00Z,2012-03-31T23:30:17Z,,2017-07-29T15:24:15Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61861.nc,42.89770,43.68760,5.87010,7.57790,2012-03-21T11:00:00Z,2012-03-31T23:30:17Z,,2017-07-29T15:24:16Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61935.nc,42.88260,43.58270,5.00580,7.50520,2012-03-07T10:00:00Z,2012-03-31T23:30:07Z,,2017-07-29T15:24:16Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61937.nc,43.60120,43.68850,7.15770,7.38170,2012-03-21T02:00:22Z,2012-03-31T23:30:19Z,,2017-07-29T15:24:16Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61938.nc,43.11600,43.60270,6.37220,7.54890,2012-03-07T10:00:00Z,2012-03-31T23:30:05Z,,2017-07-29T15:24:19Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61939.nc,42.98460,43.41540,6.95900,7.42460,2012-03-21T05:30:00Z,2012-03-31T23:31:07Z,,2017-07-29T15:24:19Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61940.nc,43.39040,43.42460,7.68700,7.78410,2012-03-20T13:30:00Z,2012-03-28T17:30:04Z,,2017-07-29T15:24:18Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61941.nc,42.95220,43.56450,5.70760,7.42280,2012-03-07T10:00:00Z,2012-03-31T23:30:26Z,,2017-07-29T15:24:18Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/GL_201203_TS_DB_61942.nc,42.75940,43.63150,4.11760,7.58620,2012-03-07T10:00:00Z,2012-03-31T23:30:25Z,,2017-07-29T15:27:31Z,R, DEPH TEMP NSCT EWCT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/MO_201203_TS_DC_108743.nc,37.73371,37.77347,19.08843,19.12160,2012-03-01T00:00:00Z,2012-03-01T06:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:05:00Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/MO_201203_TS_DC_300234010168380.nc,40.58330,41.95284,9.57632,11.48234,2012-03-01T00:00:00Z,2012-03-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:05:00Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/MO_201203_TS_DC_3700572.nc,37.33028,38.66776,6.28530,8.97993,2012-03-01T00:00:00Z,2012-03-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:05:00Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/MO_201203_TS_DC_6100850.nc,42.75228,44.10556,8.96500,9.85536,2012-03-21T03:00:00Z,2012-03-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:05:00Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/MO_201203_TS_DC_6100851.nc,42.52947,43.33353,6.13354,7.48251,2012-03-21T15:00:00Z,2012-03-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:05:00Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/monthly/drifter/201203/MO_201203_TS_DC_6100852.nc,42.75580,43.90901,9.53587,9.86809,2012-03-21T03:00:00Z,2012-03-31T21:00:00Z,OGS (Istituto Nazionale di Oceanografia e di Geofisica Sperimentale),2017-08-30T15:05:00Z,D, EWCT NSCT DEPH DROGUE_PRESENCE
Examples... HISTORY from MED DU
#Title : in-situ observations catalog
#Description : catalog of available in-situ observations per platform
#Project : Copernicus
#Format Version : 1.0
#Last Update : 20170925110015
#catalog_id,file_name,geospatial_lat_min,geospatial_lat_max,geospatial_lon_min,geospatial_lon_max,time_coverage_start,time_coverage_end,provider,date_update,data_mode,parameters
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_PR_TE_61501.nc,42.12,42.82,5.6,7.4,1999-09-10T08:58:00Z,1999-11-15T06:48:00Z,MEDS,2017-02-02T04:50:47Z,R, PRES TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_PR_TE_61502.nc,41.76,42.01,3.67,3.79,1999-09-10T07:42:00Z,1999-10-04T10:36:00Z,MEDS,2017-02-02T04:50:47Z,R, PRES TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_PR_TE_61504.nc,42.18,43.45,6.15,7.62,1999-09-10T07:42:00Z,1999-11-11T14:37:00Z,MEDS,2017-02-02T04:50:48Z,R, PRES TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_PR_TE_61537.nc,41.15,41.34,3.77,4.13,1999-12-13T16:56:00Z,1999-12-15T05:30:00Z,US NAVAL OCEANOGRAPHIC OFFICE(BAY ST LOUIS MISSISSIPI),2017-02-02T04:50:48Z,R, PRES TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100536.nc,40.67270,42.19970,7.23520,7.67410,2015-07-01T00:00:00Z,2015-07-28T20:00:00Z,Unknown institution,2017-07-13T14:52:30Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100537.nc,38.91750,39.05520,1.11350,1.22300,2015-07-01T00:00:00Z,2015-07-19T06:00:00Z,Unknown institution,2017-07-13T14:52:30Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100538.nc,37.36790,39.92880,5.12680,7.26450,2015-07-01T00:00:00Z,2015-10-13T06:00:00Z,Unknown institution,2017-07-13T14:52:31Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100539.nc,37.03650,37.71640,-1.08950,0.92860,2015-07-01T00:00:00Z,2015-08-18T05:00:00Z,Unknown institution,2017-07-13T14:52:31Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100863.nc,39.97820,42.57650,0.62040,3.96730,2015-08-19T21:00:00Z,2015-10-13T06:00:00Z,Unknown institution,2017-07-13T14:52:35Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100864.nc,36.52040,40.09410,7.36450,11.22990,2015-08-19T21:00:00Z,2015-09-23T09:00:00Z,Unknown institution,2017-07-13T14:52:35Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100865.nc,40.17630,41.11510,2.94810,4.08950,2015-08-20T00:00:00Z,2015-10-13T06:00:00Z,Unknown institution,2017-07-13T14:52:36Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100866.nc,39.96680,40.63280,5.04820,7.15950,2015-08-20T01:00:00Z,2015-10-13T05:00:00Z,Unknown institution,2017-07-13T14:52:41Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_100867.nc,42.41490,43.72860,3.60250,8.72870,2015-08-19T23:00:00Z,2015-10-13T06:00:00Z,Unknown institution,2017-07-13T14:52:41Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_1200518.nc,31.49640,34.60820,27.04060,32.00920,2017-05-24T12:53:00Z,2017-08-22T18:00:00Z,Unknown institution,2017-09-01T15:18:07Z,R, DEPH ATMS TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_12506.nc,25.94000,28.06200,34.79800,35.84900,2004-12-08T19:00:00Z,2005-03-07T19:00:00Z,Naval Oceanographic Office (NAVO),2017-07-13T14:57:24Z,R, DEPH TEMP ATMP ATPT ATMS
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_1300574.nc,36.79732,37.13324,1.24871,3.70004,2016-03-09T14:45:00Z,2016-03-22T09:59:00Z,Unknown institution,2017-07-13T15:07:10Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_1300905.nc,32.27688,39.61784,-2.65949,16.06133,2015-10-13T09:00:00Z,2016-08-16T17:00:00Z,Atlantic Oceanographic and Meteorological Laboratory (AOML) National Oceanic and Atmospheric Administration (NOAA),2017-07-13T15:07:14Z,R, DEPH TEMP
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13527.nc,1.26800,37.78300,-61.64000,8.88800,2004-10-23T00:30:00Z,2015-10-12T23:57:00Z,Unknown institution,2017-07-13T15:11:42Z,R, DEPH TEMP ATMP ATMS ATPT
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13566.nc,8.75600,40.05600,-47.07300,2.99600,2009-08-19T20:01:00Z,2016-01-12T10:44:00Z,Unknown institution,2017-07-13T15:11:46Z,R, DEPH TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13574.nc,10.67900,38.74600,-88.81300,1.78600,2009-11-25T12:59:00Z,2016-03-08T22:51:00Z,Unknown institution,2017-07-13T15:29:44Z,R, DEPH TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13586.nc,11.32800,38.11300,-68.44500,5.99900,2005-06-05T08:54:00Z,2016-01-26T14:27:00Z,Meteo-France / Centre de Meteorologie Marine (CMM),2017-07-13T15:29:55Z,R, DEPH TEMP PSAL
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13640.nc,11.84100,36.83500,-57.42200,-5.39900,2006-07-13T15:59:00Z,2015-10-12T23:57:00Z,Unknown institution,2017-07-13T15:42:59Z,R, DEPH TEMP ATMP ATPT ATMS
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13904.nc,-0.54400,38.10600,-74.16700,3.01300,2006-06-22T14:40:00Z,2014-10-23T20:00:00Z,Atlantic Oceanographic and Meteorological Laboratory (AOML)
National Oceanic and Atmospheric Administration (NOAA),2017-07-13T15:43:02Z,R, DEPH TEMP ATMP ATPT ATMS
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13905.nc,13.76800,44.31600,-76.04300,9.63600,2006-06-28T12:23:00Z,2015-10-12T23:00:00Z,Unknown institution,2017-07-13T15:43:02Z,R, DEPH TEMP ATMP ATPT ATMS
MYO-MOON-01,ftp://medinsitu.hcmr.gr/Core/INSITU_MED_NRT_OBSERVATIONS_013_035/history/drifter/GL_TS_DB_13906.nc,11.80100,41.91700,-74.50200,4.63900,2006-06-28T05:25:00Z,2014-11-10T18:00:00Z,Scripps Institution of Oceanography La Jolla (SIO),2017-07-13T15:43:02Z,R, DEPH TEMP ATMP ATPT ATMS
1. Let's connect to a DU ftp and download the index file!
2. Let's go through the index file and download netCDFs!
3. Let's schedule it
Donwload Anaconda
https://www.anaconda.com/download/
✔ Python
✔ Jupyter notebooks
Look and feel: https://try.jupyter.org/
On the right corner select: New Pyhton 3
Import the libraries we are going to use
import ftplib
import numpy as np
Download the index file of a given DU
user = '' #type CMEMS user name
password = '' #type CMEMS password
product_name = '' #type aimed CMEMS in situ product
distribution_unit = '' #type aimed hosting institution
index_file = '' #type aimed index file
ftp=ftplib.FTP(distribution_unit,user,password)
ftp.cwd("Core")
ftp.cwd(product_name)
aimedFileName = index_file
local_filename = aimedFileName
local_file = open(local_filename, 'wb')
ftp.retrbinary('RETR ' + aimedFileName, local_file.write)
local_file.close()
ftp.quit()
#ready when 221 Goodbye.!
Go through the index file and make your selection!
index = np.genfromtxt(index_file, skip_header=6,
unpack=False,
delimiter=',', dtype=None,
names=['catalog_id', 'file_name',
'geospatial_lat_min', 'geospatial_lat_max',
'geospatial_lon_min', 'geospatial_lon_max','
time_coverage_start', 'time_coverage_end', 'provider', 'date_update', 'data_mode', 'parameters'])
Example: selection by parameter - metadata
#read file lines (iterate over them)
for entry in index:
file_name = entry['file_name'].decode('utf-8')
parameters = entry['parameters'].decode('utf-8')
param_list = parameters.split(' ');
#set up a selection criteria: i.e specific parameter
if 'PSAL' in param_list:
last_idx_slash = file_name.rfind('/')
ncdf_file_name = file_name[last_idx_slash+1:]
folders = file_name.split('/')[3:len(file_name.split('/'))-1]
host = file_name.split('/')[2] #distribution unit
ftp=ftplib.FTP(host,user,password)
for folder in folders:
ftp.cwd(folder)
local_file = open(ncdf_file_name, 'wb')
ftp.retrbinary('RETR '+ncdf_file_name, local_file.write)
local_file.close()
ftp.quit()
Example: selection by data_types - name_convention
#read file lines (iterate over them)
for entry in index:
file_name = entry['file_name'].decode('utf-8')
last_idx_slash = file_name.rfind('/')
ncdf_file_name = file_name[last_idx_slash+1:]
#history: position 2
#monthly and latest: position 3
data_type = ncdf_file_name.split('_')[position]
aimed_data_type = ''#choose a data_type
#set up a selection criteria: i.e specific datatype
if data_type == aimed_data_type :
folders = file_name.split('/')[3:len(file_name.split('/'))-1]
host = file_name.split('/')[2] #distribution unit
ftp=ftplib.FTP(host,user,password)
for folder in folders:
ftp.cwd(folder)
local_file = open(ncdf_file_name, 'wb')
ftp.retrbinary('RETR '+ncdf_file_name, local_file.write)
local_file.close()
ftp.quit()
Schedule the task!
Rely on crontab!
Rely on the equivalent task escheduler??
microsoftMore at:
https://github.com/CopernicusMarineInsitu/INSTACTraining/