Process Level 0 data to Level 1 (raw2proc)

Process Level 0 (raw form) to Level 1 (monthly netCDF), which generally involves parsing ascii or binary data, combining measured parameters to get specific derived parameters of interest, and perform some basic data quality checks. While standing up the framework to add new platforms and packages to the processing stream has been accomplished to arrive at Version 0.1 (raw2proc-v0.1), much remains to be done and holes to fill.

Pyhton Module Dependencies

• numpy
• pycdf
• dateutil

Subcomponents

raw2proc module

Processes raw ascii- or binary-data from different NCCOOS sensors (ctd, adcp, waves-adcp, met) based on manual or automated operation. raw2proc manages what configuration files to use for the specified platform and month to process. Then based on specified package, it determines (via rigid directory structure) what raw files to look for to process. Finally, raw2proc loads parsing and output methods pre-determined in config files for specific package-type to handle processing for specific data formats. For example, adcp data has multiple data formats that we might want to process (log data, spec data, and binary data for both currents and waves). Each format has different information that dictates what can be output.

• ncutil module
• procutil module

STILL TO DO

• organize/brainstorm where to store configuration files (for now they are in the trunk)
• ditto for proc-filetype modules

raw2proc

• add command-line interface thru getopt
• in find_raw() override looking in "yyyy_mm" rawdata subdirectories for multi-month deployments for adcp (e.g. _RDI_000.000, _RDI_001.000, _RDI_003.000 in one subdirectory for one deployment.)

ncutil

• add get_last_sample_time() -- get last time-stamp in netCDF

procutil

test

• Sara needs to learn more about test code and knows she should have been doing it all along
• lots of testing needed