__color__ __group__ ticket summary component milestone type created _description_ _changetime _reporter 3 cbc 3 Add test framework sodar sodar-1.0 enhancement 1181589537 Add a unittest based framework. 1181589935 cbc 3 cbc 10 Correct for missing samples sodar sodar-1.0 task 1187623904 Fill in Data copy with !NaNs for missing samples. 1187645854 cbc 3 cbc 15 Correct for missing altitudes sodar sodar-1.0 task 1187624603 Fix up data copies for missing altitudes so that arrays are orthogonal. 1187624609 cbc 3 cbc 16 Mark the maximum altitude with good data for each sample sodar sodar-1.0 task 1187625199 ...and find the minimum of those maximums across all samples. 1187625214 cbc 3 cbc 17 Mark the minimum altitude with good data for each sample sodar sodar-1.0 task 1187625334 ...and find maximum of those minimums across all samples. 1187625351 cbc 3 cbc 19 Construct colorspec for c matrix sodar sodar-1.0 task 1187625574 Convert speed to matplotlib colorspec. 1187625706 cbc 2 haines 30 error parseing dates raw2proc raw2proc-1.0 defect 1291930181 We started having a problem with filt_datetime() and the regexp that YYYYMMDD or YYMMDD for dates of Nov and Dec 2010. The date string '2010-12' was parsed as (2020, 10, 12) not (2010, 12, 01). We should try using the parser provided by the python module dateutil-1.2 Try {{{>>from dateutil.parser import parse}}} {{{>>parse('2010_12', fuzzy=True)}}} {{{datetime.datetime(2010, 12, 1, 0, 0) }}} 1291931620 haines 3 haines 24 verify salinity values from AVPs YSI 6600 V2 raw2proc defect 1222287439 Data from the YSI 6600 V2 unit on the automated vertical profilers (AVPs) report lower salinity values than calculated salinity from cond, depth and temp using UNESCO seawater function. We need to verify which is correct value. Profile Time: 23:30:00 Profile Date: 09/22/2008 Profile Depth: 362.0 cm Profile Location: Morgan Bay Serial No: 00016B55, ID: Ford_SERDP 09/22/08 23:30:07 22.81 33.57 21.06 0.155 7.91 1.9 10.5 7.84 month, day, year, hour, min, sec, temp (deg. C), conductivity (mS/cm), salinity (ppt), depth (meters), pH, turbidity (NTU), chlorophyll (micrograms per liter), DO (micrograms per liter) mS/cm == mmho/cm matlab salin= sw_salt(cond/sw_c3515, wtemp, press) >> salin= sw_salt(33.57/sw_c3515, 22.81,0.155) salin = 22.0735 Compared to 21.06 reported from ysi. See [/dataproc/wiki/raw2proc/docs raw2proc page] for info on this component. 1222289987 haines 3 haines 28 Apply additional algorithm to help with ADCP surface masking raw2proc defect 1224272751 Using either error or echo determine best algorithm using other resources from the ADCP to insert NAN values in z-locations that contain or are above the surface. This is a follow-on to other tickets ([ticket:26 Ticket 26] and [ticket:27 Ticket 27]). 1224272829 haines 3 haines 31 change dim names to match var name in netCDF (ntime to time) raw2proc raw2proc-1.0 defect 1340724268 In each proc_*.py creator module, change dimension names to match corresponding variable name. It might not make reading the file by humans easy, but it will allow pydap to work correctly to grab data dependency. For example, pydap cannot load d2.wtemp[:] because ntime dimension does not match time var. pydap is looking for the dependency based on dimension values. {{{ >>> d2= pydap.open_url('http://whewell.marine.unc.edu/dods/nccoos/level1/b2/ctd1/b2_ctd1_2011_11.nc') >>> d2.wtemp {'wtemp': , 'ntime': } }}} {{{ >>> d2.wtemp[:] Traceback (most recent call last): File "", line 1, in File "/opt/env/haines/dataproc/lib/python2.6/site-packages/pydap/model.py", line 662, in __getitem__ var.data = var.data[slice_] File "/opt/env/haines/dataproc/lib/python2.6/site-packages/pydap/proxy.py", line 115, in __getitem__ resp, data = request(url) File "/opt/env/haines/dataproc/lib/python2.6/site-packages/pydap/util/http.py", line 50, in request raise ServerError(msg) pydap.exceptions.ServerError: 'Server error 42: "\'ntime\'"' }}} However, we can still read the data via pydap, but have to know to get time separately, e.g. {{{ >>> d2.wtemp.array[:] array([ 23.10750008, 23.14500046, 23.38870049, ..., 22.0807991 , 22.06220055, 22.07489967], dtype=float32) >>> d2.time.array[:] array([1320121566, 1320132726, 1320137406, ..., 1322696548, 1322696908, 1322697268]) }}} So change the following in creator(): {{{ # dimension names use tuple so order of initialization is maintained dim_inits = ( ('ntime', NC.UNLIMITED), ('nlat', 1), ('nlon', 1), ('nz', 1), ) }}} to {{{ # dimension names use tuple so order of initialization is maintained dim_inits = ( ('time', NC.UNLIMITED), ('lat', 1), ('lon', 1), ('z', 1), ) }}} and any ntime to time (or nlat to lat or nlon to lon or nz to z, etc) {{{ # (varName, varType, (dimName1, [dimName2], var_inits = ( # coordinate variables ('time', NC.INT, ('time',)), ('lat', NC.FLOAT, ('lat',)), ('lon', NC.FLOAT, ('lon',)), ('z', NC.FLOAT, ('z',)), # data variables ('wtemp', NC.FLOAT, ('time',)), ('cond', NC.FLOAT, ('time',)), ('press', NC.FLOAT, ('time',)), # derived variables ('depth', NC.FLOAT, ('time',)), ('salin', NC.FLOAT, ('time',)), ('density', NC.FLOAT, ('time',)), ) }}} 1340724582 haines 3 haines 25 apply magnetic variation to wind data from AVPs raw2proc task 1222287830 Verify that u,v component and wind direction are NOT adjusted for magnetic variation at the AVP sites (Hampton, Morgan, and Stones). Then apply the correction from mvar value in proc_avp_ysi_6600_v2.py provided in each config-file for the platform location. This info is from Tony Whipple. Wind: Date, time, speed (m/s), direction (magnetic), compass direction (magnetic), North (m/s), East (m/s), n-samples See [/dataproc/wiki/raw2proc/docs raw2proc page] for info on this component. 1223485579 haines 3 haines 29 clean up procutil module raw2proc task 1291755512 There is no need for functions like meters2feet(), etc now that udunits is implemented. Clean-up code and make appropriate changes to calls to old conversions throughout plotting and processing. 1291755545 haines