-
Notifications
You must be signed in to change notification settings - Fork 0
/
ChangeLog
523 lines (262 loc) · 15.3 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
2018-02-15 Ajay Saini <[email protected]>
* Changed the command line arguments according to NCO documentation
* Print helpful error messages when input file does not exist
* IMAU raw files (both antartica and greenland) placed on grele at /users/sainiak/data/jaws/imau/raw/
* Worked on converting IMAU data. Still in progress
* Read about invoking automatic testing and conda packaging
2018-02-14 Ajay Saini <[email protected]>
* Added the default netCDF _FillValue for all variables
* Functionalized data writing code
* Common variables moved to jaws.py
* num_lines calculation changed
* Used localize function from pytz to correct timezone calculations
2018-02-13 Ajay Saini <[email protected]>
* Resolved bug related to timezone
* PROMICE- station long names included
* PROMICE- timezone functionality added
* GCNet- Added _FillValue for julian_decimal_time
* PROMICE- lat_lon_GPS calculation changed to reduce run time
2018-02-13 Charlie Zender <[email protected]>
* Add pandas to dependency list
2018-02-12 Ajay Saini <[email protected]>
* station_dict made ordered to be compatible with Python 2
* analysis - indexing corrected
* Added functionality to choose timezone from command line
* Added new argument 'analysis' to calculate day and month in GCNet, only if user wants to do further analysis
* Plot errors corrected, all plots are now running
2018-02-11 Ajay Saini <[email protected]>
* PROMICE- lat_lon_GPS calculation corrected
* PROMICE- ice_velocity_GPS calculation corrected
* Variables intialization code restructured
* GCNet- time recalculated using strptime to reduce runtime
* Time calculation made compatible with Python 2
* Worked on making lat_lon_GPS conversion faster. Still in progress
2018-02-09 Ajay Saini <[email protected]>
* Hour exception adjusted in GCNet
* Implemented xarray to write the data for PROMICE
* GCNet- station number made scalar
* Naked constatns removed in GCNet and AAWS
2018-02-08 Ajay Saini <[email protected]>
* AAWS- time calculation condensed
* Implemented xarray to write the data for GCNet
* GCNet- quality control variables' calculation condensed
* GCNet- Resolved issue of qc_25 variable error
2018-02-07 Ajay Saini <[email protected]>
* Implemented xarray to write the data for AAWS
* Asked questions about derived variables calculation on SO
2018-02-06 Ajay Saini <[email protected]>
* GCNet - Hour calculation made faster
* AAWS- using pandas to convert the data
* Common function defined to get data from the raw files
* GCNet - implemented another approach for calculating quality control variables but not much runtime difference. Will ask on SO tommorow
* PROMICE - using pandas to convert the data
* Read xarray documentation and best practices
2018-02-05 Ajay Saini <[email protected]>
* GCNet - Used pandas to copy data from ascii file, runtime decreased by huge factor. Work in progress, need to implement for calculated variables.
* GCNet- Hour calculation condensed
* Completed 3 required courses of NASA training
2018-02-02 Ajay Saini <[email protected]>
* Changed string matching to float in PROMICE and AAWS
* Removed naked constants in GCNet in PROMICE and AAWS
* time and time_bounds calculation changed in PROMICE and AAWS
* Created and calculated new variables year, month, day, hour in AAWS
* time_calc(for calculating time) and solar(for calculating solar zenith angle) common functions defined in common.py
* Rearranged code
2018-02-01 Ajay Saini <[email protected]>
* Condensed calculation of day and month in GCNet
* time_calc function defined to calculate time
* Changed string matching of missing values to float
* Removed naked constants in GCNet
* Hour calculation uses numeric calculations instead of string
2018-01-31 Ajay Saini <[email protected]>
* Canonical ordering fixed
* Error handling if no input provided
* Global attribute 'time_convention' added
* Station name to be retrieved frm input file in analysis.py
* Analysis plot debugged and working fine now
2018-01-30 Ajay Saini <[email protected]>
* Output name for GCNet files will now be 'gcnet_station-name' if the input name of file is default of GCNet (e.g.: 01c.txt)
* Images moved to grele webserver and completely expunged from repository. Links updated in README.
* GCNet station names edited for all the raw files to have the convention 'networkname_stationname_startdate_enddate'. Changes made on grele and jaws.
* Worked on fixing canonical ordering. In progress
* News section added in README
2018-01-29 Ajay Saini <[email protected]>
* Group meeting and bi-weekly JAWS meeting
* Computed x and y components of GPS-derived ice velocity
* Worked on debugging analysis plots, not working properly. Still in progress
* Looked for ways to provide reference for files from remote server(grele) and make it publicly available. Still in progress.
2018-01-26 Ajay Saini <[email protected]>
* Calculated GPS-derived ice velocity using haversine formula for stations that archive GPS position
* Downloaded and explored datasets from DMI and ASAIQ
* Some datasets didn't convert after new changes, debugged GCNet and AAWS code to make changes
* Updated instructions about optional arguments in analysis example section
2018-01-25 Ajay Saini <[email protected]>
* Used PySolar to calculate solar zenith angle in Python 3, worked on debugging for Python 2, later found that functions used in calculation aren't supported in Python 2
* Explored other modules including SunPy, sun-position and PyEphem. Used sun-position to calculate sza as it worked well for all the stations and both Python versions
* In GCNet and PROMICE, changed time calculation to seconds since 1970 to make it common across all networks
* Generalized common variables' attributes
* Added attributes for derived quality control variables in GCNet
2018-01-24 Ajay Saini <[email protected]>
* Identified all unique post decimal values for julian time in hour calculation and corrected the code accordingly
* Added option to choose year and month for analysis plots
* Added more GCNet stations to the lat, long table
* Generealized code for all different analysis plot in a single file
* Updated Readme with analysis example
2018-01-23 Ajay Saini <[email protected]>
* station_name retrieved using full names instead of single characters for all networks
* Code generalized for station_name provided through argument
* count variable calculation optimised using list comprehension
* Dictionary created to store lat, lon, and long_name for all stations of all networks. Values for variables are retrieved using this table
* CPU pegged during GCNET conversion for one station. Debugged time variable calculation and corrected the error.
2018-01-23 Charlie Zender <[email protected]>
* Add features to L3 value-added science list
2018-01-22 Ajay Saini <[email protected]>
* Corrected calculation of hour in GCNET to accomodate different values after decimal in different files
* Common NC file setup code moved to jaws.py
* count variable calculation optimised using list comprehension
* Dimensions and common variables moved to jaws.py
* jaws 0.2 release procedure:
git tag -a 0.2 -m 'Pale Catshark: Conversion of GCNET, PROMICE and AAWS networks complete'; git push --tags
2018-01-19 Ajay Saini <[email protected]>
* MIT station_name array size corrected
* Changes to make compatible with Python 3
* Filename in print to take only basename not path
* Plot scripts can now be run from command line
* Added std deviation on top of mean of plots
* Automated plot labels to get month and year from data
* Added functionality to provide month from command line in plots
* Annual_cycle plot can now be run for leap years as well
* Added functionality to choose any variable from command line on which analysis need to be performed
2018-01-19 Charlie Zender <[email protected]>
* Use stn_nm_lng_max to dimension station_name character arrays
* Add copyright/license to README.md
2018-01-18 Ajay Saini <[email protected]>
* GCNET: Added new temp variables to store quality control variables
* GCNET: Compatible with classic netCDF formats
* GCNET: corrected calculation of hour variable to include other values
* Added global attribute featureType and cf_role for station_name
* Meeting with Wenshan to discuss coordinates and cell_methods attribute
* Coordinates attribute added to more variables
* Added cell_methods attribute
2018-01-18 Charlie Zender <[email protected]>
* Fix usage example on README.md
2018-01-17 Ajay Saini <[email protected]>
* Added station_name variable for all datasets
* Added functionality to change station_name from command line for all datasets
* Corrected _FillValue for GCNET
* Explored argparse, optparse and getopt modules to fix the canonical ordering. Still in progress.
* Changed sample data file for GCNET
2018-01-16 Ajay Saini <[email protected]>
* Presented plots in group meeting
* JAWS bi-weekly meeting to discuss issues and priorities of tasks
* Made latitude and longitude scalar
* Added the "coordinates" attribute with values "latitude longitude"
* Corrected file name matching for lat and long
* Renamed sample data files with station name
* Corrected _FillValue for all the variables
2018-01-12 Charlie Zender <[email protected]>
* Change degree_east/north to CF-preferred units degrees_east/north
2018-01-12 Ajay Saini <[email protected]>
* Created monthly diurnal cycle plot
* Created monthly values plot
* Created annual cycle plot
* Created climatological seasonal cycle plot
* Added _FillValue attribute for all datasets
* Worked on fixing canonical ordering of arguments. Still in progress
2018-01-11 Ajay Saini <[email protected]>
* GCNET- Corrected calculation of temp and pressure variables to skip '999' values.
* GCNET- For classic formats, facing issues with only quality control variables. Still in progress.
* PROMICE- Converted ddmm units to degree for GPS_latitude and GPS_longitude.
* Populated values of latitude and longitude of AAWS, GCNET stations. (Closed Issue #28)
* Calculated time_bounds variable for all datasets. (Closed Issue #29)
* Discussed Issue #9 with Wenshan. She is working on it, will let me know how to calculate ice velocity.
* Discussed Issue #8 with Wenshan. She said to wait for it as data isn't clean yet.
2018-01-10 Ajay Saini <[email protected]>
* GCNET- Recalculated time to includes hourly changes. Changed quality control variables and data-types.
* GCNET- Worked on making it compatible with classic formats. Still in progress.
* PROMICE- New latitude, longitude variables and corrected latitude_gps, longitude_gps
* PROMICE- Adjusted calculation of time variable to increase monotonically
* AAWS- New latitude, longitude variables created but no values yet. Need to discuss with Wenshan
* Read about bounds variable from CF document. Added to time coordinate for all datasets, work in progress to indicate exact temporal bounds. Need to discuss with Wenshan
2018-01-09 Ajay Saini <[email protected]>
* Edited netCDF format arguments
* Arguments can now be passed in any order
* GCNET - calculated new variables month, day and hour
* Worked on daily and monthly plots
2018-01-08 Ajay Saini <[email protected]>
* Solved bug to skip blank lines in input file of PROMICE
* Adjusted calculations of temp and pressure variables to avoid changing -999 values
* Default output name is now same as input name
* PROMICE and AAWS compatible with netCDF classic formats
2018-01-05 Ajay Saini <[email protected]>
* Changed algotrithms of AAWS, PROMICE for writing data to accomodate unlimited time dimension
* Functions to convert units to SI units (e.g. degC to kelvin)
* Edited the long names
* Extracted individual quality control variables for GCNet
* Changed units for all networks
* Worked on netCDF classic formats error. Not resolved yet.
2018-01-04 Ajay Saini <[email protected]>
* Added functionality to calculate date from current variables for GCNET.
* Working on python/matplotlib for plots.
* Completely expunged the sensitive data using filter-branch.
2018-01-03 Ajay Saini <[email protected]>
* Added optional argument to choose file format. For netCDF3 formats, the code is breaking at some points. Working to resolve it
* Explored seaborn and xarray package
* Created $DATA/jaws directory on JAWS at "C:\\Users\\Ajay\\data\\jaws". Raw data available and nc data is available as when converted.
2018-01-03 Charlie Zender <[email protected]>
* Add submitted quarterly report
2018-01-02 Ajay Saini <[email protected]>
* Created daily and monthly plots of summit data
* Working on adding output option for different netCDF formats
2018-01-02 Charlie Zender <[email protected]>
* Polish homepage
2018-01-01 Ajay Saini <[email protected]>
* aaws2nc.py: Created new variable 'time' to calculate seconds since '1970-01-01'
* aaws2nc.py: Adjusted 'station_name' variables to accomodate stations having long names
* Added functionality to automatically count timesteps in the dataset
* Corrected the time calculation function in gcnet2nc.py
2017-12-31 Ajay Saini <[email protected]>
* Added variable 'station_name' to aaws2nc and promice2nc
* Removed nrows argument
* Added functionality to automatically count timesteps in the dataset
* Corrected the time calculation function in gcnet2nc.py
2017-12-30 Ajay Saini <[email protected]>
* Created master list of variables that need standard names
* gcnet2nc.py, promice2nc.py: Edited op_file to name it as ip_file.nc
* jaws.py: edited the parser function to take input accordingly
2017-12-29 Ajay Saini <[email protected]>
* aaws2nc.py: astropy.io to write data properly in nc format
* Added sample one-day datasets of GCNET, PROMICE, AAWS
2017-12-26 Ajay Saini <[email protected]>
* Completed NASA SATERN learning assignment
* aaws: Collected data and created variables with their units and standard names
2017-12-25 Ajay Saini <[email protected]>
* Restructured the source directory and created jaws.py as main function
* gcnet2nc.py and promice2nc.py scripts respectively for GCNET and PROMICE networks
2017-12-24 Ajay Saini <[email protected]>
* Draft of Quarterly Technical Report
2017-12-23 Ajay Saini <[email protected]>
* imau2nc.py: Collected data and created variables with their units and standard_names
2017-12-14 Ajay Saini <[email protected]>
* jaws.py: Combined program to convert GCNET and PROMICE
2017-12-01 Ajay Saini <[email protected]>
* promice2netcdf.py: Convert PROMICE to NetCDF format
2017-11-27 Ajay Saini <[email protected]>
* gcnet2netcdf.py: Added functionality to take row_count input from user
2017-11-22 Ajay Saini <[email protected]>
* gcnet2netcdf.py: Derived quality control variables
2017-11-20 Ajay Saini <[email protected]>
* gcnet2netcdf.py: Added command line arguments
2017-11-14 Ajay Saini <[email protected]>
* gcnet2netcdf.py: Renamed some variables
2017-11-13 Ajay Saini <[email protected]>
* gcnet2netcdf.py: script for converting GCNet L2 ASCII data to netCDF format
2017-10-29 Ajay Saini <[email protected]>
* jaws 0.1: Updated installation commands
2017-10-25 Ajay Saini <[email protected]>
* Created new directories
2017-10-23 Ajay Saini <[email protected]>
* jaws 0.1: Original scripts from Wenshan
2017-10-09 Ajay Saini <[email protected]>
* Initial commit
* README.md: Instructions about jaws