Fix issues reported from users and prototype manual reduction #41

wdzhou · 2018-11-04T21:22:56Z

In this PR, multiple issues reported from users will be fixed. Minor improvements will be implemented according to user's suggestion. And prototype on manual reduction will be made.

Tests (Criteria)

Fixes and Mentions
It mentions #37, #40, #38, #42, #45, #47 and #48.

verify_pyrs_reduction.sh 1

…fitting.

1. Make the pyrscore for each interface instance unique. 2. Add a 'Save' button to peak fit interface to save fit result. In mplgraphicspolar.py To-do: 1. Test pole figure plotting. 2. Do a general test to launch multiple

pyrs/core/mantid_fit_peak.py:249: # TODO - 20181101 - Need to expand this method such that all fitted parameters will be added to output pyrs/interface/ui/diffdataviews.py:113: # TODO - 20181101 - Enable after auto_scale is fixed: self.auto_rescale() pyrs/interface/ui/mplgraphicsview1d.py:287: # TODO FIXME - 20181101 - This is a broken method. Fix it! pyrs/core/strain_stress_calculator.py:497: # TODO FIXME - 20181001 - Temporarily disabled in order to clean up for the new workflow pyrs/core/strain_stress_calculator.py:1417: # TODO - 20181010 - check whether all the raw files (e11/e22/e33) are ready for next! pyrs/interface/strainstresscalwindow.py:388: # TODO - 20181010 - Refactor! pyrs/interface/strainstresscalwindow.py:872: # TODO - 20181011 - Implement (from cleaning old codes) pyrs/interface/strainstresscalwindow.py:873: # TODO - ASAP - 20181011 - Continue from here! pyrs/interface/strainstresscalwindow.py:1056: # # TODO - 20181010 - Consider how to integrate method plot_peak_parameter() pyrs/interface/manualreductionwindow.py:140: # TODO - 20181009 - Need to refine pyrs/interface/manualreductionwindow.py:207: # TODO - 20181006 - Implement ASAP pyrs/interface/manualreductionwindow.py:297: # TODO - 20181008 - ASAP tests/unittest/straincalculationtest.py:75: # TODO FIXME - 20181001 - Make an individual method for .... ---> New workflow! tests/unittest/straincalculationtest.py:77: # TODO FIXME - 20181001 - This is a new suite of methods to analyze the sample grids

Also including 1. improved UI for reduced data view and detector view;

And options to show fitting error.

1. Implemented more codes on reduction including auto reduction script; 2. Started to develop a new widget containing 2 subplots for data/mode and residual respectively.

wdzhou · 2018-11-26T21:54:05Z

2018.11.26

A bug (in development) to fix:

[DB...BAT] Data key: 16-1_TD.cor_Log.hdf5_44 is of type <type 'str'>
Traceback (most recent call last):
  File "build/scripts-2.7/peakfitgui_test.py", line 61, in <module>
    test_window = test_main()
  File "build/scripts-2.7/peakfitgui_test.py", line 30, in test_main
    fit_window.do_load_scans()
  File "/home/wzz/Projects/PyRS/pyrs/interface/fitpeakswindow.py", line 257, in do_load_scans
    self.do_fit_peaks()
  File "/home/wzz/Projects/PyRS/pyrs/interface/fitpeakswindow.py", line 287, in do_fit_peaks
    self._core.fit_peaks(data_key, scan_log_index, peak_function, bkgd_function, fit_range)
  File "/home/wzz/Projects/PyRS/pyrs/core/pyrscore.py", line 332, in fit_peaks
    peak_optimizer.calculate_peak_position_d(wave_length_vec=wave_length_vec)
  File "/home/wzz/Projects/PyRS/pyrs/core/mantid_fit_peak.py", line 95, in calculate_peak_position_d
    centre_vec = self.get_fitted_params(param_name='centre')
  File "/home/wzz/Projects/PyRS/pyrs/core/mantid_fit_peak.py", line 380, in get_fitted_params
    raise KeyError(err_msg)
KeyError: "Function parameter centre does not exist. Supported parameters are ['wsindex', 'peakindex', 'LorentzAmp', 'LorentzPos', 'LorentzFWHM', 'GaussianFWHM', 'A0', 'A1', 'chi2']"

… file reader and writer.

wdzhou · 2018-12-04T21:53:09Z

pyrs/core/polefigurecalculator.py:560:    # TODO - 20181204 - Implement head #36 - ASAP(3)
pyrs/core/reductionengine.py:63:        # TODO - 20181204 - Refer to "WANDPowderReduction" - ASAP(0)
pyrs/interface/textureanalysiswindow.py:503:        # TODO - 20181204 - Replace getSaveFileName by a self-extended method to be fine with both Qt4 and Qt5 ASAP(1)

…PyRS into prototype_manual_reduction_032

Next: Find out the difference of pixel positions of shifted detector panel. It is important to locate the source of discrepancy.

…d shift.

…bins.

Nex step: add normalization by number of data points in a bin. Note: need to take care of zero data point in a bin.

Next step: 1. complete tests/unittest/reduction_study.py 2. wait for a modified version of ResampleX which can accept same X.

Next step: 1. Make "reduction_study" work (verify with compare_reduction_engines_test 2. VecE will be used for pixel IDs 3. Find a cycle from dip to dip from reduced data 4. After SoftXAxis examine this cycles by printing out all the 2theta, count, pixel IDs 5. Create a normalization workspace (sorted 2theta as (4), then a vector of 1). 6. After ResampleX, check intensity/counts vs number of bins

wdzhou · 2019-04-25T20:20:54Z

2019.04.26

Finished work include

Verified that Mantid Resamplex and numpy.histogram can render very close result
- compare unsorted 2theta between PyRS and Mantid engine
- use reduction_study to compare 2 histogram methods: ResampleX and numpy.histogram
An insight how the patterned dips are formed

wdzhou added 14 commits October 29, 2018 17:05

Refs #38. Minor changes at strain/stress UI.

3d507f0

Refs #40. Enable peak fit (core) to calculate center in dspace after …

9ebd37a

…fitting.

Refs #38. Made 1D slice work.

c1a7a03

Refs #38. Caught an except for a non-existing direciton E33.

67c8c81

Refs #39. Tried to fix auto scale with plot.

c313cff

Refs #42 and #43 and #44. Initialize pole figure with NaN and.

daf7f5e

1. Make the pyrscore for each interface instance unique. 2. Add a 'Save' button to peak fit interface to save fit result. In mplgraphicspolar.py To-do: 1. Test pole figure plotting. 2. Do a general test to launch multiple

Refs #32. Checkpointing some progress.

60c42e7

Refs #32. Checkpointing progress in implementing data reduction engine.

69602ed

Refs #32. Added a test file for reduction.

b578373

Also including 1. improved UI for reduced data view and detector view;

Refs #47 and #48. Improved UI for separate fit result window.

06778f6

And options to show fitting error.

Refs #47 and #32. Added auto reduction script and new mpl widgets.

c8e0973

1. Implemented more codes on reduction including auto reduction script; 2. Started to develop a new widget containing 2 subplots for data/mode and residual respectively.

Refs #32. Fixed many issues of the data/model and residual plot widget.

ec9796a

Refs #32. Enabled synchronization of zoom operation on 2 subplots.

3f2e2e6

wdzhou added 4 commits November 30, 2018 19:22

Refs #45. Checkpointing progress to implement calibration information…

4ef20d8

… file reader and writer.

Refs #45. Checkpointing progress on implementing calibration manager.

27ae144

Re-organize the prototyp scripts.

f9eed8e

Refs #32 and #36. Changed the MTEX's posfix to jul.

82d1174

wdzhou added 10 commits December 4, 2018 21:30

Refs #32 and #36. Fixed MTEX header issue and progress in reduction.

da41b4f

Refs #40 and #48. Set up the fitting result table dynamically.

788f5cc

Refs #32. Continuing to modify peak fitting result table.

b29547f

Refs #45. Created class for calibrated geometry parameters.

ee359ac

Refs #32. Added test data.

012dd64

Merge branch 'prototype_manual_reduction_032' of github.com:neutrons/…

d161732

…PyRS into prototype_manual_reduction_032

Refs #32. Removed a prototpye.

ce3d75d

Refs #32. Refactored reduction UI and added IPython console.

11baa89

Refs 32. Added reduction prototype and modifed reduction UI.

3748b5e

Refs #32. Added prototype files.

a383d34

wdzhou added 21 commits April 1, 2019 23:22

Refs #32. Checkpointing progress to implement PyRS instrument class.

32f3767

Refs #32. Checkpointing progress in converting to 2theta comparison.

ebf2a04

Refs #32. Fixed the issue to convert to 2theta by Mantid.

a0580f7

Next: Find out the difference of pixel positions of shifted detector panel. It is important to locate the source of discrepancy.

Refs #32. Tried to compare pixel by pixel for position.

bf04989

Refs #32. fixed a bug.

255ab6a

Refs #32. Find out the source of discrepancy: resolution of calibrate…

e4808e0

…d shift.

Refs #32. Checkpointing progress to compare reduction result.

425394d

Refs #32. Disabled to outputs.

3f2b867

Refs #32. Implemented a simple histogram method for comparison check.

11c16dc

Refs #32. Used numpy histogram for Mantid reduction.

90a27cc

Refs #32. Checkpointing progress to testing reducing to 2theta.

d2b41c3

Refs #72. The reduced value diffrence may come from different 2theta …

81ce1e7

…bins.

Refs #32. Passed no mask reduction.

53f9a9d

Refs #32. Passed 0 and 10 degree mask test.

0ea4abf

Nex step: add normalization by number of data points in a bin. Note: need to take care of zero data point in a bin.

Refs #32. Added normalization to mantid one.

dd02c59

Refs #32. Added normalization for mantid.

863716b

Refs #32. Added another testing script for using PyRS only.

29ccc46

Next step: 1. complete tests/unittest/reduction_study.py 2. wait for a modified version of ResampleX which can accept same X.

Refs #32. Complete a new script to study the binning.

6c13b90

Refs #32. Refactored PyRS instrument builder.

4c69c27

Refs #32. Resolved the conflict.

b19b2ba

wdzhou added 4 commits April 26, 2019 08:46

Refs #32. Implemented reduction script.

8b61c04

Refs #32. Checkpointing progress.

640801a

Refs #32. Fixed a plot issue.

619766f

Refs #32. Final commit on this long (time) PR with minor change.

edf96fd

wdzhou merged commit f149e4e into master Apr 26, 2019

peterfpeterson deleted the prototype_manual_reduction_032 branch October 3, 2019 20:18

wdzhou restored the prototype_manual_reduction_032 branch October 11, 2019 16:09

peterfpeterson deleted the prototype_manual_reduction_032 branch November 6, 2019 20:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix issues reported from users and prototype manual reduction #41

Fix issues reported from users and prototype manual reduction #41

wdzhou commented Nov 4, 2018 •

edited

Loading

wdzhou commented Nov 26, 2018 •

edited

Loading

wdzhou commented Dec 4, 2018

wdzhou commented Apr 25, 2019

Fix issues reported from users and prototype manual reduction #41

Fix issues reported from users and prototype manual reduction #41

Conversation

wdzhou commented Nov 4, 2018 • edited Loading

wdzhou commented Nov 26, 2018 • edited Loading

wdzhou commented Dec 4, 2018

wdzhou commented Apr 25, 2019

wdzhou commented Nov 4, 2018 •

edited

Loading

wdzhou commented Nov 26, 2018 •

edited

Loading