Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enhance adaptive suprathreshold #2302

Open
wants to merge 28 commits into
base: main
Choose a base branch
from
Open

Enhance adaptive suprathreshold #2302

wants to merge 28 commits into from

Conversation

t-b
Copy link
Collaborator

@t-b t-b commented Nov 26, 2024 •
edited
Loading

Close #2258

@t-b t-b self-assigned this Nov 26, 2024
@t-b t-b force-pushed the feature/2302-another-adaptive-revision branch from daec7e1 to 886e9e6 Compare November 27, 2024 22:53
@t-b t-b mentioned this pull request Jan 6, 2025
Open
3 tasks
@t-b t-b force-pushed the feature/2302-another-adaptive-revision branch 2 times, most recently from d7b9af8 to 7a4e927 Compare January 21, 2025 17:56
@t-b t-b force-pushed the feature/2302-another-adaptive-revision branch 3 times, most recently from 87bc99b to 03ebd15 Compare January 26, 2025 20:44
@t-b

This comment was marked as outdated.

Sign in to view
@t-b t-b force-pushed the feature/2302-another-adaptive-revision branch 2 times, most recently from c643b38 to eaa0825 Compare January 27, 2025 18:48
@t-b t-b assigned timjarsky and unassigned t-b Jan 27, 2025
@t-b t-b marked this pull request as ready for review January 27, 2025 18:48
@t-b t-b assigned t-b and unassigned timjarsky Jan 27, 2025
t-b added 11 commits January 28, 2025 14:36
@t-b
PSQ_DAScale (Adapt): Allow to look further back for passing adaptive …
cffcd8a 
…sweeps

The new FailingAdaptiveSCIRange analysis parameter allows to tune the
formerly hardcoded (1) number of SCIs we look back for failing adaptive
sets.

It defaults to 1, can be turned off with 0 and allows inf for always
taking all SCIs into account.
@t-b
PSQ_DAScale (Adapt): Round calculated DAscale to integer values
fce7ff5 
We don't need the sub pA precision. In PSQ_DS_GatherOvershootCorrection we
now also check if the calulated value is the same as the before/after
value, as this can now happen with rounding.

In PSQ_DS_CalculateDAScale we also use ceil for the too large/small cases
so that we always get a different value.
@t-b
PSQ_DS_GatherOvershootCorrection: Optimize execution
0064cb0 
We now perform the already measured check only if we have a candidate at all.
@t-b
PSQ_DAScale (Adapt): Add the type to the stored future DAScale values
5563bde 
Debugging the future DAScales labnotebook entries can be challenging as it
is not obvious form where the values where added.

We now fix that by adding a type to the futureDAScales waves which is then
also stored in the labnotebook.
@t-b
PSQ_DAScale (Adapt): Add negative fI slope QC entries
c426c0a 
Both variants for the adaptive data and for RhSuAd.
@t-b
tests: Add enableRegExp to RunWithOpts
797efaa
@t-b
EqualValuesOrBothNaN: Fix wording in documentation
b25f02f
@t-b
Add LesserEqualValuesOrBothNaN and LargerEqualValuesOrBothNaN
e964bc3
@t-b
FindNeighbourDuplicates: Rename and add support for predicates
c92384a 
For a future application we want to check for something different than
equality, so making the predicate configurable here is required.
@t-b
Packages/MIES/MIES_AnalysisFunctions_PatchSeq.ipf: Remove duplicated …
de24366 
…PSQ_FMT_LBN_SWEEP_PASS

This has a different explanation but in this sense it is never used.
Introduced in c53b5d0 (PSQ_DaScale: Enhance adaptive mode, 2024-07-26).
@t-b
Tests/GenerateAnalysisFunctionTable: Update it
b4a3395
t-b added 17 commits January 28, 2025 14:36
@t-b
PSQ_FMT_LBN_DA_AT_FI_SLOPE: Add it
a1bdcb6 
Having a duplicate version for PSQ_DAScale Adaptive makes it easier to
check if we have all variants (offsets, RhSuAd, DAScale, etc.) properly
done.
@t-b
PSQ_DS_StoreFitOffsetAndSlope: Factor it out
fbbeb68
@t-b
PSQ_DS_ConsecutivePasses: Factor it out
dc47d32
@t-b
PSQ_DAScale (Adaptive): Add new labnotebook entry being sweep QC with…
ae6b79d 
…out baseline QC

Needed for a future application.
@t-b
MIES_AnalysisFunctions_PatchSeq.ipf: Add forgotten PSQ_FMT_LBN_DA_AT_…
2b4d3d4 
…RSA_FI_OFFSETS
@t-b
PSQ_DS_GetFutureDAScalesFromLBN: Factor it out
385af1a
@t-b
NumericWaveToList: Fix documentation
32d71ac
@t-b
MergeSortStableInplace: Add it
518db9a 
The stock Igor Pro functions don't maintain the relative order of the same
key values. So we had to roll our own.
@t-b
SortKeyAndData: Add it
53a89a1
@t-b
PSQ_DAScale (Adapt): Return always a copy of the waves for PSQ_DS_Fil…
9790282 
…terFrequencyCurrentData
@t-b
PSQ_DS_AdaptiveIsFinished: Handle oorDAScale case earlier
abd7eb9 
In that way this is not dead code in the tests. It is also clearer that we
check this error condition first.
@t-b
PSQ_DAScale (Adapt): Enhance it
0a00e7d 
- Ignore sweeps with negative fI slopes for the DAScale estimation
- Store a new QC value if the fI slope is finite and negative
- Using this new QC we are now also done if:
  - We have to consecutive negative fI slope values (as usual failing
    sweep QC sweeps can be in between)
  - or we have a passing fI slope and a consecutive (as above) negative fI
    slope value
  If we finish due to these two conditions we also fillin another DAScale
  in between the positive FI slope sweep and the first negative fI slope
  value.
- We now use the new analysis parameter DAScaleNegativeSlopePercent,
  defaulting to 50%, to calculate a new DAScale value after a negative
  slope. We use the percentage of the maximum DAScale step for that.
- Add a new labnotebook entry which denotes if a value is filled in or not
- Also a bug is fixed which did not ignore fillin sweeps when trying to
  determine if the set is passing.
- And similiar bug could result in fillin sweeps being marked as fI slope
  passing.
@t-b
UTF_PatchSeqDAScale_Adapt.ipf: Update test matrix
c33ede9
@t-b
PSQ_DA_SCALE_VERSION: Raise it
3b6346f
@t-b
patch-seq-dascale.dot: Adapt it
3a083ad
@t-b
LABNOTEBOOK_VERSION: Raise it
c4277fd
@t-b
PSQ_DS_FitFrequencyCurrentData: Remove it
17d9733 
In (PSQ_DAScale: Change fitting logic and drop NumPointsForLineFit,
2024-07-10) we added dropping duplicated AP frequencies in the input data
for fitting. We do catch that now with the neg fI slope QC better. And
that also removes the hack to find a good DAScale in case this was
filtered out.
@t-b t-b force-pushed the feature/2302-another-adaptive-revision branch 2 times, most recently from 00b6e6f to ae1e170 Compare January 28, 2025 14:38
@t-b t-b assigned timjarsky and unassigned t-b Jan 28, 2025
@t-b
Copy link
Collaborator Author

t-b commented Jan 28, 2025

@timjarsky Ready to test.

@t-b t-b force-pushed the feature/2302-another-adaptive-revision branch from ae1e170 to 17d9733 Compare January 28, 2025 14:45
@t-b t-b self-assigned this Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@timjarsky timjarsky timjarsky left review comments

@MichaelHuth MichaelHuth Awaiting requested review from MichaelHuth MichaelHuth is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

@t-b t-b

@timjarsky timjarsky

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Adaptive enhancements
2 participants
@t-b @timjarsky