Create metric to identify images that are affected by non-linearity

[detoma]

Non linearity is commonly seen in UCOMP in the bright chromospheric lines, which we do not serve to the public. It is not usually a problem for coronal lines whose intensity is below the non linearity threshold of the cameras. However, when the sky is bright an area around the occulter can fall inside the non linearity regime.

We do not currently have a non-linearity correction for the OWL cameras.

During the v1.1 reprocessing, we want to estimate how many files are affected by non-linearity. At this point we are not trying to monitor this issue at the frame level, but just to get a sense of the magnitude of it.

[detoma]

This test needs to be done on dark-corrected data. Since like-state are averaged together before being dark-corrected, we will do this counting on the average dark-corrected frames before they are flat fielded. This is not a precise counting but is sufficient for a first assessment of the problem. The non linearity we see in coronal images is caused by bight sky and when this happens it typically affects most/all frames in a file.

We want to count how many pixels are above certain threshold in each dark corrected frame, this number can be added to the header. In the database, we want to record the average number of non-linear pixels per file.

Non linearity thresholds are a function of camera. For each camera, there are 3 levels of non linearity, I would like to track:

• above 1% non-linearity
• above 2% non-linearity
• above 5% non linearity

I will update the thresholds later, if necessary, but my initial analysis indicates they are the following:

camera 10006, e.g. camera 0
DN > 2750 non-linearity of 1% or more
DN > 2800 non linearity of 2% or more
DN > 2960 non -linearity of 5% or more

camera 10055, e.g. camera 1
DN > 2495 non-linearity of 1% or more
DN > 2640 non linearity of 2% or more
DN > 2820 non -linearity of 5% or more

This is for the cameras in use until November 2022. All unites are dark-corrected DN. It applies to high gain and an exposure of 80ms, which is most/all of the coronal data we have. The NUC does not change the values significantly. We will not compute the non-linearity pixels for low gain or exposures shorter than 80ms for now.

If a non linearity correction is necessary, this must be applied to the individual dark-corrected frames, not on the average dark-corrected frame, and will require to change the order of operation in the pipeline.