Create master flats

[detoma]

The flats in KCor (and COMP) are taken with the occulter in. Science data and flats do not have the flats necessarily in the same position as the occulter moves over the course of the day. As a result, there is a region of pixels around the occulter for which we cannot performa a proper flat-fielding. I think an average flat value is used for that area in the KCor pipeline.

A way to improve this and avoid artifacts is to create a master flat. The idea is that by combining many flats together which do not have the occulter in the same location, we can cover a larger area of the detector and can flat-field every pixels in a science image.

The way to do this is to use all the good flats available for each cal_epoch and create a 3D array. Flats must be dark-corrected and scaled using vdimref. We will then compute the median flat value for each pixel in the detector that is not under the occulter using all flats. We can also compute the sigma to test flat stability over time.

Since flats tend to be taken around the same time every day, I am not sure that the occulter moves enough from one flat to the next to cover all the area of the detector that we need. I will take a look at the period 20220316.103744 to 20220830.123347 to test this. If the flats we already have do not cover enough area of the detector, when we reopen MLSO, we can take flats at different times of the day to acquire the data we need. Another option is to take flats without the occulter, which I think it is possible for KCor, but stray light could be different in the two configurations, so we need to test how they compare.

The project is pretty simple. The time consuming part is to collect the flats and darks to process for each epoch.

[detoma]

Joan can you please double check if the steps below look right to you to create a master flat:

• stay within a calibration epoch (we have identified a long one in 2022 for testing)
• loop through all the flats
  1. remove dark
  2. divide by the SGS vdimref value at the time of the flat
  3. identify the occulter and mark it as NaN
  4. store flats into a 3D array
• use the 3D array to compute the median and sigma for each pixel in the detector excluding values that are NaN.

Is there any other correction, e.g. camera non-linearity correction, that I missed?

If this works, since the flats are already normalized by vdimref, when we apply it to the dark corrected data in the crescent moon, we simply need to divide those pixels by vdimref at the time of the image and for the master flat.

[mgalloy]

Here is a better histogram of when the KCor flats (cover="out" and darkshut="out" and diffuser="in" and calpol="out") were taken:

Found using the query:

select date_obs, cover, darkshut, diffuser, calpol from kcor_cal where cover="out" 
  and darkshut="out" and diffuser="in" and calpol="out" order by date_obs;

[detoma]

Thank you Mike for the histogram. This is encouraging. We may have enough images to cover more of the detector.