Data Reduction

The term reduction refers to the removal/reduction of extra counts via the calibration of a light image. This allows astronomers to account for the unwanted signal from bias and thermal effects as well as from non-uniformities in the detector or optical system. This is absolutely necessary for any physical values to be extracted from an image.

Redux Pipeline

This page outlines the best practices for our system. It's currently a work in progress.

• Thermal Signal
• Bias Signal
• Uniformity

We can represent each $i$ and $j$-th pixels in a light frame as the sum of the contributions from the following signals:

$$L_{ij}(\lambda) = \Bigl[\epsilon_{ij}(\lambda)I_{ij}(\lambda) + T_{ij}(\lambda)\Bigr]t + B_{ij}(\lambda)$$

$L_{ij}(\lambda)$ is measured in counts and is the signal shown in SharpCap and saved in the data portion of the FITS file. This is the raw data in any light frame. While not super important for this data reduction example, it's otherwise important to note that these counts are limited in their numerical precision and therefore in their ability to precisely represent continuous physical values, i.e., $L_{ij}(\lambda)\in \mathbb{Z}<2^{\tiny{BITPIX}}-1$.

$\epsilon_{ij}(\lambda)$ is the pixel-by-pixel uniformity or deviation from uniformity. This is measured as a fraction with each pixel carrying a (to computer precision) continuous value in the range $\epsilon_{ij}(\lambda)\in \mathbb{R}_{[0,1]}$. In an ideal system, $\epsilon_{ij}(\lambda)=1~\forall~i,j$. Broadly, this is the pixel-by-pixel sensitivity to incident photons. “Baked-in” is more than the detector quantum efficiency, it “bakes-in” the reduced efficiency due to any filter, any non-uniformity due to hardware or software amplification via gain modulations, as well as any other imperfections in the optical system that impact the ability for the detector to record photons incident on the primary mirror (all of that schmutz on our mirrors!).

$I_{ij}(\lambda)$ is the source signal. It is measured in counts, and precision in the counts recorded come at the expense of signal contributions from the Dark and Bias signals (see below). The aim of any reduction pipeline is to take the signals in a raw light frame and reduce the contributions of other signals such that only this source signal is present. The result is called a science frame or science image and hypothetically is the closest thing to $I_{ij}(\lambda)$ we can muster with our system and reduction pipeline.

$T_{ij}(\lambda)$ is the thermal signal, it is a component of the dark signal along with the bias (below). Thermal signal is a measure of the number electrons that accrued enough thermal energy to liberate themselves from the valence band of the CMOS semiconductor substrate and be read-off as signal by the Analog-Digital-Converter (ADU). As such, the thermal signal is time-dependent. I.e., the longer you give electrons in the valence band time to explore microstates made available by some heat bath, the more likely some will find the microstates that allow them to get from the valence band and into the conduction band to be read-off. Since the thermal signal is not generated by photons and the quantum efficiency of the pixels play no part, we can safely drop any implied $\lambda$ dependence from the thermal signal term. $T_{ij}(\lambda) \rightarrow T_{ij}$

$t$ is the exposure or integration time. “Integration” time is favored by the UMBC Obs Group because it reminds the reader that we are integrating radiance or flux over time. This integration also couples time with the counts generated by the source and thermal signals.

$B_{ij}(\lambda)$ is the bias signal, it is a component of the dark signal along with the thermal signal (above). Bias signal is a measure of the electrons that were liberated by the internal voltages that are constantly and non-uniformly applied across all pixels. These internal voltages are responsible for shuttling liberated electrons (either by photons or thermal excitation) into storage circuits until read-time by the ADC. These internal voltages are responsible for some initial 'jolt' of extra elections, but after this initial 'jolt', thermal electrons can continue to trickle into the storage circuits (these contribute to the thermal signal). In this way, the bias signal is not time-integrated. Further, since the bias signal is not generated by photons and the quantum efficiency of the pixels play no part, we can safely drop any implied $\lambda$ dependence from the bias signal term. $B_{ij}(\lambda) \rightarrow B_{ij}$

$$L_{ij}(\lambda) = \Bigl[\epsilon_{ij}(\lambda)I_{ij}(\lambda) + T_{ij}(\lambda)\Bigr]t + B_{ij}(\lambda)$$

Removing spectral dependence

$$L_{ij}(\lambda) = \Bigl[\epsilon_{ij}(\lambda)I_{ij}(\lambda) + T_{ij}\Bigr]t + B_{ij}$$

Distribute integration time $$L_{ij}(\lambda) = \epsilon_{ij}(\lambda)I_{ij}(\lambda)t + T_{ij}t + B_{ij}$$

Remove Bias Signal $$L_{ij}(\lambda) - B_{ij} = \epsilon_{ij}(\lambda)I_{ij}(\lambda)t + T_{ij}t $$

Remove Thermal Signal $$L_{ij}(\lambda) - B_{ij} - T_{ij}t = \epsilon_{ij}(\lambda)I_{ij}(\lambda)t $$

Divide through by uniformity $$\frac{L_{ij}(\lambda) - B_{ij} - T_{ij}t}{\epsilon_{ij}(\lambda)} = I_{ij}(\lambda)t $$

Finally arrive at science frame (Note that the integration time was left with the Source Signal) $$I_{ij}(\lambda)t = \frac{L_{ij}(\lambda) - B_{ij} - T_{ij}t}{\epsilon_{ij}(\lambda)}$$

Some may choose to view the Dark Signal as one object – the time-integrated Thermal Signal and constant Bias Signal $$I_{ij}(\lambda)t = \frac{L_{ij}(\lambda) - (B_{ij} + T_{ij}t)}{\epsilon_{ij}(\lambda)}$$

$$I_{ij}(\lambda)t = \frac{L_{ij}(\lambda) - D_{ij}(\lambda)}{\epsilon_{ij}(\lambda)}$$

In an effort to be super clear, the Dark Signal (and DARK frames as we'll discuss later) is built from the time-integrated thermal signal and the constant (but not uniform) bias signal. This is often glossed-over, but plays an important part in our data reduction pipeline as well as contributing to best practices for calibration frame collection.

$$D_{ij}(\lambda) = B_{ij} + T_{ij}t$$

There is no conclusion! Not yet, anyway … We need to be careful in how we collect these Light, Flat, Thermal, and Bias calibration frames. Further, since they are fundamentally samplings of random processes, we need to ensure we're reasonably sampling these random processes. So we will take multiple calibration frames for each type. And given this, it'll be important that we find some measure of central tendency to effectively summarize the contribution of the various unwanted signals to the light frames.

Refer to the Redux Pipeline and Unwanted Signals pages above for more information on each of these.