Differences

This shows you the differences between two versions of the page.

--- wiki:observational_astronomy:data_reduction [2023/11/07 13:15] – Roy Prouty
+++ wiki:observational_astronomy:data_reduction [2023/11/09 13:50] (current) – removed Roy Prouty
@@ Line 1: / Line 1: @@
-====== Data Reduction ======
-The term //reduction// refers to the removal/reduction of extra counts via the calibration of a light image. This allows astronomers to account for the unwanted signal from bias and thermal effects as well as from non-uniformities in the detector or optical system. This is absolutely necessary for any physical values to be extracted from an image.
-=== Our Pipeline ===
-This page outlines the best practices for our system. It's currently a work in progress. [[wiki:observational_astronomy:pipeline|Redux Pipeline]]
-=== Unwanted Signals ===
-  * [[wiki:observational_astronomy:thermal_signal|Thermal Signal]]
-  * [[wiki:observational_astronomy:bias_signal|Bias Signal]]
-  * [[wiki:observational_astronomy:uniformity|Uniformity]]
-=== Theory & Background ===
-We can represent each $i$ and $j$-th pixels in a light frame as the sum of the contributions from the following signals:
-$$L_{ij}(\lambda) = \Bigl[\epsilon_{ij}(\lambda)I_{ij}(\lambda) + T_{ij}(\lambda)\Bigr]t + B_{ij}(\lambda)$$
-== Raw Light ==
-$L_{ij}(\lambda)$ is measured in counts and is the signal shown in SharpCap and saved in the data portion of the FITS file. This is the raw data in any light frame.
-While not super important for this data reduction example, it's otherwise important to note that these counts are rather limited in their numerical precision and therefore in their ability to precisely represent continuous physical values, i.e., $L_{ij}(\lambda)\in \mathbb{Z}<2^{\tiny{BITPIX}}-1$.\\
-== Uniformity ==
-$\epsilon_{ij}(\lambda)$ is the pixel-by-pixel uniformity. This is measured as a fraction with each pixel carrying a (to computer precision) continuous value in the range $\epsilon_{ij}(\lambda)\in \mathbb{R}_{[0,1]}$ Broadly, this is the pixel-by-pixel sensitivity to incident photons. "Baked-in" is more than the detector quantum efficiency, it "bakes-in" the reduced efficiency due to any filter, any non-uniformity due to hardware or software amplification via GAIN modulations, as well as any other imperfections in the optical system that impact the ability for the detector to record photons incident on the primary mirror (all of that schmutz in our system!).\\
-== Source Signal ==
-$I_{ij}(\lambda)$ is the source signal. It is measured in counts, and precision in the counts recorded come at the expense of signal contributions from the Dark and Bias signals (see below). The aim of any reduction pipeline is to take the signals in a raw light frame and reduce the contributions of other signals such that only this source signal is present. The result is called a science frame or science image and hypothetically is the closest thing to $I_{ij}(\lambda)$ we can muster with our system and reduction pipeline.\\
-== Thermal Signal ==
-$T_{ij}(\lambda)$ is the thermal signal, it is a component of the dark signal along with the Bias (below). Thermal signal is a measure of the electrons that gained enough thermal energy to liberate themselves from the valence band of the CMOS semiconductor substrate and be read-off as signal by the Analog-Digital-Converter (ADU). As such, the thermal signal is time-dependent. I.e., the longer you give electrons in the valence band time to explore microstates made available by some heat bath, the more likely some will find the microstates that allow them to get to the conduction band. Since the thermal signal is not generated by photons and the quantum efficiency of the pixels play no part, we can safely drop any implied $\lambda$ dependence. $T_{ij}(\lambda) \rightarrow T_{ij}$ \\
-== Integration Time ==
-$t$ is the exposure or integration time. "Integration" time is favored by the UMBC Obs Group because it reminds the reader that we are integrating radiance or flux over time. This integration also couples time with the counts generated by the source and thermal signals.\\
-== Bias Signal ==
-$B_{ij}(\lambda)$ is the bias signal, it is a component of the dark signal along with the thermal signal (above). Bias signal is a measure of the electrons that were liberated by the internal voltages that are constantly and non-uniformly applied across all pixels. These internal voltages are responsible for 'storing' liberated electrons (either by photons or thermal excitation) until read-time by the ADC. These internal voltages are responsible for some initial 'jolt' of extra elections, but after this initial 'jolt' electrons that trickle into the storage circuits are liberated by thermal agitation. In this way, the Bias signal is not time dependent. Further, since the bias signal is not generated by photons and the quantum efficiency of the pixels play no part, we can safely drop any implied $\lambda$ dependence. $B_{ij}(\lambda) \rightarrow B_{ij}$\\
-=== Dark Signal ===
-It's likely clear from the above that the Dark Signal is built from a time-integrated thermal signal as well as constant (but non-uniform) bias signal.
-$$D_{ij}(\lambda) = B_{ij} + T_{ij}t$$
-==== Isolating Source Signal ====
-$$L_{ij}(\lambda) = \Bigl[\epsilon_{ij}(\lambda)I_{ij}(\lambda) + T_{ij}(\lambda)\Bigr]t + B_{ij}(\lambda)$$
-Removing spectral dependence
-$$L_{ij}(\lambda) = \Bigl[\epsilon_{ij}(\lambda)I_{ij}(\lambda) + T_{ij}\Bigr]t + B_{ij}$$
-Distribute integration time
-$$L_{ij}(\lambda) = \epsilon_{ij}(\lambda)I_{ij}(\lambda)t + T_{ij}t + B_{ij}$$
-Remove Bias Signal
-$$L_{ij}(\lambda) - B_{ij}  = \epsilon_{ij}(\lambda)I_{ij}(\lambda)t + T_{ij}t $$
-Remove Thermal Signal
-$$L_{ij}(\lambda) - B_{ij} - T_{ij}t = \epsilon_{ij}(\lambda)I_{ij}(\lambda)t $$
-Divide through by uniformity
-$$\frac{L_{ij}(\lambda) - B_{ij} - T_{ij}t}{\epsilon_{ij}(\lambda)} = I_{ij}(\lambda)t $$
-Finally arrive at science frame (Note that the integration time was left with the Source Signal)
-$$I_{ij}(\lambda)t = \frac{L_{ij}(\lambda) - B_{ij} - T_{ij}t}{\epsilon_{ij}(\lambda)}$$
-Some may choose to view the Dark Signal as one object -- the time-integrated Thermal Signal and Constant Bias Signal
-$$I_{ij}(\lambda)t = \frac{L_{ij}(\lambda) - (B_{ij} + T_{ij}t)}{\epsilon_{ij}(\lambda)}$$
-$$I_{ij}(\lambda)t = \frac{L_{ij}(\lambda) - D_{ij}(\lambda)}{\epsilon_{ij}(\lambda)}$$
-This is not the full story ... We need to be careful in how we collect these Light, Flat, Dark, and Bias frames. Further, since they are fundamentally samplings of random processes, we need to ensure we're sampling these random processes well and also find some measure of central tendency to use -- or we risk introducing more noise!\\
-Refer to the Redux Pipeline and Unwanted Signals pages above for more information on each of these.