MUSE, a 2nd generation VLT instrument, will become the world's largest integral field spectrograph. It will be an AO assisted instrument which, in a single exposure, covers the wavelength range from 465 to 930 nm with an average resolution of 3000 over a field of view of 1'x1' with 0.2'' spatial sampling. Both the complexity and the rate of the data are a challenge for the data processing of this instrument. We will give an overview of the data processing scheme that has been designed for MUSE. Specifically, we will use only a single resampling step from the raw data to the reduced data product. This allows us to improve data quality, accurately propagate variance, and minimize spreading of artifacts and correlated noise. This approach necessitates changes to the standard way in which reduction steps like wavelength calibration and sky subtraction are carried out, but can be expanded to include combination of multiple exposures.
Deep Dive into Advanced Data Reduction Techniques for MUSE.
MUSE, a 2nd generation VLT instrument, will become the world’s largest integral field spectrograph. It will be an AO assisted instrument which, in a single exposure, covers the wavelength range from 465 to 930 nm with an average resolution of 3000 over a field of view of 1’x1’ with 0.2’’ spatial sampling. Both the complexity and the rate of the data are a challenge for the data processing of this instrument. We will give an overview of the data processing scheme that has been designed for MUSE. Specifically, we will use only a single resampling step from the raw data to the reduced data product. This allows us to improve data quality, accurately propagate variance, and minimize spreading of artifacts and correlated noise. This approach necessitates changes to the standard way in which reduction steps like wavelength calibration and sky subtraction are carried out, but can be expanded to include combination of multiple exposures.
MUSE is planned to be a giant integral field spectrograph and going to be commissioned as a second generation VLT instrument starting in 2012. Its 1 ′ ×1 ′ field of view is sampled at 0. ′′ 2 resolution in wide-field mode. In the adaptive-optics supported mode, observing in a narrow-field mode with sampling at 0. ′′ 75 is possible. Each exposure delivers about 90000 spectra, using 24 slicer integral field units (IFUs), each equipped with a separate 4k 2 CCD. The full wavelength range from 465 to 930 nm is covered with resolution between 2000 and 4000.
The light at the VLT focal plane is divided into 24 sub-fields, each of which has a size of 60 ′′ ×2. ′′ 5 on the sky. Within each IFU, the sub-field is then further decomposed into 48 slices. Each slice has a size of 15 ′′ ×∼0. ′′ 2 on the sky. The spectrograph in each IFU disperses the light and focuses the spectral pattern on a 4k×4k CCD. This means that the data appears on the CCDs as 48 strips of spectra which are 75 px wide and 4096 px high. In between these strips of data, dark regions of ∼6 px width can be used to identify stray-light.
Each raw exposure has a size of roughly 800 MB; by using the full capabilities of the Euro3D format, a processed exposure can require about 4.5 GB of 2 Weilbacher et al.
storage. Typical observing nights will have 50 GB of raw data per night (calibrations plus science exposures), but the data rate may be as high as 150 GB per night. Each exposure will contain approximately 350 million illuminated pixels.
The main goals of the MUSE data processing pipeline are to fully reduce all exposures into ready-to-use datacubes (Euro3D format or FITS with NAXIS=3) without creating a backlog (a requirement on scope of the reduction and the speed), to track bad pixels, propagate error information, and to minimize the rebinning steps. Especially the last two goals are connected and not yet commonly implemented in existing data reduction systems. We therefore discuss the details in the following sections.
We take the approach of Gössl & Riffeiser (2002) and estimate the variance from the raw data:
The terms describe the read-out noise, the error of the read-out noise estimate, and the photon noise of each pixel. This variance is then tracked through all processing steps using Gaussian error propagation. This approach assumes that the CCD pixels are uncorrelated which should be true to first order. During data analysis, the variance, together with the data value itself, makes it possible to directly read off the S/N estimate at each position in the output datacube.
If we followed the classical approach to determine a calibration function for one type of distortion and applied it (requiring resampling), before determining the next one, MUSE data would have to be resampled four to five times. Every resampling step always adds noise to a dataset. To be able to display and analyze data one always needs a rectangular format, so at least one resampling step has to be carried out. Resampling more than once complicates the noise propagation, passing it through five steps is near impossible.
The approach we use for the one-step resampling is centered around what we call a “pixel table” (Davies 2007 describes a very similar approach that was developed for the KMOS instrument). This table contains a list of all illuminated pixels (the data value, the bad pixel status, and the variance) of one exposure together with the coordinates (x, y, and λ). As soon as first estimates of the three coordinates are known, i. e. following tracing and wavelength calibration, the CCD-based data is transfered into such a pixel table.
Once the data are listed in the pixel table, each transformation that would classically require an interpolation step can then be carried out just by changing the coordinates of each table entry. Shifting the data (such as done when combining multiple exposures) just adds a constant to the column representing the spatial axis; tilting it (as needed to correct for differential atmospheric refraction) applies a function to both spatial axes. Astrometric calibration can be carried out by replacing pixel values in the two spatial axes with values of RA and DEC (using a function describing an astrometric solution).
To determine the function that needs to be applied to the dataset for a given transformation one in most cases needs to resample the data already into a datacube. An example is the flux calibration where the standard star exposure needs to be concerted to a datacube before one can measure the flux for each wavelength to derive the response function. But as the intermediate datacubes are generated of auxiliary data and are not used after the measurement, this does not affect the data quality of the science data. Additionally, for some operations it is possible to generate the temporary datacube with coarse resolution, and should therefore not have a large impact on the speed of the operation when compared to the classical
…(Full text truncated)…
This content is AI-processed based on ArXiv data.
