MOS data reduction based on IRAF

MOS data reduction

Outline

This page describes how to use FOCAS MOS reduction software consisting of IRAF scripts and C programs.
Note that the interface is always IRAF task. C programs are called from IRAF tasks.

### NOTE1 ###
It is assumed the data is taken with Hamamatsu FDCCDs.
Reduction of MOS data taken with the former MIT CCDs is still to be tested and not yet supported.

### NOTE2 ###
This software corrects the instrument distortion, extract spectra, and then calibrate for wavelength.
Therefore, there will be sub-pixel interpolation twice (distortion-correction and wavelength-calibration).
This may result in enhanced correlation of noise between neibouring pixels.

The software contains the following tasks.

#	procedure	task(s)	language
1	overscan subtraction	ovsub_hpk ovsub_hpk_med	IRAF
2	distortion correction and combine chip 1 and 2	bigimage	IRAF, C
3	extract spectra	speccomb mosextract	IRAF, C
4	creating comparison frame	mkcomparison	IRAF
5	identification of comparison/night-sky lines	mosident	IRAF

Setting up the Software

Download and extract the latest FOCASRED package.

See "IRAF setup" and "Compilation" in FOCASRED/README for the instruction to set up the software.

Example

Here is an example of commands for a dataset.

Overscan Subtraction

 focasred> ovsub_hpk @list.in @list.out

This task subtracts bias-level by using overscan regions in each channel.
The CCD chip number and binning information in FITS header are used to set the overscan region.
In the output image, overscan regions are removed and the pixel counts were converted to electrons.

Note that there is a bias pattern even after this process and it should be removed by subtracting the standard bias frame which is also overscan-subtracted.

Sometimes, a bright star located at the boundary of two channels can affect the overscan region(s) and result in over-subtraction of bias level.
If you notice such effect, please try ovsub_hpk_med instead of ovsub_hpk.

Flat Fielding

Because sub-pixel interpolation will be made in the following procedure, bigimage, flat-fielding should be done before it.
Therefore, normalization of the flat-frame can be done only with a constant value.
The current software does not support any other (more sophisticated) way, such as polynomial-fitting, to normalize the flat-frame.

Note that, for MOS data taken with a VPH grism, the efficiency curve depends on the position of the slit and the normalization of flat-frame should be done with a constant value, anyway.

Distortion Correction and Combine Chip 1 and 2

 focasred> bigimage chip1.fits chip2.fits output.fits obsmode=spec

This task corrects for the instrument distortion and combines the two CCD images.
Make sure the option obsmode is set to spectroscopy, not imaging.
The following task for MOS extraction can only be used after this process.

Please make sure to use the correct distortion file before executing the task bigimage (option "distdata").

 focasred> epar bigimage

 focasred> lpar bigimage

Jun. 2010 - Jun. 2011	: distdata=DistChipDat.HP.062910.dat
Oct. 2012 -	: distdata=DistChipDat.HP.101112.dat

Image Registration

A spectroscopic observation usually takes multiple exposures to deal with cosmic-rays, CCD defects, and so on.
Due to the instrument flexure, there will be shifts between those exposures.
Before extracting spectra, it is recommended to register (correct for those shifts) the images.

If you use Arc frame for wavelength calibration and it is taken before/between/after the science exposures, they should be registered to the exposure closest to the Arc frame.
If you use night-sky lines for wavelength calibration, you do not necessarily have to register exposures.
However, to reduce the time for wavelength calibration which is done interactively and takes long time, it is recommended to register exposures first so that identification of night-sky lines for one exposure can be applied to all the other exposures.

An IRAF task images.immatch.xregister can be used for this purpose. See the example text above.

Extract Spectra

 focasred> mosextract input.fits input.sbr

You need the mask design file, input.sbr, for this task.
The .sbr file is the output of the mask design software, wmdp and sent to the observatory for mask cutting before the observation.
If you do not have it, check the FITS header parameter "SLIT" (the value should be like 'SCFCMS0123') and ask SS for the sbr file.
As for bigimage, please make sure to use the correct distortion file (see above).

There are several hidden parameters for this task.

parameter name	description	more detailed information
distdata	distortion data file	the file is in the directory `FOCASRED/DATA/`
rot	MOS rotation in degree	During setupfield of MOS observation, the MOS masks is rotated to match the angle of the mask and the sky. If it is larger than 0.1 degree, this option should be used. This value is not recorded in FITS header and you need to check the observation log. Please consult SS if you don't have this information.
dx	shift for slit positions in pixel	This task creates a file named input.fits.reg. Open the input.fits image with DS9 and then load the .reg file. The positions of the slits will be shown. If you find a systematic shift between the actual and calculated positions of the slits, use this parameter to refine the positions of extraction.
mdpfile	optional mdp file for comments	If you specify a .mdp file with this option, the comments will be stored in the FITS headers of extracted spectra. Note that the .mdp file must have the same number of targets as the .sbr file.

This task also creates an IRAF script, input.fits.cl, which can be used to check what are actually done.
It is just an iteration of imcopy and hedit.
The output fits files are named as input.001.fits, input.002.fits, and so on. The number corresponds to the row number of the input .sbr file.
There will be skipped numbers which are alignment holes and not extracted.

This is an example of using the .reg file on DS9.

configuration files

The following files are used in this task.

FOCASRED/DATA/barcode.dat	The information of grism and filter are stored in the FITS header. Because they are in barcode format, this file is used to convert the barcode to its actual name. (the name is only used within the task and the values of FITS header are not changed)
FOCASRED/DATA/specmode.dat	This file contains the nominal values for wavelength parameters such as central wavelength, dispersion, and wavelength-range. They are selected according to the used grism and filter(s). If you want to change the wavelength range of extraction, you need to edit this file.

speccomb

There is another task speccomb for quickly checking MOS data.
The parameters are the same as mosextract except for the name of output fits file.

 focasred> speccomb input.fits input.sbr output.fits

It has the same hidden parameters as mosextract.
After extracting the spectra, this task combines them to one fits file with approximate wavelength information.
See below for the example.

This is an example of speccomb output.

Create Comparison Frame

This is an optional procedure.
If you have two arc frames for red and blue wavelength range, you need to combine them to create a comparison frame.
If you use night-sky lines or one arc frame, please skip this procedure.

 focasred> mkcomparison @list.blue @list.red @list.out

The parameters list.blue and list.red are the lists of input files which are extracted from arc-frame for blue and red, respectively.
The task uses the approximate wavelength information in FITS header written by the task mosextract.
In default case, it uses list.red for >6900A and list.blue for <6900A.
and joins the two spectra. The threshold wavelength can be changed by the hidden paramter.

Identification of comparison/night-sky lines

This is one of the most difficult and time-consuming part of MOS data reduction.
The task mosident uses the IRAF task noao.onedspec.autoidentify combined with the approximate wavelength information from mosextract.

 focasred> mosident @list.comp linedata="thar.300.dat" order=5

It tries to identify the lines automatically but you have to check all of the identification visually and make correction if necessary.
Because of severe blending of lines in blue wavelength region, the identifications are usually incomplete and need correction.

The following table shows the recommended paramters for several spectroscopic settings.
The typical rms can be used to check the goodness of your identification/fitting.

Grism	Filter	Spectrum	Line Data	Fitting Order	Typical RMS (with 0".8 slit)
300R/300B	O58/Y47	arc (ThAr)	thar.300.dat	5	0.2
300R/300B	O58/Y47	night sky	skyline.dat	4	0.2-0.3
300B	none/L600	arc (ThAr)	thar.300.dat	5	0.2-0.3
VPH520	none/L600	arc (ThAr)	thar.vphblue.dat	4	0.2
VPH850	O58	night sky	skyline.dat	4	0.2-0.3
VPH900	O58	night sky	skyline.dat	4	0.1

Step by step

In this example, data taken with VPH520+L600 is used.

First, make the irafterm window wider, e.g., 2 times wider than the original, so that you can see the arc lines and their identifications more clearly.

Second, check an example plot of identify in FOCASRED/IRAF/comparison/.
In the wavelength range of VPH520, important lines are 5760, 5720, 4764, and 4545Å.
They are relatively strong and isolated lines, and can also be used in lower dispersion grism such as 300B.
You should be able to identify those lines by eyes.
Note that the wavelength range of a slit depends on its position on the detector.
So, the lines you can see may be different from the example plot.

Run mosident.

 focasred> mosident @list.comp linedata="thar.vphblue.dat" order=4
 autoidentify comp.001 5250.00 -0.780 coordlist=/home/hattori/FOCASRED//IRAF/thar.vphblue.dat section=column 6 aidpars.crpix=1028.1 aidpars.aidord=5 nsum=5 niter=1 order=4 func=cheb fwidth=3.14 logfile=mosident.log inter- dbwrite=YES

 AUTOIDENTIFY: NOAO/IRAF V2.15.1a hattori@T7500 Mon 10:12:08 29-Apr-2013
   Spectrum                 # Found   Midpoint Dispersion        RMS
   comp.001[6,*]                 31    5254.84     -0.783      0.757
 identify comp.001[6,*]

If autoidentify finds a solution, identify will be called next and you will see a plot like below.

In this case, the RMS=0.757 (Å) is clearly worse than expected (see the table above).
Also, identifications at the right side including 5720 and 5760Å lines look wrong.

If autoidentify cannot find a solution, mosident calls reidentify and use the previous image as a reference.
Therefore, it is recommended to run mosident for all of the comparison data.
In the above example, list.comp is a list of all comparison data.

Push 'f' in the window to see the fitting plot.

In this plot, the ordinate is the residual from the polynomial fitting.
At the left side, there is a group of arc lines in a V-shaped pattern.
If fitting doesn't go well, a group of correctly identified lines show a systematic deviation like this.
On the other hand, the residuals at the right side are random with large scatter.
These are wrongly identified lines.
This is a typical behavior when identifications are only partly good.

Remove the points with red circles using 'd' and go back to the spectrum plot by pushing 'q' once.

At this stage, it will be a good idea to identify the 5760A line manually to help the fitting.

Then, push 'f' to refit.

Go back to the spectrum plot and push 'l' which tries to identify the lines listed in linedata=thar.vphblue.dat using the current fitting solution (dispersion function).

Check the new identifications by eye.

Refit and check the fitting results.

Now the RMS is about 0.2Å and you have a good fitting which spans the whole wavelength range of this spectrum.

Then, quit identify by pushing 'q' in the spectrum plot window.
You will be asked to save the data and answer 'yes' (or just type return-key).

 Write feature data to the database (yes)?

 REIDENTIFY: NOAO/IRAF V2.15.1a hattori@T7500 Mon 10:16:40 29-Apr-2013
   Reference image = comp.001, New image = comp.001, Refit = yes
           Image Data    Found     Fit Pix Shift  User Shift  Z Shift      RMS
 comp.001[1,*]          34/34   33/34     0.0415      -0.032  -6.6E-6    0.249
 comp.001[11,*]         34/34   33/34     -0.167       0.133  2.52E-5    0.265
 comp.001[16,*]         34/34   33/34     -0.168       0.132  2.58E-5    0.278
 comp.001[21,*]         34/34   32/34    -0.0689      0.0522  1.12E-5    0.261
 comp.001[26,*]         33/34   32/33     -0.153       0.122  2.33E-5    0.276
 comp.001[31,*]         32/33   30/32     -0.096      0.0769  1.45E-5    0.237
     ** Too many features lost **

mosident then calls reidentify to identify/fit the other columns. This part is automatically done but you should check the RMS values.
In mosident, reidentify is executed with nlost=2 (maximum number of line lost).
If it goes to 3 or over, the process will be stopped as the above example.
If you see such a situation, try reidentify manually.

 focasred> reident comp.001 comp.001 nlost=3 section="column 6" step=5 nsum=5

 REIDENTIFY: NOAO/IRAF V2.15.1a hattori@T7500 Mon 10:17:03 29-Apr-2013
   Reference image = comp.001, New image = comp.001, Refit = yes
           Image Data    Found     Fit Pix Shift  User Shift  Z Shift      RMS
 comp.001[36,*]         33/33   32/33     -0.148       0.119  2.23E-5    0.268
 comp.001[46,*]         33/33   32/33     -0.174       0.138  2.66E-5    0.222
 focasred>

Wavelength Calibration

The fittings in identify and reidentify are only in one dimension.
It is necessary to fit a two-dimensional function to the results obtained above using the IRAF task fitcoords.

 focasred> fitcoords @list.comp xorder=2 yorder=4

The orders should be the lowest one which does not show systematic deviation in the fitting.
Usually, xorder=2 is enough for 10-20 arcsec-long slits.
yorder should be same (or +1, maybe) as order in mosident.

To wavelength-calibrate the data, an IRAF task transform is used.
Because transform accept only one set of fitting results, you need to repeat executing the task for every slits; if you have 30 slits, you need to execute transform 30 times.
Instead, a task mostrans can be used.

mostrans @list.obj @list.obj.wc @list.comp y1=4000 y2=6050 dy=0.39

In this case, the values of y1, y2, dy (options for transform) are fixed so that all of the output spectra have an identical dispersion function (the relation between pixel coordinate and wavelength).
Originally witten by T. Hattori.