Data_Reduction/Imaging_Data_Reduction の履歴(No.18) - SWIMS : Simultaneous-color Wide-field Infrared Multi-object Spectrograph for TAO 6.5m Telescope

Data Reduction Pipeline for Imaging Data†

Overview†

SWIMS Imaging Data Reduction Pipeline, written in Python, processes SWSB*/SWSR* FITS data to produce a final stacked image in a standard manner.

Reduction Flow Chart

↑

Download†

↑

Core scripts†

File	Contents
swsred_20210423a.tgz	Python core scripts (swsred/*py) and configuration/utility files

↑

BPM†

File	Contents
swsred_calib_bpm_s18a_20210131.tgz	Bad pixel mask files for S18A data
swsred_calib_bpm_s18b_20210131.tgz	Bad pixel mask files for S18B data
swsred_calib_bpm_s20b_20210131.tgz	Bad pixel mask files for S20B data
swsred_calib_bpm_s21a_20210418.tgz	Bad pixel mask files for S21A data

↑

Mosaic†

File	Contents
swsred_calib_mosaic_s18a_20210131.tgz	Mosaic parameter files for S18A data
swsred_calib_mosaic_s18b_20210131.tgz	Mosaic parameter files for S18B data
swsred_calib_mosaic_s21a_20210413.tgz	Tentative use only!! Mosaic parameter files for S21A data

↑

Flat†

BBF: Broad-band filters, MBF: Medium-band filters, NBF: Narrow-band filters


File	Contents
swsred_calib_flat_bbf_s18a_20210131.tgz	Flat files for S18A BBF data
swsred_calib_flat_bbf_s18b_20210131.tgz	Flat files for S18B BBF data
swsred_calib_flat_mbf_s18b_20210131.tgz	Flat files for S18B MBF data
swsred_calib_flat_nbf_blue_s18b_20210131.tgz	Flat files for S18B Blue NBF data
swsred_calib_flat_nbf_red_s18b_20210131.tgz	Flat files for S18B Red NBF data

swsred_calib_flat_bbf_s20b_20210131.tgz	Flat files for S20B BBF data

swsred_calib_flat_bbf_s21a_20210418.tgz	Flat files for S21A BBF data
swsred_calib_flat_mbf_s21a_20210418.tgz	Flat files for S21A MBF data

↑

Sample data†

Use these test data (taken in S18B) to check if your swsred works.

Object: FS12
Dithering: 4-point
Filter: J1-band and H1-band

File	Contents
swsred_sample_b.tgz	J1-band data (4 frames x 2 arrays)
swsred_sample_r.tgz	H1-band data (4 frames x 2 arrays)

The reduced images produced with two iterations (n_iter=2) should be like these.

File	Contents
fs12_j1_b1_stack1.fits	Stacked image (after two iterations) on the blue array #1
fs12_j1_b2_stack1.fits	Stacked image (after two iterations) on the blue array #2
fs12_h1_r1_stack1.fits	Stacked image (after two iterations) on the red array #1
fs12_h1_r2_stack1.fits	Stacked image (after two iterations) on the redarray #2
fs12_j1.fits	Mosaicked image (after two iterations) on the blue arrays Mosaicking parameters still under verification. Use this image (and mosaicking methods) only to check if swsred works correcctly.
fs12_h1.fits	Mosaicked image (after two iterations) on the red arrays Mosaicking parameters still under verification. Use this image (and mosaicking methods) only to check if swsred works correctly.

↑

Installation†

↑

prerequisites†

SWSRED requires the following python packages and astrOmatic softwares.

Package	Tested version
Python	2.7.17	3.7.7
astropy	2.0.9	4.0
numpy	1.16.6	1.18.1
scipy	1.2.1	1.4.1
matplotlib	2.2.3	3.1.3
sextractor	2.19.5	2.19.5
scamp	2.7.8	2.7.8
swarp	2.38.0	2.38.0

Also, Internet connection is required during reducing data to access Pan-STARRS/2MASS catalogs.

↑

Install SWSRED†

Download SWSRED files you need.

Extract all the tar balls on a directory you want.

% cd SWSRED_DIRECTORY
% tar xvf swsred_20210414c.tgz
% tar xvf swsred_calib_bpm_s21a_20210418.tgz
% tar xvf swsred_calib_mosaic_s21a_20210413.tgz
% tar xvf swsred_calib_flat_bbf_s21a_20210418.tgz
% tar xvf swsred_calib_flat_mbf_s21a_20210418.tgz

set PYTHONPATH to SWSRED_DIRECTORY.

% export PYTHONPATH="SWSRED_DIRECTORY:${PYTHONPATH}

↑

Usage†

SWSRED has no documentations yet, but you can use a dirty script 'swsred/reduce_all.py' as a quick tutorial to learn how to use the software.

Be sure to use SWSRED within python interpreter (e.g., in ipython or jupyter, etc.), not via the command line as no command-line interfaces prepared.

Here is a quick introduction to use SWSRED.

First of all, prepare a list of input FITS files including absolute paths as a python 'array-like' (list, tuple, or numpy array) object, for example as follows.

> import glob
> in_arr1 = glob.glob("DATA/PATH/SWSB*1.fits")  # blue chip1 data 
> in_arr1.sort()  # must be sorted before using.

Note that as shown in the flow chart above, data of chip1 and chip2 should be reduced separately (before stacking).

Data taken within the same dithering set are grouped by '2_DSETID' header keyword. Using this, you can make a input list like follows,

import glob
import astropy.io.fits

dct1 = {}
dct2 = {}

infiles = glob.glob("SWS*fits")
infiles.sort()

for infile in infiles:
    hdr = astropy.io.fits.getheader(infile)

    ch = hdr["2_DETCH"]
    if ch == 1:
        dct = dct1
    else:
        dct = dct2

    if hdr["2_DSETID"] not in dct.keys():
        dct[hdr["2_DSETID"]] = []

    dct[hdr["2_DSETID"]].append(infile)

Make filenames of intermediate files.

> import swsred.make_file_lst
> file_lst1 = swsred.make_file_lst.make_file_lst(in_arr1)

'file_lst1' is a dictionary which defines various filenames such as flat-fielded FITS filename. You can edit any values as you want.

For example, you get the following result with an input ["data/path/SWSB00000001.fits"] as in_arr1,

Key	Value	Description
in	data/path/SWSB00000001.fits	input filename
ff	ffSWSB00000001.fits	filename of flat-fielded data
ms	msSWSB00000001.fits	filename of self-sky data
ss	ssSWSB00000001.fits	filename of sky-subtracted data
wc	wcSWSB00000001.fits	filename of WCS-corrected data
bpm	/SWSRED_DIRECTORY/calib/bpm_sXXY_b1.fits	filename of Bad-pixel mask (XXY denotes a semester)
wht	SWSB00000001_wht.fits	filename of weight map (created by make_weight_map module)
omsk	SWSB00000001_omsk.fits	filename of object mask
ldac	ssSWSB00000001.ldac	filename of SExtractor LDAC catalog
ascii	ssSWSB00000001.ascii	filename of SExtractor ASCII catalog
head	ssSWSB00000001.head	filename of SExtractor HEAD file

Then, start your data reduction. For example, to perform flat-fielding,
```
> import swsred.flat_field
> swsred.flat_field.flat_field(file_lst1["in"], file_lst1["ff"])
```
- By default, the values 'file_lst1["ff"]' do not have a path which means that 'file_lst1["ff"]' files will be created on the current directory. If you want to specify any directory to put intermediate files, edit and add your path in 'file_lst1'.

Refer to 'swsred/reduce_all.py' for all the procedures.

↑

Version 2021.01.31†

First release

↑

Contact†

Please send your bug reports or questions to konishi__at__ioa.s.u-tokyo.ac.jp.

SWIMS Wiki

Imaging_Data_Reduction の履歴(No.18)