Imaging_Data_Reduction の履歴(No.34)

• 履歴一覧
• 差分 を表示
• 現在との差分 を表示
• ソース を表示
• Data_Reduction/Imaging_Data_Reduction へ行く。
  • 1 (2021-01-27 (水) 16:36:03)
  • 2 (2021-01-27 (水) 17:07:41)
  • 3 (2021-01-28 (木) 10:18:44)
  • 4 (2021-01-29 (金) 11:38:39)
  • 5 (2021-02-01 (月) 11:18:40)
  • 6 (2021-02-01 (月) 11:18:40)
  • 7 (2021-02-01 (月) 11:18:40)
  • 8 (2021-02-01 (月) 11:18:40)
  • 9 (2021-03-27 (土) 00:28:13)
  • 10 (2021-03-27 (土) 08:29:31)
  • 11 (2021-04-10 (土) 16:37:57)
  • 12 (2021-04-13 (火) 23:26:07)
  • 13 (2021-04-14 (水) 10:11:34)
  • 14 (2021-04-14 (水) 20:51:23)
  • 15 (2021-04-14 (水) 23:17:29)
  • 16 (2021-04-18 (日) 19:29:48)
  • 17 (2021-04-20 (火) 08:30:18)
  • 18 (2021-04-23 (金) 19:02:24)
  • 19 (2021-05-17 (月) 15:45:45)
  • 20 (2021-05-26 (水) 20:44:46)
  • 21 (2021-05-27 (木) 20:26:04)
  • 22 (2021-05-27 (木) 22:39:34)
  • 23 (2021-06-10 (木) 11:20:23)
  • 24 (2021-07-03 (土) 13:16:56)
  • 25 (2021-07-10 (土) 21:55:10)
  • 26 (2021-07-11 (日) 15:24:00)
  • 27 (2021-07-13 (火) 12:10:23)
  • 28 (2021-07-16 (金) 10:55:58)
  • 29 (2021-07-18 (日) 13:40:57)
  • 30 (2021-07-18 (日) 19:15:18)
  • 31 (2021-07-20 (火) 19:31:21)
  • 32 (2021-07-20 (火) 19:31:21)
  • 33 (2021-09-01 (水) 17:01:43)
  • 34 (2022-02-21 (月) 14:57:19)
  • 35 (2022-03-11 (金) 12:40:36)
  • 36 (2022-04-03 (日) 01:16:44)
  • 37 (2022-04-06 (水) 20:24:04)

---

Data Reduction Pipeline for Imaging Data†

Overview†

SWIMS Imaging Data Reduction Pipeline, written in Python, processes SWSB*/SWSR* FITS data to produce a final stacked image in a standard manner.

Download†

Core scripts†

BPM†

Put files that you need on swsred/calib/bpm/ directory.

Imaging Flat†

Put files that you need on swsred/calib/flat/ directory.

Mosaic†

Usually, mosaicking is performed (using swarp) based on WCS information.
These files are needed only when your data do not have enough stars for astrometry.

Sample data†

Use these test data (taken in S18B) to check if your swsred works.

• Object: FS12
• Dithering: 4-point
• Filter: J1-band and H1-band

Here is a code to reduce the data. --> sample recipe

The reduced images produced with two iterations (n_iter=2) using swsred version 20210901 and the execution log (swsred.log) should be like these.

Installation†

prerequisites†

SWSRED requires the following python packages and astrOmatic softwares.

The astrOmatic softwares can be installed as follows if you have a anaconda environment.

% conda config --append channels conda-forge
% conda install sextractor
% conda install -c conda-forge astromatic-scamp
% conda install -c conda-forge astromatic-swarp

Also, Internet connection is required during reducing data to access Pan-STARRS/2MASS catalogs.

Install SWSRED†

1. Download SWSRED files you need.
2. Extract all the tar balls on a directory you want.

   % cd SWSRED_DIRECTORY
   % tar xvf swsred_20210527.tgz
   % tar xvf swsred_calib_mosaic_s21a_20210413.tgz
   % mkdir -p swsred/calib/bpm swsred/calib/flat
   

3. set PYTHONPATH to SWSRED_DIRECTORY.

   % export PYTHONPATH="SWSRED_DIRECTORY:${PYTHONPATH}
   

Usage†

SWSRED has no documentations yet, but you can use a dirty script 'swsred/reduce_all.py' as a quick tutorial to learn how to use the software.

Be sure to use SWSRED within python interpreter (e.g., in ipython or jupyter, etc.), not via the command line as no command-line interfaces prepared.

Here is a quick introduction to use SWSRED.

1. First of all, prepare a list of input FITS files including absolute paths as a python 'array-like' (list, tuple, or numpy array) object, for example as follows.

   > import glob
   > in_arr1 = glob.glob("DATA/PATH/SWSB*1.fits")  # blue chip1 data 
   > in_arr1.sort()  # must be sorted before using.
   

   • Note that as shown in the flow chart above, data of chip1 and chip2 should be reduced separately (before stacking together).
   • Data taken within the same dithering set are grouped by '2_DSETID' header keyword. Using this, you can make a input list like follows,

     import glob
     import astropy.io.fits
     
     dct1 = {}
     dct2 = {}
     
     infiles = glob.glob("SWS*fits")
     infiles.sort()
     
     for infile in infiles:
         hdr = astropy.io.fits.getheader(infile)
     
         ch = hdr["2_DETCH"]
         if ch == 1:
             dct = dct1
         else:
             dct = dct2
     
         if hdr["2_DSETID"] not in dct.keys():
             dct[hdr["2_DSETID"]] = []
     
         dct[hdr["2_DSETID"]].append(infile)
     

2. Make filenames of intermediate files.

   > import swsred.make_file_lst
   > file_lst1 = swsred.make_file_lst.make_file_lst(in_arr1)
   

   • 'file_lst1' is a dictionary which defines various filenames such as flat-fielded FITS filename. You can edit any values as you want.
     • For example, you get the following result with an input ["data/path/SWSB00000001.fits"] as in_arr1,

       Key	Value	Description
       in	data/path/SWSB00000001.fits	input filename
       ff	ffSWSB00000001.fits	filename of flat-fielded data
       ms	msSWSB00000001.fits	filename of self-sky data
       ss	ssSWSB00000001.fits	filename of sky-subtracted data
       wc	wcSWSB00000001.fits	filename of WCS-corrected data
       bpm	/SWSRED_DIRECTORY/calib/bpm/bpm_sXXY_b1.fits	filename of Bad-pixel mask (XXY denotes a semester)
       wht	SWSB00000001_wht.fits	filename of weight map (created by make_weight_map module)
       omsk	SWSB00000001_omsk.fits	filename of object mask
       ldac	ssSWSB00000001.ldac	filename of SExtractor LDAC catalog
       ascii	ssSWSB00000001.ascii	filename of SExtractor ASCII catalog
       head	ssSWSB00000001.head	filename of Scamp HEAD file

1. Then, start your data reduction. For example, to perform flat-fielding,

   > import swsred.flat_field
   > swsred.flat_field.flat_field(file_lst1["in"], file_lst1["ff"])
   

   • By default, the values 'file_lst1["ff"]' do not have a path which means that 'file_lst1["ff"]' files will be created on the current directory. If you want to specify any directory to put intermediate files, edit and add your path in 'file_lst1'.

Refer to swsred/reduce_all.py for all the procedures.

Questions & Answers†

1. Astrometry (performed by scamp) is failed or is not good. How do I work for it?
   • Try changing the following parameters,
     • detect_thresh_wcs: detection threshold to make a source catalog used in scamp.
     • mag_max_wcs: maximum magnitude of Pan-STARRS sources to be used in scamp.

Known bugs†

Changes†

Version 2021.01.31†

• First release

Contact†

Please send your bug reports or questions to konishi__at__ioa.s.u-tokyo.ac.jp.