This doc describes the simulation of SKA observed images, which are used to demonstrate the proposed EoR signal separation method based on the CDAE and evaluate its performance.
Two sets of simulated image cubes with realistic instrumental effects integrated are provided for download:
- EoR signal (1)
(md5:
bbbd03884960b967c56f05cfc6eba5ff) - EoR signal (2)
(md5:
4ae28b8820765c2100da0b06fae9cbbc) - Foreground emission (1)
(md5:
634ca459edaf3fda6b861c0463850c76) - Foreground emission (2)
(md5:
f7bb9ec96f3455e486dc09c7b8225165)
Each file is about 50 MiB in size. All the image cubes cover a sky patch of size 2x2 deg^2 with a pixel size of 20 arcsec and span a frequency band of 154-162 MHz with channel width of 80 kHz. The size of image cubes is thus 360x360x101.
Also used in the paper for comparison, the ideal skymaps (i.e., for which the observational simulations are performed) are also provided:
- EoR signal (skymap)
(md5:
7446c159d8c0b6964a9eead881db3995) - Foreground emission (skymap)
(md5:
913ef156017749785e24e21c146d201f)
We use the FG21sim software that we developed based on our previous works (Wang et al. 2010, 2013) but with significant improvements especially to the simulation of radio halos.
Follow the instructions there to install the FG21sim software and
obtain the necessary template maps.
-
Get the template data (see this data page for more details):
-
Extract the template patch from the all-sky HEALPix template by using
get-healpix-patch:$ get-healpix-patch --smooth --center 0,-27 \ --size 1800,1800 --pixelsize 20 \ haslam408_dsds_Remazeilles2014_ns512.fits \ haslam408_eor0_fov10.fits $ get-healpix-patch --smooth --center 0,-27 \ --size 1800,1800 --pixelsize 20 \ synchrotron_specind2_ns512.fits \ specindex_eor0_fov10.fits -
Create configuration file (
gsyn.conf):[foregrounds] galactic/synchrotron = True [sky] type = "patch" [[patch]] # ra, dec [deg] xcenter = 0.0 ycenter = -27.0 # patch image size xsize = 1800 ysize = 1800 # pixel size [arcsec] pixelsize = 20 [frequency] type = "calc" step = 0.08 start = 154.0 stop = 162.0 [galactic] [[synchrotron]] template = "haslam408_eor0_fov10.fits" template_freq = 408.0 indexmap = "specindex_eor0_fov10.fits" add_smallscales = False prefix = "gsyn" output_dir = "output"
-
Do the simulation:
$ fg21sim gsyn.conf
The simulated sky maps are named gsyn_<freq>.fits.
-
Get the template data (see this data page for more details):
-
Extract the template patch from the all-sky HEALPix template:
$ get-healpix-patch --smooth --center 0,-27 \ --size 1800,1800 --pixelsize 20 \ Halpha_fwhm06_ns1024.fits \ halpha_eor0_fov10.fits $ get-healpix-patch --smooth --center 0,-27 \ --size 1800,1800 --pixelsize 20 \ SFD_i100_ns1024.fits \ SFD_eor0_fov10.fits -
Create configuration file (
gff.conf):[foregrounds] galactic/freefree = True [sky] type = "patch" [[patch]] # ra, dec [deg] xcenter = 0.0 ycenter = -27.0 # patch image size xsize = 1800 ysize = 1800 # pixel size [arcsec] pixelsize = 20 [frequency] type = "calc" step = 0.08 start = 154.0 stop = 162.0 [galactic] [[freefree]] halphamap = "halpha_eor0_fov10.fits" dustmap = "SFD_eor0_fov10.fits" prefix = "gff" output_dir = "output"
-
Do the simulation:
$ fg21sim gff.conf
The simulated sky maps are named gff_<freq>.fits.
-
Create configuration file (
halos.conf):[foregrounds] extragalactic/clusters = True [sky] type = "patch" [[patch]] # ra, dec [deg] xcenter = 0.0 ycenter = -27.0 # patch image size xsize = 1800 ysize = 1800 # pixel size [arcsec] pixelsize = 20 [frequency] type = "calc" step = 0.08 start = 154.0 stop = 162.0 [extragalactic] [[psformalism]] dndlnm_outfile = "dndlnm.npz" [[clusters]] catalog_outfile = "catalog.csv" halos_catalog_outfile = "halos.csv" halo_dropout = 0 mass_min = 2e14 prefix = "halos" output_dir = "output"
-
Do the simulation:
$ fg21sim halos.conf
The simulated sky maps are named halos_<freq>.fits.
This foreground component is simulated by using the radio-fg-simu-tools/pointsource tool. Refer to there for more details to build the tool.
-
Place all the Wilman2008 simulation data at a directory (
wilman2008_db) and generate the list:$ ls wilman2008_db/*.txt > wilman2008_db.list
-
Generate the list of frequencies using the
freq2z.pytool:$ freq2z.py 154:0.08:162 | awk '/^1/ { print $1 }' > freqs.list
-
Perform the simulation of point sources:
$ make_ptr_map -o ptr_ -O ptr_sources.csv \ -i wilman2008_db.list \ $(cat freqs.list)
The simulated sky maps of point sources are named ptr_<freq>.fits.
The maps of the EoR signal are created by using the data of the faint galaxies simulation case released by the Evolution Of 21 cm Structure project.
-
Get the light travel cube in redshift range of 5-9.568, i.e., the file named
delta_T_v3_no_halos__zstart005.00000_zend009.56801_FLIPBOXES0_1024_1600Mpc_lighttravel. -
Convert the cube from 21cmFAST binary format to FITS format by using
21cmfast_lightcone.py:$ 21cmfast_lightcone.py --z-min 5 --z-max 9.56801 \ delta_T_v3_no_halos__zstart005.00000_zend009.56801_FLIPBOXES0_1024_1600Mpc_lighttravel \ Tb_lightcone_N1024_L1600_z05.000-09.568.fits -
Extract slices at requested frequencies (i.e., redshifts) by using
get_slice_zfreq.py:$ mkdir deltaTb_slices && cd deltaTb_slices $ get_slice_zfreq.py \ -i Tb_lightcone_N1024_L1600_z05.000-09.568.fits -f $(cat freqs.list)
The extracted slices are named as
deltaTb_f<freq>_z<redshift>.fits. -
Tile the slices to match the FoV and pixel size by using
tile_slice.py:$ mkdir deltaTb_tiled && cd deltaTb_tiled $ for slice in ../deltaTb_slices/*.fits; do \ tile_slice.py -f 158 -F 10.0 -N 1800 -i ${slice}; \ done
By employing the latest SKA1-Low layout configuration (released on 2016 May 21), the OSKAR simulator is used to perform the observational simulation to generate the visibility data for each sky map. Then the visibility data is imaged by the WSClean imager to create the simulated SKA images.
-
Generate the telescope model for the OSKAR simulator by using
make-ska1low-model:$ make-ska1low-model -o ska1low.tm
Or use the pre-generated one by me at atoolbox/astro/oskar/telescopes/ska1low.tm.
-
Convert the sky maps into sky models for OSKAR by using
fits2skymodel.py:$ for f in <skymaps-dir>/*<freq>.fits; do \ fits2skymodel.py -o skymodel ${f}; \ outfile=skymodel/$(basename ${f%.fits}.osm); \ echo "<freq> ${outfile}" >> skymodel.list; \ done
(NOTE: replace the contents of
<???>above to match your own case. The same substitution rule applies to some of the commands below as well.)For the Galactic emission, the sky maps of synchrotron and free-free emissions are combined before converting to the OSKAR sky models.
As for the extragalactic point sources, the bright sources with a 158 MHz flux density greater than 10 mJy are removed first, which corresponds to pixels with value greater than ~1400 K are removed.
$ fits2skymodel.py --pixel-size 20 --max-value 1400 \ -f 158 -o skymodel --create-mask mask.fits \ <skymaps-dir>/ptr_158.00.fits $ rm skymodel/ptr_158.00.osm $ for f in <skymaps-dir>/*<freq>.fits; do \ fits2skymodel.py --pixel-size 20 -f <freq> \ --mask mask.fits -o skymodel ${f}; \ outfile=skymodel/$(basename ${f%.fits}.osm); \ echo "<freq> ${outfile}" >> skymodel.list; \ done -
Prepare the base OSKAR configuration file (
oskar.ini):[General] app=oskar_sim_interferometer [simulator] max_sources_per_chunk=524288 double_precision=false use_gpus=true keep_log_file=true write_status_to_log_file=true [sky] advanced/apply_horizon_clip=false oskar_sky_model/file= [telescope] aperture_array/array_pattern/enable=true aperture_array/element_pattern/dipole_length=0.5 input_directory=ska1low.tm aperture_array/element_pattern/dipole_length_units=Wavelengths aperture_array/element_pattern/functional_type=Dipole pol_mode=Scalar normalise_beams_at_phase_centre=true allow_station_beam_duplication=true station_type=Aperture array [observation] phase_centre_ra_deg=0.0 phase_centre_dec_deg=-27.0 start_time_utc=2000-01-01T06:30:00.000 length=21600.0 num_time_steps=144 num_channels=1 start_frequency_hz= [interferometer] uv_filter_max=max time_average_sec=150.0 uv_filter_units=Wavelengths channel_bandwidth_hz=80000.0 uv_filter_min=min oskar_vis_filename= ms_filename=
-
Do observational simulations by using
run_oskar.py:$ run_oskar.py -c oskar.ini -l skymodel.list -o visibility
-
Create images by using
wsclean.py, which wraps on WSClean to be easier to use:$ mkdir images && cd images $ wsclean.py --threshold <threshold> --weight natural \ --uv-range 30:1000 --size 1800 --pixelsize 20 \ --circular-beam --fit-spec-order 2 \ --name <name_prefix> \ --ms ../visibility/*.ms
-
Convert image units from [Jy/beam] to [K] by using
jybeam2k.py:# Get the average beam size of the images among the frequency band $ beamsize *-????-image.fits $ for f in *-????-image.fits; do \ jybeam2k.py -b <beamsize_avg> ${f} ${f%.fits}K.fits; \ done
-
Create image cube and crop by using
fitscube.py:$ fitscube.py create -z 154e6 -s 80e3 -u Hz \ -o <prefix>_cube.fits -i images/*-imageK.fits $ fitscube.py crop -n 360 \ -i <prefix>_cube.fits \ -o <prefix>_n360_cube.fits -
Add foreground components together:
$ fitscube.py add \ -o fg_n360_cube.fits -i <prefix1>_cube.fits ... \