Inspecting the current status¶
Musered provides several sub-commands that allows to inspect the content of the database and the state of the reduction:
info: Print info about raw and reduced data.info-exp: Show reduction log for a night or exposure.info-raw: Print a table with raw exposures.info-warnings: Print warnings.
Contents
Datasets, nights, exposures¶
Most commands can take either a night or exposure name as input, so it is useful to get easily the list of nights or exposures:
$ musered info --datasets --nights --exps
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
Datasets:
- IC4406 : 6 exposures
Nights:
- 2017-04-23
- 2017-06-13
- 2017-06-15
- 2017-06-17
- 2017-06-18
- 2017-06-19
- 2017-10-25
- 2017-10-26
Exposures:
- IC4406
- 2017-06-16T01:34:56.867
- 2017-06-16T01:37:47.867
- 2017-06-16T01:40:40.868
- 2017-06-16T01:43:32.868
- 2017-06-16T01:46:25.866
- 2017-06-16T01:49:19.866
Raw data¶
info-raw takes a list of key:value items that define a selection on the
raw table. For example, to list all raw calibration files for a given
night:
$ musered info-raw night:2017-06-17 DPR_CATG:CALIB
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
name EXPTIME OBJECT DROT_POSANG MODE TEMP7_VAL OBS_NAME AG_FWHMX_MED FWHM_MED AIRM_END AIRM_START TPL_START
----------------------- ------- --------- ----------- -------- --------- ------------ ------------ -------- -------- ---------- -------------------
2017-06-18T11:03:46.725 0.0 BIAS 0.0 WFM-AO-N 10.03 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:04:56.419 0.0 BIAS 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:06:06.027 0.0 BIAS 0.0 WFM-AO-N 10.03 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:07:15.699 0.0 BIAS 0.0 WFM-AO-N 10.03 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:08:25.283 0.0 BIAS 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:09:34.817 0.0 BIAS 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:10:44.492 0.0 BIAS 0.0 WFM-AO-N 10.03 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:11:54.365 0.0 BIAS 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:13:04.113 0.0 BIAS 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:14:13.724 0.0 BIAS 0.0 WFM-AO-N 10.03 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T11:15:23.414 0.0 BIAS 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T11:03:09
2017-06-18T12:36:44.111 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:38:06.109 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:39:28.111 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:40:53.111 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:42:15.111 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:43:38.112 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:45:00.109 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:46:25.111 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:47:50.110 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:49:12.111 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:50:34.111 1.155 FLAT,LAMP 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:35:49
2017-06-18T12:52:06.110 1.012 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T12:53:28.111 1.012 WAVE 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T12:54:50.111 1.012 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T12:56:15.111 1.012 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T12:57:37.111 1.012 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T12:59:03.111 8.76 WAVE 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:00:24.111 8.76 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:01:46.111 8.76 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:03:07.112 8.76 WAVE 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:04:28.111 8.76 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:05:54.111 0.539 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:07:16.111 0.539 WAVE 0.0 WFM-AO-N 10.01 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:08:37.110 0.539 WAVE 0.0 WFM-AO-N 9.99 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:09:58.110 0.539 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
2017-06-18T13:11:19.110 0.539 WAVE 0.0 WFM-AO-N 10.0 Calibrations 0.697 0.0 None None 2017-06-18T12:51:47
Reduction status¶
Without arguments the command gives an overview of the current state of the reduction, with the number and type for the raw data, processed calibration, and reduced data:
$ musered info
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
Reduction version 0.1
155 files
Datasets:
- IC4406 : 6 exposures
Runs:
- GTO17 : 2017-04-01 - 2017-06-30, 6 exposures (1 flagged)
Raw data:
name BIAS FLAT,LAMP FLAT,LAMP,ILLUM IC4406 WAVE STD FLAT,SKY
---------- ---- --------- --------------- ------ ---- --- --------
2017-06-15 11 11 3 6 15 -- --
2017-06-17 11 11 -- -- 15 -- --
2017-06-18 -- 11 2 -- 15 1 4
2017-06-19 11 -- -- -- -- -- --
Processed calib data:
name bias flat lsf wavecal twilight standard scibasic
---------- ---- ---- --- ------- -------- -------- --------
2017-06-15 1 3 1 2 -- -- --
2017-06-17 1 3 1 2 -- -- --
2017-06-18 -- 3 1 2 2 4 1
2017-06-19 1 -- -- -- -- -- --
Processed science data:
name scibasic scipost ... exp_combine zap mpdaf_combine
----------------------- -------- ------- ... ----------- --- -------------
2017-06-16T01:34:56.867 1 9 ... -- -- --
2017-06-16T01:37:47.867 1 9 ... -- -- --
2017-06-16T01:40:40.868 1 9 ... -- -- --
2017-06-16T01:43:32.868 1 9 ... -- -- --
2017-06-16T01:46:25.866 1 9 ... -- -- --
2017-06-16T01:49:19.866 1 9 ... -- -- --
IC4406_drs -- -- ... 2 -- --
IC4406_mpdaf -- -- ... -- 4 5
OFFSET_LIST_drs -- -- ... -- -- --
It is possible to view only a specific table with the --table argument
(possible values are raw, calib, science):
$ musered info --tables raw
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
Reduction version 0.1
155 files
Datasets:
- IC4406 : 6 exposures
Runs:
- GTO17 : 2017-04-01 - 2017-06-30, 6 exposures (1 flagged)
Raw data:
name BIAS FLAT,LAMP FLAT,LAMP,ILLUM IC4406 WAVE FLAT,SKY STD
---------- ---- --------- --------------- ------ ---- -------- ---
2017-06-15 11 11 3 6 15 -- --
2017-06-17 11 11 -- -- 15 -- --
2017-06-18 -- 11 2 -- 15 4 1
2017-06-19 11 -- -- -- -- -- --
Reduction log for a night or exposure¶
The info-exp sub-command allows to see all the recipes that have been
executed for a given night or exposure, with the execution date, log file,
output directory, etc.:
$ musered info-exp --night 2017-06-17 --short
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
2017-06-18T11:03:09
★ Recipe: muse_bias
- date : 2018-11-14T18:58:26.549235
- log : ./reduced/0.1/logs/muse_bias-2018-11-14T18:57:14.051104.log
- json : ./reduced/0.1/MASTER_BIAS/2017-06-18T11:03:09.WFM-AO-N/muse_bias.json
- frames : [1mMASTER_BIAS[0m
- path : ./reduced/0.1/MASTER_BIAS/2017-06-18T11:03:09.WFM-AO-N
- runtime : 799.8 (user) 81.8 (sys)
2017-06-18T12:35:49
★ Recipe: muse_flat
- date : 2018-11-14T19:44:25.820095
- log : ./reduced/0.1/logs/muse_flat-2018-11-14T19:41:38.846366.log
- json : ./reduced/0.1/MASTER_FLAT/2017-06-18T12:35:49.WFM-AO-N/muse_flat.json
- frames : [1mMASTER_FLAT[0m, [1mTRACE_TABLE[0m, [1mTRACE_SAMPLES[0m
- path : ./reduced/0.1/MASTER_FLAT/2017-06-18T12:35:49.WFM-AO-N
- runtime : 2201.1 (user) 199.9 (sys)
2017-06-18T12:51:47
★ Recipe: muse_wavecal
- date : 2018-11-14T20:03:11.072881
- log : ./reduced/0.1/logs/muse_wavecal-2018-11-14T19:51:15.452569.log
- json : ./reduced/0.1/WAVECAL_TABLE/2017-06-18T12:51:47.WFM-AO-N/muse_wavecal.json
- frames : [1mWAVECAL_TABLE[0m, [1mARC_RED_LAMP[0m
- path : ./reduced/0.1/WAVECAL_TABLE/2017-06-18T12:51:47.WFM-AO-N
- runtime : 2893.3 (user) 832.0 (sys)
★ Recipe: muse_lsf
- date : 2018-11-14T20:21:26.366658
- log : ./reduced/0.1/logs/muse_lsf-2018-11-14T20:18:27.887743.log
- json : ./reduced/0.1/LSF_PROFILE/2017-06-18T12:51:47.WFM-AO-N/muse_lsf.json
- frames : [1mLSF_PROFILE[0m
- path : ./reduced/0.1/LSF_PROFILE/2017-06-18T12:51:47.WFM-AO-N
- runtime : 3225.2 (user) 577.7 (sys)
$ musered info-exp 2017-06-16T01:46:25.866
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
2017-06-16T01:46:25.866
★ GTO logs:
★ Weather Conditions:
- 00:06 ThinCirrus thin clouds, 25 % of sky with thick clouds.
- 02:04 ThinCirrus Thin condition
- 03:20 ThinCirrus N/A
- 04:29 ThinCirrus N/A
- 05:18 Clear clear condition, few thin clouds leaving, a line clouds passed fast
- 06:30 Clear clear condition, some thin clouds are coming from W
- 07:13 Clear a line clouds passed fast
- 08:33 Clear clear condition, thin clouds around the summit, about 20 deg
- 09:57 Clear N/A
- 22:04 ThickCirrus we started night with thick clouds
- 23:16 ThickCirrus same condition, more than half sky covered by thick clouds, more clouds are coming from West.
★ Recipe: muse_scibasic
- date : 2018-11-15T15:29:01.544258
- log : ./reduced/0.1/logs/muse_scibasic-2018-11-15T15:27:54.322498.log
- json : ./reduced/0.1/scibasic/2017-06-16T01:46:25.866/muse_scibasic.json
- frames : [1mPIXTABLE_OBJECT[0m
- path : ./reduced/0.1/scibasic/2017-06-16T01:46:25.866
- runtime : 620.5 (user) 166.4 (sys)
★ Recipe: muse_scipost_rec
- date : 2018-11-15T15:43:05.955865
- log : ./reduced/0.1/logs/muse_scipost-2018-11-15T15:41:05.559036.log
- json : ./reduced/0.1/scipost-not-recentered/2017-06-16T01:46:25.866/muse_scipost_rec.json
- frames : [1mDATACUBE_FINAL[0m, [1mIMAGE_FOV[0m
- path : ./reduced/0.1/scipost-not-recentered/2017-06-16T01:46:25.866
- runtime : 2944.3 (user) 238.7 (sys)
★ Recipe: muse_scipost
- date : 2018-11-15T16:38:09.655303
- log : ./reduced/0.1/logs/muse_scipost-2018-11-15T16:30:46.578663.log
- json : ./reduced/0.1/scipost/2017-06-16T01:46:25.866/muse_scipost.json
- frames : [1mDATACUBE_FINAL[0m, [1mIMAGE_FOV[0m, [1mPIXTABLE_REDUCED[0m, [1mRAMAN_IMAGES[0m, [1mSKY_IMAGE[0m, [1mSKY_MASK[0m, [1mSKY_SPECTRUM[0m, [1mSKY_LINES[0m, [1mSKY_CONTINUUM[0m
- path : ./reduced/0.1/scipost/2017-06-16T01:46:25.866
- runtime : 14848.6 (user) 1254.3 (sys)
- warning : 1
★ Recipe: muse_scipost_make_cube
- date : 2018-11-15T22:27:37.415940
- log : ./reduced/0.1/logs/muse_scipost_make_cube-2018-11-15T22:25:27.770568.log
- json : ./reduced/0.1/scipost_cubes/2017-06-16T01:46:25.866/muse_scipost_make_cube.json
- frames : [1mDATACUBE_FINAL[0m, [1mIMAGE_FOV[0m
- path : ./reduced/0.1/scipost_cubes/2017-06-16T01:46:25.866
- runtime : 3230.3 (user) 319.0 (sys)
Recipe warnings¶
The info-warnings sub-command allows to list the warnings from the DRS:
$ musered info-warnings
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
name muse_scipost muse_scipost_make_cube muse_wavecal
----------------------- ------------ ---------------------- ------------
2017-06-16T01:34:56.867 1 3 --
2017-06-16T01:37:47.867 1 3 --
2017-06-16T01:40:40.868 1 -- --
2017-06-16T01:43:32.868 1 -- --
2017-06-16T01:46:25.866 1 -- --
2017-06-16T01:49:19.866 1 3 --
2017-06-19T12:20:06 -- -- 5
$ musered info-warnings --mode list
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
recipe_name ...
---------------------- ...
muse_scipost ...
muse_scipost_make_cube ...
muse_scipost ...
muse_scipost_make_cube ...
muse_scipost ...
muse_scipost ...
muse_scipost ...
muse_scipost ...
muse_scipost_make_cube ...
muse_wavecal ...
QC parameters¶
To view the QC parameters for a given type (DPR_TYPE) and night or exposure:
$ musered info --qc MASTER_FLAT --date 2017-06-17
[Errno 2] No such file or directory: '/home/docs/.esorex/esorex.rc'
2017-06-18T12:35:49
filename hdu ... QC_FLAT_MASTER_STDEV QC_FLAT_MASTER_INTFLUX
---------------- ------ ... -------------------- ----------------------
MASTER_FLAT.fits CHAN01 ... 15421.67 396126100000.0
MASTER_FLAT.fits CHAN02 ... 15473.66 393503500000.0
MASTER_FLAT.fits CHAN03 ... 15529.19 414231400000.0
MASTER_FLAT.fits CHAN04 ... 15543.17 406139900000.0
MASTER_FLAT.fits CHAN05 ... 15950.46 417421700000.0
MASTER_FLAT.fits CHAN06 ... 15825.45 415001000000.0
MASTER_FLAT.fits CHAN07 ... 16025.97 418396300000.0
MASTER_FLAT.fits CHAN08 ... 15597.07 414043500000.0
MASTER_FLAT.fits CHAN09 ... 16133.19 423498400000.0
MASTER_FLAT.fits CHAN10 ... 16368.74 429870300000.0
MASTER_FLAT.fits CHAN11 ... 16085.13 424515800000.0
MASTER_FLAT.fits CHAN12 ... 16306.88 429704400000.0
MASTER_FLAT.fits CHAN13 ... 16819.5 434821100000.0
MASTER_FLAT.fits CHAN14 ... 16748.14 440856200000.0
MASTER_FLAT.fits CHAN15 ... 16444.98 431420600000.0
MASTER_FLAT.fits CHAN16 ... 15732.93 408157200000.0
MASTER_FLAT.fits CHAN17 ... 17133.99 437652400000.0
MASTER_FLAT.fits CHAN18 ... 16763.75 437573400000.0
MASTER_FLAT.fits CHAN19 ... 16853.64 438535000000.0
MASTER_FLAT.fits CHAN20 ... 16881.67 438573000000.0
MASTER_FLAT.fits CHAN21 ... 16527.18 433892300000.0
MASTER_FLAT.fits CHAN22 ... 16667.54 434641700000.0
MASTER_FLAT.fits CHAN23 ... 16220.93 423059500000.0
MASTER_FLAT.fits CHAN24 ... 15816.5 403002600000.0