This thread describes how to create a data quality map, used to exclude certain aberrant detectors. Typically, the SDC creates a data quality map in their automatic pipeline. This task provides information on how to create a new file if one is missing. The process involves creating detector plane image, and then searching for noisy detectors using the bathotpix task.
Read this thread if you want to: Mask out noisy or defective detectors from GRB analysis.
Last update: 2006-09-21
If it has not been run before (i.e. missing files).
To capture improvements in the newer software/ground calibration.
To focus on a narrow energy or time range where some detectors became noisy.
BAT detector enable/disable map (in obsid/bat/hk/swNNNNNNNNNNNbdecb.hk.gz);
BAT event data (in obsid/bat/event/swNNNNNNNNNNNbevsh*.evt.gz);
Access to Calibration Database (CALDB).
The SDC should be producing a detector enable/disable file for every observation. If it is not present, then you must locate the nearest one in time in the trend data in the archive (see below).
The process involves three main steps. First, a detector plane image of the event data is created. Second, using the observation time of the image, a master quality map is retrieved from CALDB. Finally, the detector plane image is searched for noisy detectors which may have flared up during the observation, using the task bathotpix.
In this case, let us create a quality map for GRB 050713A (obsid 00145675 000).
Change directories to the location of the event data.
cd 00145675000/bat/event
Create a detector plane image for the time or energy range of interest. This is a temporary file which will be checked for noisy detectors.
For our example, we will simply sum all the event data, including all times and all energy ranges. This should normally be acceptable, since summing over a long period of time should produce the most conservative number of detectors to exclude.
batbinevt infile=sw00145675000bevshsp_uf.evt.gz outfile=total.dpi \ outtype=DPI timedel=0 timebinalg=u energybins=- \ weighted=NO outunits=COUNTS clobber=YES
infile is the event file accumulated;
outfile is the temporary detector plane image;
outtype=DPI indicates that a detector plane image will be created;
timedel=0 indicates to sum all the data into one map. It would be possible to set tstart= and tstop= to narrow the range of time if desired;
energybins=- indicates to sum all energies. A narrower energy range can be given as ELOW-EHIGH;
weighted=NO indicates to make an unweighted detector map
outunits=COUNTS indicates that the map should be in units of counts instead of rate;
The BAT team maintains a list of known bad detectors, which is stored in the Calibration Database (CALDB). These maps only contain detectors (usually detector modules) which were known for operational reasons to have issues which affect science analysis. The user is still responsible for filtering the data for noisy detectors (see the next section) or other obvious problems. The 'batdetmask' task is used to retrieve the appropriate detector quality map from CALDB.
batdetmask date=total.dpi outfile=master.detmask clobber=YES \ detmask=../hk/swt0142921337bdecb.fits.gz
date tells batdetmask to retrieve the observation date from the detector plane image you just created
outfile is the output master quality map for the observation
detmask is the detector enable/disable map (usually in the bat/hk directory)
We use the detmask parameter to combine the existing flight enable/disable map with the quality map based on ground analysis.
Noisy detectors are found with the bathotpix task.
bathotpix infile=total.dpi outfile=total.qmap \ detmask=master.detmask clobber=YES
infile is the detector plane image you just created
outfile is the resulting quality map
detmask is the quality map we made in the previous step
The quality map should be used in all subsequent analysis. In particular, it should be used during the mask weighting process (using the batmaskwtevt task), and in the generation of all light curves and spectra (using the batbinevt task) and images (using the batfftimage) task. All of these tasks have an input parameter called detmask= which accept a detector quality map. This information is used to exclude noisy detectors, and to properly normalize the resulting output fluxes.
There is a difference between the enable/disable map and the quality map we just produced. The enable/disable map is produced by the flight software, and records which detectors were disabled at the time. Thus, this map indicates which detectors were noisy in the past, were disabled, and now have zero counts. The quality map contains that information, but also excludes any currently noisy detectors, i.e. detectors with too many counts. The BAT flight software does not catch all noisy detectors. Some detector noise is sporadic, or below the count rate threshold for disabling. Thus, the best quality map is a combination of both the disabled detectors and the currently noisy ones.
Detector enable/disable maps are periodically produced by the BAT flight software to indicate which detectors have been disabled in flight. When noisy detectors appear and are detected by the flight software, they are automatically disabled to prevent them from interfering with the BAT triggering process.
It is important to understand that the detector enable/disable map production is not synchronized with the observing plan. The map may have been produced during a previous observation.
Normally the SDC retrieves the proper map for GRB data and includes it with the data set. If there are processing errors, or for non-GRB observations, the detector enable/disable maps may be missing. If that is the case, then the map must be found manually from the trend data.
Search in the BAT trend data by month. See /swift/data/trend/YYYY_MM/bat/bdetflag on the HEASARC FTP site.
You want the next earliest detector enable/disable map produced before the event data in question. The files are listed by time (expressed as mission elapsed time, or MET), so one needs to scan the list until the right time is found.
First, determine the time of the data you are interested in. Usually you can look at the TSTART keyword,
ftlist sw00145675000bevshsp_uf.evt.gz K | grep TSTART
which produces the following
TSTART = 142921443. / Start time
A quick scan of the trend directory in /swift/data/trend/2005_07/bat/bdetflag shows that the next earliest detector enable/disable map is swt0142921337bdecb.fits.gz. In this case the time listed in the file name is about 106 seconds before the start of the event data.
It is not vitally important to get exactly the right detector map. Having a few detectors incorrectly labeled as enabled or disabled will not substantially alter the analysis.
2006-08-09 - first revision
2006-09-20 - add section on batdetmask