Calibration Benchmarking

Contents

Overview

In preparation for the Preliminary Design Review for the On-The-Fly Calibration (OTFC) system, benchmarking runs of SDAS calibration software were performed to gauge the performance of the current WFPC-2, NICMOS, and STIS calibration executables, as well as to provide an estimate of the performance of the planned ACS calibration executable.

Test datasets were constructed for each instrument, and these same datasets were run repeatedly on the target architectures, using both single copies of the calibration executable, and multiple copies (parallel processing). Runs were made both during normal work hours and after-hours. Where the difference was significant, the best after-hours performance is shown.

Architectures

Three SESD systems were chosen for comparison:

system: hoth config: 1 300MHz CPU (SPARC Ultra-30) SunOS 5.6 256MB memory 19MB/sec disk access (/hoth partition) system: Q config: AlphaServer 2100 5/250 1 250 MHz ALPHA CPU VMS 7.1 256 MB memory ? MB/sec disk access (disk$draco device on a 90 MB/sec line, but access rate depends on load) system: whaler config: Digital 4100 2 ALPHA 5/300 CPUs Digital UNIX 4.0 512 MB memory 9 MB/sec disk access (/c1d2pg partition)

All data and reference files were stored on disks local to the host system (not network served) unless noted otherwise.

CALWP2 (WFPC-2) version 1.3.5.2

The WFPC-2 dataset consists of 24 FULL, EXTERNAL exposures.

SPARC Ultra-30

Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 384 1 20.96 0.06 216 2 16.33 96 3 15.79 48 4 13.15 16 1 63.38 * 1.18 8 1 61.50 ** 1.70 48 1 20.05 *** 0.61 * - run with both data files and reference files on NON-local disk ** - run with data files on NON-local disk, reference files on local disk *** - run with data files on local disk, reference files on NON-local disk (NON-local disk NFS-served from 'deathstar' system)

AlphaServer 2100

Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 48 1 83.76 0.12

Digital 4100

Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 72 1 9.85 0.10 96 2 6.61 96 3 5.83

CALNICA (NICMOS)

The CALNICA NICMOS dataset consists of 60 MULTIACCUM exposures (NSAMP ranges from 4 to 19, avg= 9.6).

SPARC Ultra-30

CALNICA (version 3.0) Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 480 1 17.09 0.13 180 2 17.09 90 3 16.35

Digital 4100

CALNICA (version 3.1.1) Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 240 1 12.50 0.05 300 2 7.16 420 3 6.26

CALNICB (NICMOS) version 2.1.2

The CALNICB NICMOS dataset consists of 9 associations containing from 3 to 19 members, avg= 7 members.

SPARC Ultra-30

Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 18 1 24.00 4.36

Digital 4100

Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 18 1 11.46 2.05

CALSTIS (STIS)

The STIS dataset consists of a variety of observation types, chosen roughly to reflect the percentages of observation types across all Cycle-7 programs. The observation mix is as follows:

Number of Percentage observations of total obs Observation Type ------------ ------------ ---------------- 12 28% IMAGING ACCUM CCD 1 2% IMAGING ACCUM MAMA 1 2% IMAGING TIMETAG MAMA 20 45% SPECTROSCOPIC ACCUM CCD 4 9% SPECTROSCOPIC ACCUM MAMA 6 14% SPECTROSCOPIC TIMETAG MAMA -- ---- 44 100%

The mix includes more TIMETAG observations than the Cycle-7 percentage dictates, since these types of calibrations were originally thought to "stress" the system and simulate a heavier-than-usual processing load. This idea was a misconception. It was extrapolated from the larger processing times required by the OPUS Generic Conversion program for TIMETAGS, but the CALSTIS times do not show great differences between ACCUM and TIMETAG data.

NOTE: Closer inspection of the calibration switches for the STIS dataset revealed that about 40% of the observations had a large number of calibration switches set to OMIT. When these datasets were removed from the sample, the processing times increase by around 80% on the Digital 4100 system, and around 50% on the SPARC Ultra-30.

SPARC Ultra-30

CALSTIS (version 1.5a) Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 176 1 72.99 0.70 220 2 65.31 176 3 67.82

Digital 4100

CALSTIS (version 1.7) Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 132 1 51.73 0.17 132 2 26.86 132 3 26.48

The following runs were made only on the IMAGING ACCUM CCD set (12 obs), which may serve as a possible benchmark for estimating ACS calibration. (The results are only slightly higher than those for the entire STIS set.)

SPARC Ultra-30

Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 36 1 74.05 0.85 36 2 67.28

Digital 4100

Number of Number of Avg. time Std. deviation observations processes per observation run (seconds) (seconds) --------- --------- --------------- -------------- 36 1 53.95 0.31 36 2 31.42