-----------------------------------------------------
Starview & DADS (Allen Farris & Warren King) -
The operational DADS system is now available for OTFC and appears to be working OK.
There remains a problem with the file list sent to users via email. It does not contain the names of the newly created files in the OTFC pipeline.
Action Item: This will be fixed in a future DADS release, 7.8.3. This is about 1 month away.
DADS support for the two destination pipes: otfc and nonotfc needs to be supported by the time OTFC is released externally. The reason is that we need to maintain two separate otfc pipelines: a test and an operational system. Code is in place to do this, but is untested.
Action Item: Allen and Joel will work this out. -----------------------------------------------------
OPUS (Mike Swam) - The calibration pipeline is working on artichoke.
There is a problem with ASN files getting split into separate requests. The OTFC pipeline attempts to process the ASN request and fails.
Action Item: Mike Swam will change the pipeline so that the ASN file gets passed directly to DADS distribution. This should be done before next week's meeting.
Mike is to develop the common rules system for OTFC. This will begin in 2 to 3 weeks. -----------------------------------------------------
getref/upref (Bernie Simon) - This is working properly. These tools will be modified in the future to handle associated data and keyword placement.
-----------------------------------------------------
Change System (Lisa Gardner, Steve Lubow, & Bernie Shiao) -
Scripts are handling the updating of the internal test system on ZEPPO to emulate the automated change system.
The automatic change system needs further development.
Action Item: Lisa is to develop support for the pending datasets table. -----------------------------------------------------
STIS (Bernie Shiao, Ed Hopkins, Melissa McGrath) - Ed and Bernie Shiao are working on the list of proposed modifications to the headers (via the otfc_keyword_repair table). Queries have been written to handle all but one of the list of changes (involving NCOMBINE).
Action Item: These changes need to be reviewed by the STIS team
Currently, the pipeline filters out all blank switch values for WAVECALs from the delta file. Getref has been modified to not supply recommendations in the case that the WAVECAL switch values are blank (null). Only new (best) switch values for WAVECALs are stored in DADS. No values have been populated because the BestSwitch system is not yet in place. A problem with WAVECALS is that not all keywords are in the database that are needed for BestSwitch. Steve Lubow sent out an email message describing the various options for determining the switches for WAVECALS. Melissa reported that the STIS team would like to have either 1) WAVECAL keywords be ingested into the DADS database or 2) the WAVECAL switches never be changed. So OTFC would not provide BestSwitch support for WAVCAL switches.
Action Item: Bernie Shiao will investigate the level of effort required to retrieve the 2700 or so WAVECAL datasets. Lisa will determine the effort required to make the database changes and the ingest process changes for WAVCALs. Since there is no short-term need to change these values, this effort is of low priority. -----------------------------------------------------
Testing (Melissa McGrath, Steve Lubow, Steve Slowinski) - This system now appears to work correctly. The current plan is to open the OTFC system to STScI internal testing in early March.
Random testing - Steve Lubow ran mixed requests of otfc (STIS and WFPC2) and nonotfc (FOS) data including cases involving bypassed calibration data and keyword repairs. Requests were also made for the same dataset, separated in time by under one minute. There were no problems. Some performance measurements were made which show the performance was about what was expected.
Regression testing Steve Slowinski ran the WFPC2/OTFC regression suite. The data appears to be correct.
Action Item: Steve will carry out stress tests, including the case of processing a dataset with a missing reference file.
Action Item: Steve will work with the WFPC2 team to determine that the data are correct, based on their set of regression data. We need to have a standard set of results to compare against in an automated manner.
Sylvia and Faith will test the consistency of newly calibrated data between the initial pipeline and OTFC. Faith is out for weeks.
Melissa is identifying a set of STIS data for a regression test (about 50 - 100 datasets.) This is about 3/4 done. It will be completed in the next one or two weeks. -----------------------------------------------------
User Interfaces (Steve Lubow)
Some issues were discussed concerning the adequacy of our user interfaces to OTFC. Melissa raised the concern that once the OTFC/WFPC2 testing is open to the entire STScI, the STIS/OTFC system will be exposed to institute-wide use, but without adequate testing. Unfortunately, there is no easy way to disable institute-wide users from STIS testing, while OTFC/WFPC2 testing takes place. The internal test version of Starview will warn users that this is a test system only and that STIS is in an early stage of testing.
Action Item: (Steve Lubow) The internal Starview screens need to be updated to reflect the state of the system.
We began a discussion about the retrieval screen for OTFC. It is confusing and needs to be clarified. -----------------------------------------------------
Operational Support
Once the internal STScI-wide testing begins, we will need to have the system monitored for problems. The support involves the following elements.
Action Item: Steve Slowinski will contact Herb about gaining experience with the OMG.