Steve Lubow will be reducing his involvement with the OTFC project to concentrate on APT.
OTFC is performing well. Nearly all requests for OTFC data are for proprietary data. NonOTFC requests involve a mixture of proprietary and nonproprietary data. Most requests (about 2/3) are for OTFC. There was a question of whether the nonOTFC requests only involved raw data. Lisa will figure this out.
There is a problem with Bestref for STIS. There are 2 parts of Bestref: One part updates reference file names for a specified dataset. The other part updates reference file names for datasets affected by a newly delivered reference file. The first part works properly, while the second part does not work properly for STIS. Bernie Shiao is working on this bug. The current entries in the database have been corrected.
Currently, requests for calibrated data from the archive use the stored calibrated data as a default, rather than OTFC. When only calibrated data is being stored in the archive, the default for data retrieval might be changed to OTFC. The decision will be determined by the archive branch.
There was discussion about the interface between OTFR and the CADC/ECF. It was pointed out that the requirements for this interface could be met by maintaining an accurate DADS catalog, which is also of benefit to Starview users. It may suffice to maintain parameters that are important for calibration. The means and overhead for maintaining the catalog needs to be determined.
Remaining issues:
The fix for the DADS test pipe to OTFC has not yet been completed. There is another DADS problem that is getting higher priority. There is no schedule for when the test pipe will be enabled. Meanwhile, we cannot use the test pipe for OTFC.
The STIS group wants a record for each OTFC request of the user name, the dataset name, and the reference files used. The purpose is to provide the group with the ability to contact users who have data calibrated with an invalid reference file. This could be implemented as a database insert that takes place as a part of the OPUS processing script. Lisa is investigating the effort and overhead required to do the database portion of this task.
The OTFC performance statistics are now being gathered in the DADS database table. Lisa is developing queries to produce reports. The cpu time has a 1 second granularity.
There may be some advantage to creating an OTFR pathfinder. In the past, some issues were better understood by the pathfinders for STIS and WFPC2. These included performance and problems with data. In the case of OTFR, the exact implementation has not yet been determined. That is, we do not know how pod files will be accessed and processed in the longer term. These issues do not need full resolution for the purposes of constructing a pathfinder. Several sources of problems have already been identified. Some keyword values may have been changed in the edt files, but not in the pod files. In addition, there are missing entries in the data podnames database relation which is critical to OTFR. We think we know what the missing entries are.
AN OTFR cache of about 600 GB would be required to store all the existing WFPC2 and STIS POD files. Some savings through data compression may occur. This could be done through either MO disks or standard disks, which are now cheap. With standard disks, the cost would be around $37,000.
OPUS is going to save trailers for requests that go to trouble. We discussed having a mechanism to automatically alert the instrument teams about calibration problems through email. The archive branch and the instrument groups want this. A script that operates as a cronjob will need to be written.