UNIVERSITY OF CALIFORNIA, SAN DIEGO ERSITY C ORNIASAN DIEGO 3 1822 04101 6486 3 1822 04101 6486 Version 1.1 July 2007 Offsite SPHERIC OPTICS GROUP (Annex-Joical Note No. 272 rnals) QC 974.5 . 143 Whole Sky Imaging of Clouds no. 272 In the Visible and IR For Starfire Optical Range Published as Final Report for ONR Contract N00014-01-D-0043 DO #11 UNIVERSITY OF CALIFORNIA SAN DIEGO Janet E. Shields Monette E. Karr Art R. Burden Richard W. Johnson William S. Hodgkiss The material contained in this note is to be considered proprietary in nature and is not authorized for distribution without the prior consent of the Marine Physical Laboratory. SCRIPPS. INSTITUTION OF OCEANOGRAPHY MARINE PHYSICAL LAB San Diego, CA 92152-6400 UNIVERSITY OF CALIFORNIA, SAN DIEGO DOPADDERS COM TARAN COCCA RESTRORRA 3 1822 04101 6486 ATMOSPHERIC OPTICS GROUP Technical Note No. 272 Version 1.1 July 2007 Whole Sky Imaging of Clouds In the Visible and IR For Starfire Optical Range Published as Final Report for ONR Contract N00014-01-D-0043 DO #11 UNIVERSITY OF CALIFORNIA SAN DIEGO Janet E. Shields Monette E. Karr Art R. Burden Richard W. Johnson William S. Hodgkiss TERT UNITS TORN The material contained in this note is to be considered proprietary in nature and is not authorized for distribution without the prior consent of the Marine Physical Laboratory. SCRIPPS INSTITUTION OCEANOGRAPHY OF MARINE PHYSICAL LAB San Diego, CA 92152-6400 Whole Sky Imaging of Clouds In the Visible and IR For Starfire Optical Range Table of Contents 1. Introduction ..... ......................................................................... .. ....... 2. Background ....... Statement of Work.......... Funding Increments and Optional Tasks........ 5. 5.1 5.2 5.3 5.4 Major Deliveries and Documentation ..... ................... Algorithm and Ground-truthing Developments ..... Analysis of Infrared Systems .. Evaluation of System Upgrades .... Refurbishment of WSI Units for Field Deployment Support and Maintenance Support .... 5.4.1 WSI Refurbishment ............. 5.4.2 Software Upgrades for the Refurbished WSIS ............. 5.4.3 Logistics, Testing, and Deployment............ ......... 5.4.3 Field Repairs... ... .......... 6. 6.1 6.2 6.3 6.4 6.5 6.6 Day Algorithm Upgrade and Analysis ........... Previous Day Algorithm Results.... ........... Algorithm Results for Data from the SOR Site...... Systematic Evaluation of SOR Results ........... Evaluation of Data from the Hazier Virginia Site.......... ............. Results of a Comparison Test with a Ceilometer ......... Summary of Day Cloud Algorithm Results and Future Plans ....... .....21 ............. .24 7. 7.1 7.2 7.3 7.4 7.5 7.6 Night Algorithm Developments and Analysis ....... Ground-truthing using Bright Stars....... Concepts for a High Resolution Night Algorithm ........... Processing of a SOR Night Database ... Upgrading the Night Algorithm ...... Evaluation of Night Data at a Very Bright Site ..... Summary of Night Algorithm Work ........... ............30 ............38 .38 ............40 8. Wavelength Options for Optical Cloud Imaging .... ........... 9. Hardware and Software Developments and System Preparation for Deployment ........ ........... Concepts for System Upgrades ............. 9.2 System Maintenance ................. 9.3 WSI Refurbishment - Hardware ......... .............45 9.4 WSI Refurbishment - Software Upgrading the Night Algorithm .........48 9.5 Deployment Logistics Support... 10. Summary ... 11. Acknowledgements ........ 12. References 12.1 In-house Technical Memoranda............. 12.2 PowerPoint Files from Presentations to SOR ....... 12.3 Published References and Technical Notes .. ............57 List of Illustrations Fig. 1 Fig. 2 Fig. 3 Fig. 4 Fig. 5 Fig. 6 Fig. 7 Fig. 8 Fig. 9 ಹ ಹ ಹ ೧ ೧ ೧ ೧ ...19 Fig. 13 Fig. 14 Fig. 15 Fig. 16 Fig. 17 Raw red and processed cloud decision from SOR site, 26 Jul 05 2200.. .............11 Raw red and processed cloud decision from SOR site, 9 Jul 05 1700 ..........11 Raw red and processed cloud decision from SOR site, 26 Jul 05 2100 ........12 Raw red and processed cloud decision from SOR site, 31 Jul 05 1500 ........ SORCloudAssess program display for user assessment of algorithm results ..........13 Fraction of correct answers, in percent, for each ROI for SOR Day Test Bed Processing ........ Example showing failure to detect thin clouds ............... Same image as Fig. 7, but run with the adaptive algorithm turned on .......... Example of failure of algorithm at sunrise ....... ............... Typical results showing ability to detect clouds near the horizon .............. Raw NIR and cloud decision from Virginia site 15 April 05 1900 ........... Raw NIR and cloud decision from Virginia site 12 April 05 1500 ............. Raw NIR and cloud decision from Virginia site 02 April 05 2200 ............ Raw NIR and cloud decision from Virginia site 18 April 05 1600 ........ Raw NIR and cloud decision from Virginia site 19 April 05 1500 ............. Fraction of correct answers, in percent, for each ROI for Day Virginia Test Bed Processing WSI Images with simultaneous ceilometer report, examples where ceilometer results are not valid ........... Alnath measured spectral irradiance on 11 Nights ........ Alnath spectral irradiance corrected for aerosol extinction.. Clear night, 14 Feb 99, 0340.......... Cloudy night, 16 June 99 0510.. ......... .............25 Broken clouds, 16 June 99, 0710........ Broken clouds, 16 June 99 0730.. Broken clouds, 16 June 99, 0840.. ........... Broken clouds, 16 June 99 0850 .......... Central Row for Clear Sky... Central Column for Clear Sky ............ .......... Clear Sky background image. .......... Central Row plotted as a function of hour angle ........ Cloud Background (black curve) compared with clear background (colored curves)....... Grey coloring indicates regions where cloud background differs by over 100%.. Clear sky sample, no moon ......... Radiance from Fig. 32 compared with nominal clear sky and opaque sky radiances ........... ......... Fig. 18 Fig. 19 Fig. 20 ........... Fig. 23 Fig. 24 Fig. 25 Fig. 26 Fig. 27 Fig. 28 ......... Fig. 29 Fig. 31 Fig. 32 Fig. 33 Fig. 34 Broken transparent cloud sample. ............... ..30 Fig. 35 ....... 31 ......... ·31 Fig. 36 Fig. 37 Fig. 38 Fig. 39 Fig. 40 Fig. 41 Fig. 42 Fig. 43 Fig. 44 . 45 Fig. 46 ....... ......... .......... 35 Radiance from Fig. 34 compared with nominal clear sky and opaque sky radiances .................. ............30 Nearly cloud-free moonlight case ........ ........... Cloudy with similar moonlight ......... Middle row extracted from Figures 36 and 37 ............ Example of a relatively good night clear sky result..... ..............32 Example of a relatively poor night clear sky result. ............. Example of a relatively good night overcast sky result ..............33 Example of a relatively good night overcast sky result ..... ............ Example of a night broken cloud result .... Example of a night case with opaque and thin clouds ........... Example of a moonlight case ............ Fraction of correct answers, in percent, for each ROI for SOR Night Test Bed for Line of Sight Test ........ Fraction of correct answers, in percent, for each ROI for SOR Night Test Bed for Region of Sight Test.. Clouds overhead in a very bright city ............ ........................ Clouds near horizon in a very bright city ............. ......... Thin clouds, imagery from a SWIR system at 1.6 um on the left, and the WSI in the visible at 650 nm 7 May 04 near 1230 Local..................40 Sunset, imagery from a SWIR system at 1.6 um on the left, and the WSI in the visible at 650 nm 11 May 04 near 1942 L ......... ........41 Initial design for computer control to replace Accessory Control Panels .....44 Sensor and Controller Configuration for Unit 7 .. ...........46 New User Interface, to enable inexperienced users to assess system status..50 Fig. 47 Fig. 48 Fig. 49 Fig. 50 Fig. 51 Fig. 52 Fig. 53 Fig. 54 Tables Table 1 Table 2 Summary of Estimated Accuracy for the Day Cloud Algorithm using SORCloudAssess for the SOR and Virginia Test Bed Data Sets ...... Summary of Estimated Accuracy for the Night Cloud Algorithm using SORCloudAssess for the SOR Test Bed Data Sets ............................37 Sample IR Evaluation Results from Previous Report Computed Cloud and Background Results for 60° Zenith Angle (Colors explained in Text) .........42 Table 3 Whole Sky Imaging of Clouds In the Visible and IR For Starfire Optical Range Janet E. Shields, Monette E. Karr, Art R. Burden, Richard W. Johnson, and William S. Hodgkiss 1. Introduction This report describes the work done for the Starfire Optical Range, Kirtland Air Force Base under Contract N00014-01-D-043 DO #11, between 02 September 2004 and 30 April 2006. This work relates to the Air Force's need to characterize the cloud distribution during day and night, for a variety of applications, including support of research into impact of clouds on laser communication and support of satellite tracking. This contract followed Contract N00014-01-D-0043 DO #4, which will be discussed in Section 2, and is documented in Shields et al 2007, Technical Note 271. Under this contract, we began preparing Whole Sky Imager systems for field experiments in support of program goals, adapting the software and refurbishing the hardware. Significant progress was made both in the related cloud algorithms and in methods to assess their accuracy. A related contract was funded through Boeing during 31 January 2005 – 30 November 2005. The tasks completed under that contract are closely related to these tasks, and will also be reported here. In particular, early portions of the night algorithm work reported in Section 7, and early portions of the hardware and software refurbishment were completed partly under the ONR contract and partly under the Boeing contract. The work under this Boeing contract was finished in May 2005. A follow-on contract, ONR N00014-01-D-0043 DO #13 was funded on 20 April 2006. The work under DO #13 will be reported under a separate report upon completion of the contract. 2. Background A series of digital, automated Whole Sky Imagers (WSI) have been developed by MPL over many years, beginning in the early 1980's (Johnson et al. 1989 and 1991, and Shields et al. 1993, 1994, 1997a and b). (Published references are listed in section 12.3.) These systems are designed to acquire accurate imagery of the full upper hemisphere in several spectral filters, in order to assess the presence of clouds at each pixel in the image. A system capable of 24-hour operation, the Day/Night WSI, was developed by MPL under funding from the Air Force, Navy, and Army in the early 1990's (Shields et al. 1998, 2003b and e, 2004 a and b, and 2005b and c). One of the first two units was fielded at the Air Force's Starfire Optical Range in October 1992. Related systems have been designed and fielded over the years. These include a new Daytime Visible/NIR WSI (Feister et al. 2000, and Shields et al. 2003d), and visible and Short-wave IR systems for airborne use (Shields et al. 2003c). An imaging system for measuring visibility was developed and successfully tested (Shields et al. 2005a and 2006). Also, a field calibration device for use with the Day/Night WSI was developed (Shields et al. 2003a). S Partly as a result of SOR’s experience with the Day/Night WSI fielded in 1992, they have funded development of additional instruments, as well as algorithm development and data analysis in recent years. Under Contract N00014-97-D-0350 DO #2 (September 1997 – June 2001), MPL was funded to develop and provide a new Day/Night WSI, which was designated Unit 12 (Shields et al. 2003b). This unit included several design upgrades developed for other sponsors. The funding also included analysis and processing of existing daytime cloud decision images to provide statistical estimates of Cloud Free Line of Sight (CFLOS) and related properties. Under the optional funding, the WSI was also upgraded to run under Windows. This was a major upgrade, involving a new camera model, a new camera software library, and new WSI instrument control software. The instrument was delivered in January 1999, and has been running well for much of the time since. The data analysis results were also delivered in 1998 and 1999, and my understanding is that these data have proven to be quite useful. as 2 W C Under Contract N00014-97-D-0350 DO #6 (May 1999 – May 2003), we were funded to provide two additional instruments, Units 13 and 14 and do additional analysis work (Shields et al. 2004b). These instruments included significant design upgrades, including integration of the control computer into the outdoor environmental housing, new control hardware and software, and new software to provide near-real-time cloud processing on an additional display computer. One of the instruments was fielded at a site in California for an experiment, and ran well prior to the completion of the experiment, at which time it was returned to MPL. The other system was kept at MPL pending sponsor readiness for deployment. It was allowed to run continuously, and ran well. In addition, under the options, we were funded to develop a night cloud algorithm based on detection of the contrast between the signal from stars and their background. The concepts had been developed under funding from another sponsor (Shields et al. 2002), and under Delivery Order 6, this concept was expanded to handle moonlight, and converted to a fieldable C-code. The appropriate geometric calibrations and background were extracted for the Unit 12 running at the SOR site, and the algorithm was installed and provided reasonable results. Under Contract N00014-01-D-0043 DO #4 (May 2001 – September 2006), we continued hardware, algorithm, and analysis work (Shields et al. 2007). We fielded WSI Unit 14 at a site in Virginia, and supported this deployment as well as the continued operation of Unit 12 at the SOR site. Although Unit 14 ran flawlessly on the earlier deployment and in extensive tests at MPL, we had more problems than normal at the Virginia site, but were able to keep it operational much of the time. The Unit 12 system generally operated well, although it required extensive repair following a lightning strike, and somewhat more repairs than normal during the period after that. We were pleased to be able to return it to operation after such an event. We also began evaluation of how to build a new WSI with currently available technology, and how to simplify the system for increased robustness and decreased cost. Also under DO #4 the day algorithm at the SOR site was updated to run in real time, and include a horizon and occultor mask, and the clear sky background was updated. A stand-alone version of the algorithm was written, and under funding from another sponsor, an extensive data base was processed and analyzed to extract CFLOS statistics. This work allowed us to determine the strengths and weaknesses of the daytime cloud algorithm. The night algorithm was further developed by determining appropriate upgrades and inputs required for the SOR site, which was more impacted by anthropogenic light sources than previous sites. The new night algorithm was installed to run in real time at the SOR site. This was the first time that we had a night algorithm installed so that it could provide processed results in real time. 1 Toward the end of this prior contract, a funding increment was received that partially funded the new contract (DO #11), and partially funded the existing contract (DO #4). The Statement of Work (SOW) for the new contract, DO #11, reflected the priorities at the time of the funding increment. However, we were asked to use the options under the existing contract, DO #4, to begin the work. As a result, we were able to provide an extensive analysis of IR systems and their pros and cons for this program. Most of this analysis concentrated on Long Wave IR (LWIR) systems in the 8 – 12 um wavelength region, because our analysis showed that Short Wave IR (SWIR) systems near the 1.6 um wavelength region would not be adequate for our purposes, and Mid Wave IR (MWIR) systems near the 3 – 5 um wavelength region had disadvantages with respect to the LWIR systems. The LWIR analysis of theoretical performance and images we had access to showed that the LWIR system probably will not detect most clouds near the horizon in normal haze environments, and will have difficulty detecting high clouds over many parts of the sky under many conditions. This analysis, as well as the other work done under DO #4, is presented in Shields et al. 2007. 3. Statement of Work The Statement of Work for the Delivery Order #11 is given in italics below. Primary Task: The contractor shall, unless otherwise specified herein, supply the necessary personnel, facilities, services, and materials to accomplish the following tasks within a one year period following receipt of funding. 1. Upgrade and Evaluation of Current WSI Capabilities: Evaluate night and daytime cloud decision algorithms. SOR will supply WSI raw images and corresponding lidar data to MPL to use in this evaluation. Implement upgrades to stand-alone versions of the cloud algorithms. Document significant findings and prevent (present) a review of recommended algorithm improvements for use in the field to achieve accurate 24/7 cloud assessment. 2. Analysis of Infrared (IR) Sensors: Analyze sensor performance and anticipated impacts of cloud and sky background flux levels for IR regions. The analysis should result in an estimate of whether an IR sensor would improve the capability of determining cloud/no cloud whole sky decisions. SOR will furnish to MPL specifications of IR sensors available for use, although MPL is not restricted to using these sensors. This effort will provide a comparison of the expected performance of an IR system in comparison to the performance of the existing visible systems. At the end of this task, MPL shall document significant findings and make recommendations on IR system development. 3. Evaluation of System Upgrades: MPL shall conduct an evaluation of overall system upgrades to make the existing visible unit more robust and to simplify hardware and software. Optional Tasks: 4. Coordinate with the sponsor regarding the most appropriate tasks and estimated costs for further development. Tasks are anticipated to include one or more of the following: 4a. Provide algorithm or hardware support of the existing SOR WSI units. 4b. Begin work toward designing and building an IR WSI unit. 4c. Begin work toward building multiple Visible WSIs units. 1 5. Provide personnel trained in the WSI and its capabilities to address these tasks to the limit of funding provided under the optional budget. These tasks may include analysis, software development, documentation, minor hardware development, and other tasks related to the WSI and are mutually agreed upon by the sponsor and by MPL to be appropriate. 4. Funding Increments and Optional Tasks The funding was sent from the Air Force to ONR in two funding increments, and the optional tasks were defined as these funding increments were received. An initial funding increment was sent to ONR in a MIPR dated 10 May 2004. Some of these funds were allocated into the previous Contract N00014-01-D-0043 DO #4, and the remaining funds were put into the new Contract N00014-01-D-043 DO #11. The new contract was received at MPL in September 2004. This funded the primary task budget, and the priority was to work on the tasks listed in the Statement of Work Primary Tasks section. As noted in Section 2, we were able to complete portions of the second item in the SOW under the earlier contract with this first split funding increment. During this time, SOR decided to field 3 sites with WSI's, in support of a broader program. They did not have funding to have us build and deploy new WSI units, however some older WSI units that had been built by MPL under a DOE program became available to MPL. It was decided that these should be refurbished for use at these test sites, and refurbishment was begun under this program. This included hardware refurbishment such as replacing worn or broken parts and integrating a second computer for real-time processing. Extensive software rewrite was also required. Support of logistics planning for deployment was included. We were given permission to start this work, as part of Primary Task 3. The second funding increment was originally issued January 2004; errors in it were corrected in March 2004, and it was received in May 2004. This increment funded most of the optional task budget; the remaining funding was never sent. Under this MIPR, the priorities were to 1. Continue with the WSI system, software, and algorithm refurbishments, to within the level possible within this funding. 2. Work toward assessing the accuracy of the algorithms and their application for multiple sites. 3. To the extent feasible under funding, support the maintenance of the fielded units, particularly the unit at the SOR site. These three items may be considered part of Optional Task 4a from the SOW. Optional Task 4b, which related to building an IR system, was not funded, because the IR analysis indicated that IR systems were unlikely to do a better job at meeting project cloud detection needs. Optional Task 4c, related to building new WSI systems, was not funded because the older WSI systems became available for refurbishment. 5. Major Deliveries and Documentation These tasks are somewhat open ended, however major progress was completed in all areas, as required to meet the Statement of Work. We would like to document the major deliveries and documentation for these tasks. In the later sections, we will provide an overview of the more significant advances related to general capabilities. Additional documentation is provided in the technical memoranda listed below and in the reference section 12.1. These technical memos can be provided to the sponsor upon request. In addition, this work was presented to sponsors at several meetings. The available Power Point files are listed in the References section 12.1. 5.1. Algorithm and Ground-truthing Developments Primary Task 1: This task is directed primarily toward evaluating the day and night algorithms, providing upgrades in the stand-alone version of the algorithm, and presenting the results. A technical discussion of the progress with the Day and Night cloud algorithms is given in Sections 6 and 7 respectively. A discussion of the actual deliveries is given in this Section 5, along with a discussion of other minor areas addressed under this category but not discussed in Sections 6 and 7. There were several deliveries related to this task. At a meeting in December 2004, we presented the results of analysis of the Day cloud algorithm done primarily under the previous funding (Shields et al. 2007). At this meeting, we were asked to concentrate first on two aspects of the night algorithm, namely how we could ground-truth the results, and how we could move to a high resolution night algorithm in the future. In response, we developed and presented methods for ground-truthing of the night cloud algorithm based on the beam transmittance to bright stars, and presented this information in the April 2005 meeting. Also, we extracted sample backgrounds and developed the concepts for a full resolution night algorithm, as documented in Memo AV05-012t, and presented at the same meeting. The results will be discussed in Section 7, and the Power Point presentation may be provided to the sponsors upon request. At the sponsors' request, we also provided a full review of the concepts behind the day and the night algorithms at a June 2005 meeting. We were asked to concentrate next on processing a set of both day and night 1-minute resolution data from SOR from July and August 2005, so that it could be used by SOR, TASC, and MPL for forecasting studies. This required updating the stand-alone algorithm and utility programs for compatibility with the SOR systems, and integrating the night algorithm into the code, as documented in Technical Memos AV05-033t. The update to the night geometric calibration is documented in Memo AV05-034t and AV05- 035t. The processing required extracting the day clear sky libraries and other inputs required to run the data. We processed approximately 15,000 day images and 3700 night images. The processing is documented in Memo AV05-032t, and the format of the results is documented in Memo AV05-031t. This memo and the data were delivered to SOR and TASC on 11 October 2005. This was our first opportunity to evaluate the night algorithm results on a substantial data set. During this time, we also worked on methods for better ground-truthing of both day and night cloud algorithm results, as documented in Memo AV05-021t. As indicated in the SOW, we had originally intended to use lidar for this purpose. However, initially the lidar data were not available to us, and then when they were delivered, they were in a form we were not yet in a position to use. Eventually, it was decided by our sponsors that the other work discussed below was more important at the time. Instead of the lidar comparison, a program to enable systematic assessment of algorithm accuracy, called SORCloudAssess, was developed and applied to this test-bed data. The results for the night algorithm are documented in Memo AV05-037t, and the results for the day algorithm are documented in Memo AV05-038t. Although this program relied on visual assessment of the results, it at least provided a consistent and systematic way to assess the results. We found the day algorithm to provide correct results, based on visual assessment of the imagery, approximately 97.4% of the time. (If sunset is not counted, the results were accurate 98.4% of the time, based on the visual assessment.) At night, the results in the line of sight were accurate approximately 87.5% of the time. This lower value is partly because the night algorithm was a moderate resolution algorithm (based on contrast). An additional test, which evaluated whether the answers were correct in the general region of the line of sight yielded an estimated accuracy of 95.0%. These results were reported in the December 2005 meeting. We also began work to develop forecast testing techniques, as documented in Memo AV05-036t. Programming on this technique was begun, although we did not finish the work due to other higher priorities. We reported on this work at the February 2006 meeting, and also recommended that rather than predicting yes/no whether the line of sight would be cloud-free, we provide a probability, and thus an ability to rank each site. In the December 2005 meeting we presented the results of a preliminary analysis of night data taken in Virginia, in a very bright urban location. Following an initial analysis documented in Memo AV04-054t, we found that there was too much scattering of city light to easily detect stars over much of the image. As a result, we adjusted the instrument to acquire spectral data at night. An analysis of this data showed that the clouds are very easily detectable visually in the image, and the typical cloud signal is more than a factor of 10 brighter than the clear sky signal. We concluded that the raw data show that an algorithm could be developed for this very bright location or similar locations if needed. The next algorithm question we were asked to tackle was to evaluate whether the day algorithm would work in very hazy locations, using Virginia data as a test bed. For this processing, we updated the algorithm to enable using the Near Infrared (NIR) data, which are expected to do better in hazy environments. Data from April 2005 were processed, because it was the closest we had to summer, when the conditions should be worst. The results were presented in the Feb 2006 meeting. (We were not asked to deliver the data.) The data from 1-hour intervals were processed, which yielded a data set of 307 images – perhaps not enough to be statistically significant, but enough to determine whether the algorithm works reasonably in hazy environments. Based on the systematic visual assessment using the SORCloudAssess program, we found that if the sunset data are not included, the data appear to be correct about 98.1% of the time in this test set. The processing and results were documented in Memo AV06-018t. During this time, we also worked on getting the algorithms ready to field with the instruments. This work is discussed in Section 5.4. This work completes Primary Task 1 in the SOW. 5.2. Analysis of Infrared Sensors Task 2 under the SOW was to analyze the possible uses of IR systems for this cloud application. Most of this work was completed under the previous contract, and is reported in detail in Shields et al. 2007 and in Memo AV07-026t. An overview of the results is presented in Section 8 below. The presentation was prepared under this contract, and presented in December 2004. This work completed Primary Task 2 in the SOW. 5.3. Evaluation of System Upgrades Task 3 under the SOW was to evaluate possible system upgrades. We evaluated several upgrades, including replacing the Accessory Control Panel with a much simpler (but slightly less flexible) system. We evaluated the current cameras and lenses that could be used, and their impact on required cooling. Some of this work was reported in the December 2004 presentation, and is discussed in Section 9.1. This work completed Primary Task 3 in the SOW. Following the decision to refurbish the WSI which became available, we concentrated more of our time on preparing these units, as discussed in the later sections. 5.4. Refurbishment of WSI Units for Field Deployment Support and Maintenance Support Early in the contract, we were asked to include, under Task 3, to begin refurbishment of existing WSI systems for support of SOR field deployments. The Optional funds were also directed to a large extent toward this task, as well as toward maintenance. The system maintenance is documented in Section 9.2. The hardware and software refurbishments are documented in Sections 9.3 and 9.4 respectively, and the site logistics support is documented in Section 9.5. A brief overview is provided in this section. 5.4.1. WSI Refurbishment The first two units for refurbishment were received in-house in February 2005, as documented in Technical Memo AV05-004t. The retired unit that was already in-house is documented in Memo AV05-020t. Other units that can serve as test units and spare parts are documented in Memos AV05-025t, AV05-028t, and AV06-022t. These instruments were originally built for the Department of Energy (DOE) and used for many years at a variety of sites. They were retired beginning in late 2004. We proposed that the retired instruments be given to MPL for use in other programs, and this was done. Typical refurbishment tasks included disassembly, replacing worn components such as the coolant tubing, replacing any failed components such as arc drive motors, and getting the cameras tested and purged at Photometrics. We also replace filters as necessary, replace the shutter, replace cables as required, clean the system, replace optical domes, and re-label connections. The occultor shades were replaced with shades appropriate to the new locations. The systems did not include processing computers for real-time algorithm processing. We added processing computers, updated the GPS system, and also added components such as and external 300 GB drive and a power switch to enable automatic reboots. The systems are refocused and radiometrically calibrated. The electronics such as the meters and the occultor arc drive control are recalibrated. The systems are thoroughly tested prior to deployment. The first system entered test in June 2005. We had hoped to deploy the first system under this contract; however the sponsor-provided site was not ready in time. Both the second and the third units were partially refurbished under this contract. These developments were reported in the talks on an ongoing basis, and are further discussed in Section 9.3. 5.4.2. Software Upgrades for the Refurbished WSI's The refurbished WSI systems have quite different control computer systems, because they run on DOS. This is because the Photometrics Camera used in those systems, the Series 200, requires a DOS operating system. SOR evaluated the cost of upgrading the cameras to the Series 300, and upgrading the computer systems, and decided to stay with the DOS system. The code was somewhat different in terms of data acquisition – for example, they acquired a full data set every 6 minutes, and red or open hole filter every 2 minutes, whereas for the SOR task, we needed every 1 minute during the daytime, and 2 minutes at night. These systems also did not have any real-time algorithms. A very extensive rework of the software was accomplished, both in order to adapt the system to the SOR needs, and in order to make the system more robust and easier to perform routine QC. These developments were reported in the talks on an ongoing basis, and are further discussed in Section 9.4. 5.4.3. Logistics, Testing, and Deployment As part of the options, we were asked to prepare the instruments for deployment. Preparation of the actual sites was being done by SOR and by another group under contract to SOR. We worked with these two groups on the deployment and site logistics regarding the required site support and deployments during early 2005, and the logistics were mostly worked out by May 2005. 5.4.4. Field Repairs The field repairs are discussed in Section 9.2. The SOR unit, Unit 12, which was originally delivered in January 1999, required several repairs, but it was kept running most of the time. The Virginia unit, Unit 14, which was built in 2000, was kept running through most of 2005 and 2006, however there were not sufficient funds to repair the camera when it failed in March 2006. We believe that the work documented in Section 5, and further documented below, meets and exceeds the requirements given in the Statement of Work. 6. Day Algorithm Upgrade and Analysis As discussed in Section 5, there was much more emphasis, from our sponsors, on evaluating and checking the day algorithm than on developing it during this contract. While Section 5 discussed the deliveries and time-lines, this section will provide more detail on the results. The integration of the algorithms into the field software is discussed in Section 9.4. 6.1. Previous Day Algorithm Results Our first in-depth analysis of the day algorithm was done under funding from another sponsor, and was reported to SOR in December 2004. This analysis was done with data from a site in Oklahoma, and the results are reported in Shields et al. 2007. In general we were quite pleased with the results. The algorithm handled opaque clouds and thin clouds quite well. We did find that there was enough variation in haze and its impact on the algorithm, that it would be helpful to develop an adaptive algorithm to handle haze. An adaptive algorithm had already been developed for the Daylight Visible/NIR WSI (Day WSI), described in Shields et al. 2003d. This algorithm for the Day WSI uses the NIR/blue ratio, which is less influenced by aerosols in very hazy environments, and also makes adjustments for the aerosol amount. At the meeting in June 2005, we gave a fairly detailed overview of the Day/Night WSI (D/N WSI) algorithm used during the daytime, including these updates to the Day WSI algorithm. 6.2. Algorithm Results for Data from the SOR Site To further evaluate the day algorithm used with the D/N WSI (which was based on the red/blue ratio, and did not include the adaptive feature), we processed a test-bed of data from July and August 2005 taken at 1-minute intervals at the SOR site. It should be noted that at this time we programmed a version of the adaptive algorithm, but did not find the results consistent enough to use routinely. It has not yet been determined whether this is due to software bugs, less than optimal input parameters, or a need for a more sophisticated approach than we used. A set of 15,000 images were processed and analyzed. The associated documentation memos are listed in Section 5.1 and in the Reference section. Sample results are shown in Figures 1 – 4. In each of these figures, the raw image on the left is the red image. The cloud decision image on the right shows the results of the cloud algorithm. In this image, black is the color code used for “no data”, blue is “no cloud”, yellow is pixels the algorithm has identified as “thin cloud”, and white is "opaque cloud”. The texture within each of these clouds is only to help the analyst assess the images. (For example, in the opaque cloud regions, colors ranging from grey to white indicate how much the ratio exceeded the opaque threshold. Similarly, the coloration within the blue and the yellow regions have to do with variations within the clear and thin cloud determination schemes.) Because the WSI looks up, the directions on these images are not the same as on a map. East is to the right, but North is at the bottom of the image. (Visualize the scene lying on your back with your toes to the north.) Fig. 1. Raw red and processed cloud decision from SOR site, 26 Jul 05 2200 SEA Fig. 2. Raw red and processed cloud decision from SOR site 9 Jul 05 1700 We felt that the day algorithm did quite well for this SOR data set illustrated in Figures 1 through 4. Figure 1 was chosen because it is fairly typical, and it also illustrates the ability of the algorithm to identify opaque clouds properly in the solar aureole region and near the horizon. Note particularly the small clouds that have been correctly identified near the horizons to the north and south, as well as the thin clouds in the south-east of the image. Figure 2 was selected to demonstrate the ability to identify small clouds, as well as optically thin clouds (shown in yellow). Figure 3 was selected to demonstrate the ability to identify both dark and bright clouds (brightness can be seen in the raw image; both were shown in white in the cloud decision image). Note particularly the dark cloud to the south of the sun, in contrast with the bright clouds to the SW and NE of the sun. Figure 4 was selected to demonstrate the ability to identify high thin clouds. In Figure 4 there is a lower layer of opaque clouds (shown in white), and a high very thin layer of clouds (shown in yellow), particularly to the west of the zenith, that was properly identified. (The presence of these thin clouds in the raw images is easier to see when the image are viewed in an image processing program than in a word document such as this report.) Fig. 3. Raw red and processed cloud decision from SOR site 26 Jul 05 2100 Fig. 4. Raw red and processed cloud decision from SOR site 31 Jul 05 1500 6.3. Systematic Evaluation of the SOR Results Although general impressions are helpful, we wanted to develop a method to systematically analyze the algorithm results. We have had many opportunities over the 12 Missing Page 13 We assessed data taken at hourly intervals from the entire test bed data set, thus assessing 310 images, or 3410 ROI's. Overall, we found that the ROI results were evaluated to be correct 97.4% of the time, with a 99.0% rate at the zenith, and a 95 – 96% rate near the horizon. Later evaluation sorted out the sunrise/sunset cases, and with these removed, the average results over the full sky were evaluated to be correct 98.4% of the time. Results were estimated to be correct over 90% of the sky 94.5% of the time. The distribution of the results as a function of the ROI position is shown in Figure 6. 98.6 97.7 94.9 | 97.8 98.3 99.0 97.6 94.8 96.5 97.7 97.9 Yes 90% No Exit? Fig. 6. Fraction of correct answers, in percent, for each ROI for SOR Day Test Bed Processing We also evaluated when the errors occurred. Of the 310 hourly images examined, 293 (94.5%) passed the test regarding whether it was evaluated to be correct over 90% of the image. Ten cases (3.2%) failed because the algorithm is not yet optimized for dawn or dusk. Six cases (1.9%) failed in identifying thin cloud. One case (0.3%) had a problem in the raw data. Of the 23 days evaluated, 6 mornings and 2 evenings had problems. The other sunrise and sunset cases were Ok. Examples of the few failures are shown in Figures 7 – 9. Figure 7 shows an example where the thin cloud identification was poor, and not all of the thin clouds were identified. (These may be seen as the light blue regions that are shaped like cloud but not colored yellow.) Figure 8 shows the same example when run with the adaptive 14 algorithm. In this case, the results of the adaptive algorithm were excellent (the yellow regions correspond well with the thin clouds apparent in the raw image). Other cases run with the adaptive algorithm were not good, and we have not yet fully debugged this feature, due to other sponsor priorities. Figure 9 shows a poor result at sunrise. Here the enhanced redness of the sky associated with the long path length results in a higher red/blue ratio, and results in the incorrect identification of portions of the clear sky as cloud. 2081201 RED RW Rad LA 52081701.cld Cid Image Yes 90% ot: No Exit? Fig. 7. Example showing failure to detect thin clouds 52081201 RED ROW Rad 52081 701.cid Cid Image ti Yes 90% * No * Exit? Fig. 8. Same image as Fig. 7, but run with the adaptive algorithm turned on 15 5711200 RED R Red - - 52111300.cld Cid Image Yes 90% * No Exit? Fig. 9. Example of failure of algorithm at sunrise 5190222 RED R Red 51902200.cid Cid Image Yes 90% NO Exit? Fig. 10. Typical results showing ability to detect clouds near the horizon At a previous meeting, an analyst from another group had stated the opinion that the WSI algorithm would never be able to handle either cloud motion or clouds near the horizon. As a result, we looked at these two categories specifically. We found only one ROI out of the 3410 cases that was affected by cloud motion. Except in the case of very fast moving clouds with very large changes in radiance, the MPL algorithm should not have a 16 problem. The algorithm was originally designed in the early 1980's) so that the location of the clouds in the cloud decision image will be the same as the location of the clouds in the red image. If cloud motion becomes a problem, we are aware of algorithm upgrades that could be made to address this. Regarding the near-horizon, as noted earlier, we found that the results were not quite as good at the horizon (95-96% estimated accuracy), but in general they were quite good. A typical sample is shown in Figure 10. This reasonable result at the horizon is partly due to an “indeterminate cloud” category that will be discussed later. The indeterminate cloud logic was added to the algorithm in the middle 1980's and has worked well in a variety of environments. As noted earlier, we programmed a version of the adaptive algorithm for the Day/Night WSI. In isolated cases, such as Figures 7 and 8, it provided significantly improved results. However, we were not satisfied with the results in general, and did not have the time at that point to determine the source of the problems. As a result, the analysis discussed here was all done with the adaptive algorithm turned off. The work with the SOR test bed data was reported in the December 2005 talk, and further examples are provided there. 6.4. Evaluation of Data from the Hazier Virginia Site Because the SOR site is a clear desert air environment, we expect it to be a relatively easy environment for the algorithm. We therefore decided next to process a test-bed of data from the Virginia site, which can be extremely hazy, especially in the summer. For this analysis, we processed data from April 2005, as it was the closest month we had to summer, when the problem should be worst. Hourly data were processed, and 307 images were analyzed. The results were documented in Technical Memo AV06-018t and reported in the February 2006 meeting, and more details are presented there. For this Virginia data set, using the SORCloudAssess program, we estimate the overall algorithm accuracy, if sunrise/sunset is excluded, at approximately 98.1%, with one caveat discussed below. The full image was estimated to be over 90% correct in 96.7% of the cases. For this site, we went ahead and programmed the NIR/blue algorithm. The NIR filters were installed in the WSI systems beginning in 1997, because we felt that they should enhance the contrast between the aerosols (i.e. small droplets with size near 0.1 um) and the thin clouds (large droplets with size near 1 - 10 um. The aerosol scattering decreases quickly as a function of wavelength near the red and NIR filter wavelengths of 650 and 800 um, with the rate of decrease depending on droplet size distribution and characterized by the Angstrom coefficient. The thin clouds, having larger droplets, scatter effectively at the longer wavelengths as well as the shorter wavelengths. The theory is shown in more detail in the July 2004 talk. 17 We had not yet optimized the adaptive algorithm, because we were asked to delay the algorithm upgrade in order to assess the Virginia data. During the processing, we found that there were 4.5 days with very hazy results, and we simulated the adaptive algorithm by changing the adaptive parameter for these 4.5 days only. Several examples of the results for the Virginia test bed data are shown in Figures 11 through 15. These images are shown in the same format as those from SOR, however the raw imagery in this case are the NIR images. Fig. 11. Raw NIR and cloud decision from Virginia site 15 April 05 1900 Fig. 12. Raw NIR and cloud decision from Virginia site 12 April 05 1500 Figure 11 is a fairly typical example. It identifies both the thin and the opaque clouds of varying size and location in the sky quite well. Figure 12 was chosen to demonstrate the 18 detection of cirrus clouds and contrails. Again, the results are quite good. However, in this image, there are some yellow speckles in the blue sky regions. This is due to the spectral dependency of the fiber optic taper used in the camera, and should be easy to remove with a calibration correction. This calibration correction is one of the features we hope to add to the algorithm in the future. Figure 13 illustrates a result with broken clouds, and we feel this result is quite good. Fig. 13. Raw NIR and cloud decision from Virginia site 02 April 05 2200 Fig. 14. Raw NIR and cloud decision from Virginia site 18 April 05 1600 Figures 14 and 15 were chosen from the 4 1/2 days with more haze. In these images, the horizon and the solar aureole have been identified by the algorithm as “indeterminate”. These regions are colored grey in the cloud decision image. The indeterminate regions are where the algorithm cannot distinguish between clear sky and cloud due to the haze. 19 The algorithm automatically detects that the distinction between haze and cloud for this region is not large enough, and identifies the pixels accordingly. Visually, this is similar to a case where the haze is so heavy and so white that one cannot see the clouds, except that by using the NIR filter, the WSI can “see” clouds that may not be seen in the haze by the visual observer. Fig. 15. Raw NIR and cloud decision from Virginia site 19 April 05 1500 We should add that the haze does attenuate the transmission in the visible. However, because we are more interested in transmission at wavelengths in the Short Wave IR near 1.6 um, we normally do not want to identify this haze as a cloud. However, it may be that the haze could somewhat affect the transmission in the SWIR. As a result, in the future, we plan to add features to the algorithm to enable it to identify heavy haze cases, and potentially use this information in assessing the relative ranking of different sites. The distribution of results as a function of ROI is shown in Figure 16. Although there are obvious improvements we can make with the algorithm, we were very pleased with these results. A summary of these daytime results for both the SOR and the VA test bed data sets is shown in Table 1. Table 1 Summary of Estimated Accuracy for the Day Cloud Algorithm Using SORCloudAssess For the SOR and Virginia Test Bed Data Sets Region SOR Results VA Results Overall 98.4% 98.1% Zenith 99.5% 97.5% Horizons 98.5% 98.8% | 90% Correct? 10% Correct? | 96.7% 96.7% 98.0 98.3 98.8 | 99.1 97.4 97.5 98.0 96. 1 98.9 98.8 97.8 Fig. 16. Fraction of correct answers, in percent, for each ROI for Day Virginia Test Bed Processing 6.5. Results of a Comparison Test with a Ceilometer During this time, under funding from another sponsor, we had the opportunity to run a WSI beside a Vaisala CT-25k ceilometer, designed to measure clouds to approximately 25k feet, as documented in Memo AV05-010t. This comparison was also valuable for the SOR program in two respects. First, it let us evaluate whether the WSI can detect all the clouds that a ceilometer can, and second it let us evaluate whether a ceilometer system provides sufficiently accurate cloud fraction estimates to be useful for this program. The ceilometer has the advantage that is an active system, i.e. it puts out a light beam, and the backscattering of that light is sensed. This could potentially make it more sensitive. The ceilometer has the disadvantage that it only senses in the one direction. As a result, the ceilometer estimates cloud fraction from a temporal average over the last 30 minutes, weighted more heavily for the last 10 minutes. In general, we found that the ceilometer and WSI agreed well for cases where the imagery showed it to be either overcast or clear. However, we found that the cloud fraction estimated by the ceilometer was very poor in other conditions. Examples are shown in Figure 17. In Figure 17, in example a, the ceilometer estimated sky cover to be 0 octals, or clear, even though the actual cloud cover is significant. In cases b and c, the ceilometer estimated sky cover to be 1 octal (1/8 cover), even though the clouds cover well over half the sky. In case d, the ceilometer estimated sky cover to be 3 octals, and it is overcast. In general, we found that if the sky is clear, the ceilometer correctly reports that result, but if the sky cover was scattered or broken, it was very often significantly low. There were many cases similar to those in Figure 17 where the clouds either stayed in one quadrant and were not seen by the ceilometer (as in case a) or where the clouds changed too quickly for a temporal average to be effective. As a result, we feel that a ceilometer would not be adequate for forecasting that a given line of sight may soon be blocked, since it does not see the clouds outside its own line of sight. Stasugarcat ization clous dane keight treet),23700 C104. laze velcree909 cloud 218 Keitt trees) 200 Ce do oktas) Keight of cloud (eel Caoud eos corbes Keight of cloud teet) Seger caoud as COKLAS) Keight of cloud Teet) Vàºàng với t (8H. Weight of cloud (test) 13500 eco 20000 20-12-2014 062863%DORZO DAS 10-12-2004 1426 LODRZDA 12-16-2004 1428821300D DAT 16-21-2004 04209 2010R2D). DAT a) Ceilometer 0/8 b) Ceilometer 1/8 b) Ceilometer 1/8 c) Ceilometer 1/8 d) Ceilometer 3/8 Fig. 17. WSI Images with simultaneous ceilometer report, examples where ceilometer results are not valid Regarding the question of whether the WSI is able to detect all the clouds the ceilometer detects, we did a detailed examination of the data taken every 10 minutes on 4 of the days in the data set, and we found only one case in which the WSI did not appear to detect a cloud detected by the ceilometer. On closer examination, the cloud could be seen in the WSI image if it was enhanced more. We did not have sufficient funding to set up the cloud algorithm, so it's possible that in this one case, the algorithm might have missed the cloud, even though it can be seen in the raw imagery. We did find several cases where the ceilometer did not detect the clouds even within its line of sight, due to the height limitation of the lidar in the ceilometer. That is, the WSI was often able to detect clouds the ceilometer could not detect. 6.6. Summary of Day Cloud Algorithm Results and Future Plans As a result of the day algorithm studies conducted under this contract, we concluded that the WSI algorithm is generally working quite well in both relatively dry environments and relatively hazy environments. However, we would like to add adaptive algorithm features to identify the enhanced haze cases and adjust for them. There are additional calibration corrections that can be added. Also, we plan to do more ground-truthing as additional information becomes available, and also work on calibrating the algorithms for optical depth. In addition, at the present time setting up the algorithm for a given site is very time intensive. We made quite a bit of progress during this interval in making these programs 22 easier to set up, but much is still needed in this direction. We need to address improving algorithms in the sunset/sunrise regime. And perhaps most importantly, as systems are deployed, we need to set up the algorithms for each of the sites, and evaluate how well they are doing 7. Night Algorithm Developments and Analysis Under the previous contract, a first-version night algorithm had been developed, and was installed at SOR running in near-real-time. This algorithm uses a very accurate geometric (angular) calibration, and detects the presence of approximately 2000 stars in a clear sky image. It uses a Gaussian best fit for the point spread function (PSF) around the star, and bases the detection on the contrast between the signal within the PSF and the background signal. The threshold contrast depends on location in the sky, as well as moon condition, and cloud category (opaque or thin). The algorithm is a moderate resolution algorithm, making an assessment in each of 356 regions or cells. These cells cover an increment of 5° in zenith angle. From zenith angles of 30 to 90°, the azimuth increment for the cells is also 5°; for zenith angles of 15 to 30° the azimuth increment is 15°, and for zenith angles of 0 to 15º the azimuth increment is 90°. Typical cloud decision images were presented in Shields et al. 2007. In addition, at that time we had developed the ability to determine an approximate transmittance value for the earth-to-space transmittance at each star. A sample of this result is also shown in Shields et al. 2007. The methods for extracting the measured irradiance from the data are documented in Shields et al. 2004a, and Shields et al. 4007, and other references listed in these reports. Similarly, the methods for determining the inherent irradiance of each star in the WSI passband are documented in these references. The transmittances are basically derived from a ratio of the apparent irradiances (ground- based measured) to the inherent irradiances (outside the atmosphere, in this case calculated). Following this work, we began working toward a night algorithm based on the earth-to- space beam transmittance. For this work, we began developing software to apply the radiometric calibrations to the night imagery, and then began evaluating the accuracy of the transmittance extraction. However, at the December 2004 meeting, we were asked to put two other items higher priority. The first of these was development of ground-truthing methods, and the second was development of concepts for a high resolution night algorithm. These results were reported at the April 2005 meeting and are discussed in Sections 7.1 and 7.2. It should be noted that much of the work reported in Sections 7.1 and 7.2 was done under the Boeing contract mentioned in Section 1, however the work is included here, as it was sponsored by SOR, and has not been reported in detail previously. The rest of the work discussed in Section 7 was done under the DO #11 that is the subject of this report. 7.1. Ground-truthing using Bright Stars TI Regarding ground-truthing, our goal is to have some means to verify that the cloud algorithm results are reasonably correct. We suggested that a few of the brightest stars not be used in the algorithm, but that their transmittance be used to provide a somewhat- independent check of the algorithm result in the region near the star. To do this, we first reviewed the current transmittance calculations, made some improvements to the logic, and also removed most of the variable stars. Next, we looked at how well the measured star irradiances compared with the inherent star irradiances for individual stars. Figure 18 shows a comparison between the measured star irradiance and the inherent star irradiance as a function of the zenith angle of the star Alnath. Methods similar in concept to Langley plots were developed to extract the aerosol transmittance. We found that the average aerosol transmittance was 91, with a range from .89 to .93. That is, a typical aerosol transmittance on a reasonably clear night for the SOR site in the WSI open-hole passband is near .91. For this study, we used a value of .91 for all stars. As discussed in Section 7.4 we later improved the value for SOR based on evaluation of more stars, and later work shows that it is site-dependent, as would be expected. The data corrected for aerosol transmittance are shown in Figure 19. ceniin nogle van. Meosured pectral irredionce Correspon: TO Nncit Location ceniin linge Vs. Measure spectral isragionce Coriesach ging To Alr:atm Luoction Merosol Correction * None Aprnical Cirrmtip je 8.90 070-9 TT . Meysu'd Spectral liradianco (yim- (www) ;)jo pod 10.02:45 Den507, C opy: -3 .. 18 ........ 1... 2*10 .... 20 46 20 60 grams) bo i enit! Angie (idecrees, Zenith Angle Fig. 18. Alnath measured spectral irradiance on 11 nights. Fig. 19. Alnath spectral irradiance corrected for aerosol extinction. In Figure 19, the dashed line is the calculated inherent irradiance for the star. Interestingly, we found a residual offset between the measured and inherent star irradiances. The mean offset for this limited data set was near 1.37, and the values ranged from 1.06 to 1.7. We are not yet certain of the cause of this offset. A fixed offset for all stars could be caused by a problem with the WSI calibration. From the average offset, we believe there may be an issue with the calibration, and we hope to investigate this in the future. The variation in the offset from star to star could be caused either by errors in the method for extracting the measured irradiances, or by uncertainties in the library magnitude or by the use of color temperature. We believe it is most likely to be the latter. The standard method of characterizing the spectral output of the star by color temperature is of course approximate. We estimate a spectrum based on the color temperature, and use this to compute the star inherent irradiance for our spectral band. This has some uncertainty, and may be the cause of the star-to-star variation in offset. We decided that it will be necessary to develop a program to automatically extract the correction constant for each star we use in the star library, as will be discussed in Section 7.3. There is also some variance, even on a clear night, in the apparent irradiance corrected for aerosol transmittance and for the star offset. Typical STD values range from 5% to 15%. We evaluated the data as a function of the background brightness, and found that the uncertainty is independent of background radiance. This indicates that the offset has little to do with the ability of the code to correctly extract the background. We also found no correlation between the offset of a star and an adjacent star, indicating that the variations are not due to small variations in the clear-night transmittance. Our system is very well focused, with a PSF of less than a pixel. This may mean that under-sampling of the Gaussian is the cause of the uncertainty. We have not yet isolated the cause of this variance. For the bright star ground-truthing, we selected those bright stars with a Standard Deviation (STD) in the aerosol-corrected irradiance of 5% or less. To choose the ground-truth stars for this test, we started with the brightest stars, and selected stars that had multiple cloud-free appearances near the zenith, and that also had a STD of 5% or less from the clear night set. We next evaluated a number of images to see if the corrected bright star transmittances appeared to be reasonable. Examples are shown in Figures 20 - 25. These plots have been corrected for the offsets; however they have not been adjusted for aerosol transmittance. That is, the transmittances include losses due to both clouds and aerosols (and molecular losses). HASOR Doto_WSK 19991.CZ14190450340_clr.cal HE SOR\Data_WSN1999 08\16191670510_CLR.COM 0.95 820.87 0-90 0.87 0.950 84 4,01 Fig. 20. Clear night, 14 Feb 99, 0340 Fig. 21. Cloudy night, 16 June 99 0510 25 Figure 20 shows a clear night. The extracted transmittances averaged .89, which is reasonable for a clear night in this wave band at the SOR site. The average variation from this average was .04, and the maximum variation from the average was .07. H\SOR\Data_IIS 1999\C5\16191670710_CLR.cal HASOR\Data_WSN1999 COV18191670730_CLR.cal 0.69 0.16 0.82 .0.15 0.885 Fig. 22. Broken clouds, 16 June 99,0710 Fig. 23. Broken clouds, 16 June 99 0730 HESOR Data_WS 19991.06116191670840_CLR.COM HA SOR\Data_WSK 1999\06.16197670850_CLR.cal ,1.58 Fig. 24. Broken clouds, 16 June 99, 0840 Fig. 25. Broken clouds, 16 June 99 0850 A very cloudy case is shown in Figure 21. Here, the areas that appear to be thinner are identified with transmittances of .10 and .14. Areas with thicker cloud had 3 stars that were not detected, and one star that was detected with a beam transmittance of .01. Figures 22 – 25 show a sequence of images with a fairly stable, but thinning, broken cloud field. When this sequence is viewed on the computer monitor, we can see that the cloud field in the upper right quadrant of the image become thinner throughout this period, because we can see more and more stars through the clouds in the image. The transmittance in this cloud region slowly changes from about .16 in Fig. 22, to .26 in Fig. 23, to about .40 in Fig. 24, and about .55 in Fig. 25. This time series gives a good sanity check that the transmittance results are reasonable. Also, in Fig. 25, we can see that the regions that appear to have thinner clouds also have higher transmittances. From this study, we believe that the bright stars provide reasonably accurate transmittances, with an uncertainty of about 5%, over a wide range of transmittances. Our plan is that we will not use these bright stars in the algorithm, but will use them for real-time checking of the algorithm results at a future time. 7.2. Concepts for a High Resolution Night Cloud Algorithm The concepts for a spatially high-resolution algorithm were first developed in 2001 (although we had informally been thinking about these concepts for many years prior to that date), and are documented in Memo AV01-069t. Basically, the concept as developed at that time is as follows: a) Extract typical radiance distributions for clear skies and for cloudy skies b) Use the star detection and associated beam transmittance determination to assign a value of opaque cloud, thin cloud, or no cloud, to selected star locations within the image. c) Near these star locations, determine how the radiance distribution differs from the typical radiance distribution at the same location. Locations that are determined to be clear sky will be used to adjust the clear sky distribution, and locations that are determined to be cloud will be used to adjust the cloud sky distribution. d) Compare the radiance in each pixel with the adjusted nominal clear sky and cloudy sky distributions to determine the presence of cloud. To examine this concept further, we first extracted the clear sky background from an average of 145 images over 11 nights. We found that at a given pixel, the variation from night to night on clear moonless nights was about 5% STD. The clear sky background plot for the central row and column are shown in Figures 26 and 27. In these plots, the black curve is the average, and the colored curves show specific nights. The average image is shown in Figure 28. Figure 29 shows the signal as a function of hour angle. It is interesting that even on a moonless night there would be some systematic variance with hour angle. It may have to do with how many people keep their lights on as the night progresses. To the extent that this variation is systematic, it is predictable and therefore useful in the algorithm development. Figures 26 – 29 show that the clear sky background is reasonably well behaved, at least in this data set. We do expect some variation with haze amount, which is why it may be important to normalize the background for a given image as indicated in step c above. However, first we need to know whether the cloudy radiance distribution differs sufficiently from the clear sky radiance distribution for this method to be worth pursuing. 27 Median Background Sky Cross-Section: Middle Row 6x10-5 FTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT Median Background Sky Cross Section: Middle Column 6x10-5 FTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT 5x10-5 5x10- 51 Feb. 14, 1999 Apr. 23, 2001 Jun. 07, 1999 Sep. 14, 2002 Oct. 17/18, 2001 Dec. 17, 2001 . 11 Combined Dates Feb. 14, 1999 -- Apr. 23, 2001 Jun. 07, 1999 Sep. 14, 2002 Oct. 17/18, 2001 - Dec. 17, 2001 11 Combined Dates 4x10-5 F 4x10-5 Background Radiance (W/m2-um-Sr) דדדדדדדדדדדדדדדדדדדדדדדדדדזזזזזזזזזזזזזזזזזזזזזזזזזזזזזז 10-5E 3X Background Radiance (W/m²-um-Sr) חדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדדד 3x 10-5F LLLLL 2x 10-5 2x10-5F 1x10-5 1x10-5 F 400 500 500 0 100 200 300 Column Fig. 26. Central Row for Clear Sky 0 100 200 300 400 Fig. 27. Central Column for Clear Sky Row .. பொய்யார்யர் SOR Hour Angle Background Images Cross-Section: Middle Row TTT TI 6x10-5 Hour Angle Range 6 to 7 Dusk 7 to 8 8 to 9 9 to 10 10 to 11 11 to 12 -12 to -11 -11 to -10 -10 to -9 -9 to -6 Dawn Combined 4x10-5 Background Radiance (W/m²-um-Sr) 2x10-5 100 200 300 Column 400 500 600 Fig. 28. Clear Sky background image Fig. 29. Central Row plotted as a function of hour angle We extracted the cloudy radiance distribution from 101 images on 5 nights. Figure 30 shows the data from Figure 29, with the cloudy background superimposed. In this plot, the color curves are the clear sky background, and the black curve is the cloudy background. Over most of the sky, there is a separation between the clear and cloudy radiance levels of more than 100% of the clear sky radiance. Figure 31 shows those regions with more than 100% separation colored in grey. The regions in which the separation between the cloudy and clear backgrounds is less than 100% are the relatively small regions shown in black. Even in those regions, the separation was significant, and should be adequate for algorithm development. 28 SOR Hour Angle Bkgrnd And Smoothed Opaque Cloud Cross-Section: Middle Row 1.5x10-4 1.0*10-4 Radiance (W/m²-um-Sr) Hour Angle Range 6 to 7 Dusk 7 to 8 8 to 9 9 to 10 10 to 11 11 to 12 -12 to -11 -11 to -10 -10 to -9 -9 to -6 Dawn Combined Opoque Cloud 5.0x10-5 U LUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU 100 200 400 500 600 300 Column Fig. 30. Cloud Background (black curve) compared with clear background (colored curves) Fig. 31. Grey coloring indicates regions where cloud background differs by over 100% Several examples of night sky images and associated radiance levels are shown in the April 05 talk, and some of these are presented here. Figure 32 shows a clear night with no moon. Figure 33 shows the associated middle row radiance (black curve) compared with the nominal clear sky (blue curve) and opaque cloud (red curve). SOR February 14, 1999 0750 UTC SOR Radiance Cross Section - Middle Row February 14, 1999 0750 UTC 1.5x10-4 TUTTI Nominal Opaque Cloud (Curve 1) Nominal Cloud-Free Background (Curve 5) - Feb. 14, 1999 Cloud-Free Image (Curve 3) 1.0610-47 Radiance (W/m²-um-Sr) 5.0x10-5. 0 100 200 300 Column Pixel 400 500 Fig. 32. Clear sky sample, no moon Fig. 33. Radiance from Fig. 32 compared with nominal clear sky and opaque sky radiances Figures 34 and 35 show similar examples with no moon, and with broken cloud. Note that in this example, the regions with cloud are brighter than the clear sky, but darker than the typical cloud curve. From the points within the clouds that we know are cloudy (based on the transmittance), we would determine the cloud radiance values and lower the nominal cloud curve to generate a cloud curve for that image, to use in the pixel evaluation. SOR June 16, 1999 0700 UTC SOR Radiance Cross Section - Middle Row June 16, 1999 0750 UTC 1.5x10-4 - Nominal Opaque Cloud (Curve 1) Nominal Cloud-Free Background (Curve 5) June 16, 1999 Broken Cloud Image (Curve 34 1.0*10-4 Radiance (W/m²-um-Sr) 5.0x10-5 100 400 500 200 300 Column Pixel Fig. 34. Broken transparent cloud sample Fig. 35. Radiance from Fig. 34 compared with nominal clear sky and opaque sky radiances From this data and other examples, we believe that the use of the radiance distributions should have a very good chance of working well under moonless conditions. Clearly, there are many details to be developed and programmed, but we do not see major show- stoppers at this point. To extend the high resolution concept to moonlight conditions, it will be necessary to characterize the clear sky and cloudy backgrounds as a function of look angle, and moon phase and position, much as is done with the daytime clear sky background ratios. Figures 36 and 37 show a clear night and a cloudy night, extracted when the moon was at similar positions and moon phases. In figure 36, the moon phase was .73, the relative brightness was .17, and the zenith angle was 59º. In Figure 37, the moon phase was .64, the relative brightness was .13, and the zenith angle was 56°. The plots for these two nights are shown in Figure 38, and show excellent separation. (The low values in the cloudy curve are from the occultor structure.) We expect that it will be important (and not trivial) to characterize the moon background well, but that if we can do that, the high resolution algorithm should work well under moonlight as well as starlight conditions. 7.3. Processing of a SOR Night Database At the April 2005 meeting where we reported on the ground-truthing and high resolution concepts, the sponsors were pleased with these results. We were asked to switch priorities, and work on updating the night algorithm for the SOR site, and to process and evaluate night data for the SOR database discussed in Section 6. The algorithm in place at the site was the contrast-based moderate resolution algorithm. This algorithm requires that the angular calibration be updated whenever the instrument is moved. The geometric calibration had not been updated in quite some time, and the 30 instrument had been moved in the interim. As a result, the night algorithm results that were being derived in the field were very poor, as we would anticipate under these conditions. (We would like to automate the update of the geo calibration in the future, or at least automate a method to detect that the instrument has been moved and the calibration needs to be updated.) SOR June 23, 1999 0550 UTC SOR June 22, 1999 0510 UTC Fig. 36. Nearly cloud-free moonlight case image Fig. 37. Cloudy with similar moonlight position and moon phase SOR Radiance Cross Section - Middle Row Moon Between 1st Quarter and Full 3.0*10-4 2.5x10-41 - June 22, 1999 0550 UTC Overcast Image June 23, 1999 0510 UTC Cloud-Free Image 2.0*10-4 Radiance (W/m²-um-Sr) 0-4 TTT TTT 1.0x 10-4 TTT 5.0*10-5 olumlu 100 200 300 Column Pixel 400 500 Fig. 38. Middle row extracted from Figures 36 and 37 We extracted the current angular calibration, and updated the geometric algorithm inputs to the algorithm, as documented in AV05-034t and -035t. We updated the stand-alone algorithm to include the night algorithm, which had previously been running only in IDL and/or in the field on the Unit 12 system. We also updated the horizon mask, which masks out objects in the field of view. There were no other algorithm updates at this 31 time, because we wanted to see how well we might expect the algorithm to perform when the geometric calibration was up to date. We processed the nighttime data from the same SOR test-bed data from July and August 05 used for the day algorithm evaluation. The night data were taken at 3-minute intervals at that time, with the result that 3698 images were processed. The results were documented in Memos AV05-031t, AV05-035t, and AV05-037t, and delivered to the sponsor with the 1-minute day data. Although this work was done with the older algorithm, because we had not yet had an opportunity to spend much time on the new algorithm development, we were pleased to have an opportunity to process a significant amount of data, and see how this first version algorithm behaved. Several examples are shown in Figures 39 through 45. These figures are shown in the format shown to the user in the SORCloudAssess program. The raw open-hole (no spectral filter) image is shown on the left, and the cloud decision is on the right. In the cloud decision image, black indicates the “no data” category. Pixels in this category can be due to physical structures on the horizon, or if the black region is in one of the cloud decision cells, it means that there were insufficient stars in the cell to make a determination. Green indicates a night thin cloud decision, and grey-to-white indicates an opaque cloud decision. *51923001CR Raw Red D 5 1911001.cld Cid Image + Yes 90% No * Exit? Fig. 39. Example of a relatively good night clear sky result In Figures 39 and 40, we see a relatively good case and a relatively poor result for a clear sky. The most obvious problem with these images is the region impacted by bright lights of Albuquerque, in the lower left of the image (regions shown incorrectly in white in the cloud decision image). This region is incorrectly being identified by the algorithm as opaque cloud. Also, some other clear areas, particularly in the Milky Way, are identified as thin cloud (two green cells). Figures 41 and 42 show two cases with relatively good and relatively poor results for overcast. The majority of the cells are identified correctly (opaque clouds shown in white-to-grey), however there are several cells that are incorrectly identified as thin cloud (green) or even clear (blue) in each of these images. 52450D DER RW.RO L 52150900.cld Cid Image * Yes 90% No Exit? Fig. 40. Example of a relatively poor night clear sky result 672200500 OR R Red TL52200500.cld Cld Image * Yes 90% 3 No Exit? Fig. 41. Example of a relatively good night overcast sky result *3224030B. GIR RAW Rad 52240308.cid Cid Image * Yes 90% No Exit? Fig. 42. Example of a relatively good night overcast sky result 452260501. CR Raw Red L 52260501.cld Cld Image # Yes. 90% No Exit? Fig. 43. Example of a night broken cloud result Figure 43 shows a broken cloud result, which had quite reasonable results. Figure 44 shows a case with a mixture of thin and opaque clouds, and again the results were reasonable (regions that appear in the raw image to be opaque are colored grey, and those that appear to be thin are colored green). Figure 45 shows a case of moonlight, and here the results were also good. In each of these three cases, most of the cells provide reasonable results, and a few do not. Figures 39, 41, and 43 – 45 were fairly typical results for this data set. (For Figures 40 and 42 we tried to find worst case results.) 52261002 CR Rad 52261002.cid Cid Image it: Yes 90% - No Exit? Fig. 44. Example of a night case with opaque and thin clouds 52031001 QR Raw Ry 52081001.cld Cid Image Yes. 90% No Exit? Fig. 45. Example of a moonlight case In order to assess these results more systematically, the SORCloudAssess program was updated to allow assessment of night imagery as well as day imagery. We made three tests. a) LOS or Line of Sight Test. In this test, we looked at each of the 11 ROI's marked in the image on the right, and marked it if the result was not correct for the line of sight, just as we did with the Day data. The results of this test are shown in Figure 46. 35 b) ROS or Region of Sight Test. We recognize that with a moderate resolution algorithm, we would never use exactly the Line of Sight to make a decision. Instead, we would assess the presence of clouds in the nearby region of the line of sight. Thus, for this test, we assessed the general Regions of Sight as follows. If the line of sight was incorrectly identified as cloud, we asked if there were clouds within the cell, and if there were, we did not mark it incorrect. If the line of sight was incorrectly identified as clear, we asked if there was an adjacent cell identified as cloud, and if there was, we did not mark it as incorrect. That is, we asked if the answer was correct in the region of the line of sight. The results of this test are shown in Figure 47. c) Instead of visually evaluating whether the algorithm was correct over 90% of the image, as was done with the day algorithm, we assessed whether it was correct over 70% of the image, because we felt it would be too difficult to assess a reduced resolution algorithm at the 90% level. The results of Figures 46 and 47 and a summary of the night SORCloudAssess results are shown in Table 2. 51930402.cld Cid Image 87.2 89.8 70.2 80.7 93. 1 94.7 90.6 87.8 91.9 88.3 88.0 # Yes 90% o No Exit? Fig. 46. Fraction of correct answers, in percent, for each ROI for SOR Night Test Bed for Line of Sight Test 51930402.cld Cid Image 96.3 96.8 81.4 87.2 | 95.7 97.9 98.9 99.5 99.5 98.4 97.9 + Yes 90% o No Exit? Fig. 47. Fraction of correct answers, in percent, for each ROI for SOR Night Test Bed for Region of Sight Test Table 2 Summary of Estimated Accuracy for the Night Cloud Algorithm Using SORCloudAssess For the SOR Test Bed Data Sets Overall Zenith Eastern Horizon | Western Horizon 70% Correct? LOS Results | ROS Results 87.7% 95.5% 93.1% 97.9% 87.8% 99.5% 70.2% 81.4% 95.2% In Figures 46 and 47, we can see that the results over most of the sky are very good for the Region of Sight test (Fig. 47), and reasonably good for the Line of Sight test (Fig. 46). This implies that if we were to continue to use the reduced resolution algorithm for 37 some reason, we would have to look at the Region of Sight, not just the Line of Sight, to assess the site. The worst problem area in each case is the region impacted by the bright lights of Albuquerque. We expect that this can be significantly improved in the future, particularly if we develop the transmittance-based algorithm. We felt that for a first version of a working night algorithm, these results shown in Figs. 46 and 47 and Table 2 were very encouraging, but they would clearly benefit from algorithm improvements. 7.4. Upgrading the Night Algorithm Following delivery of the SOR night algorithm processed results, we immediately went ahead with work toward integrating the beam transmittance calculations into the moderate resolution night algorithm. Under this contract, we got the algorithm programmed in IDL, and ran a few test cases. As part of this work, we extracted the star inherent irradiance offset for 7600 stars using the techniques discussed earlier. We extracted both the correction factor and the standard deviation in irradiance, and rejected those cases with standard deviations above 15% (this is an input file variable, so it can be changed). We found that for this larger data set, a better aerosol correction to use was .87 for this SOR site, and used this for all of the stars. A plot of the measured spectral irradiance vs. the theoretical spectral irradiance similar to Fig. 20 in Shields et al. 2007 was regenerated, with improved results. Work to understand and isolate the causes of the temporal variations in these irradiances, as well as the magnitude and variations (from one star to another) in the correction factors, is continuing As reported in the December 2005 talk, we used a preliminary IDL version of the transmittance-based algorithm, and tested the cases shown in Figures 40 and 42, and found the results to be better. An overview of the first version of the transmittance-based algorithm is given in Memo AV06-009t. Under the next contract, we were asked to adapt the transmittance algorithm to one of the instrument sites, Site 2, and at that time we went ahead and converted it to a high resolution algorithm. This work will be reported in future reports. 7.5. Evaluation of Night Imagery at a Very Bright Site Under the current contract, we were also asked to look at whether we think it will be possible to use the WSI in very bright environments at night. As reported in Shields et al. 2007, the instrument fielded in Virginia is in a very bright environment. Under the previous contract, we evaluated the night imagery, and decided that there was too much stray light to be able to detect the stars over enough of the night sky, as documented in Memo AV04-054t. As a result, in June 2004 we changed the setup to start acquiring spectral data at night. Under this contract, we did further evaluation of the open hole data taken prior to June, and we found that there is a very large contrast between data with and without clouds. For example, we compared two images taken under starlight, once with a cloud-free sky, and one with overcast. The typical signal under the cloud-free conditions was near 550, and the typical signal under the overcast was near 8400, which is about a factor of 15 brighter. This should be more than enough signal difference to enable algorithm development. Examples of images with clouds overhead, and with clouds near the horizon, are shown in Figures 48 and 49. In these images we also had very good contrast between cloud and clear sky. Although we cannot detect many stars, the high resolution algorithm may require many fewer stars, so the high resolution algorithm may be adequate, particularly with data taken with the spectral filters. Otherwise, for sites this bright, we would have to develop a new night algorithm, but there is plenty of contrast in the raw data with which to develop such an algorithm. We do not expect to use the instrument at another site this bright, so we agreed with the sponsors that we will wait and evaluate the light field at the upcoming sites before making a bright-lights algorithm a priority. Fig. 48. Clouds overhead in a very bright city Fig 49. Clouds near horizon in a very bright city We also further evaluated sunrise and sunset. We found that in the SOR data set, out of 23 days, the results at 80° Solar Zenith Angle (SZA) were good on all but 6 mornings and 2 evenings. The test adaptive algorithm improved results in most cases. We also found that the opaque cloud ratio appeared to be independent of SZA. This is in contrast to the results with the Day WSI, where we did obtain improvements by making the opaque threshold be SZA-dependent. We don't know yet whether we can develop the sunset algorithm so that it always provides reliable results at sunset, although the algorithm already provides reliable sunset/sunrise results the majority of the time. This was just a preliminary look. The sunset/sunrise regime is an area that will take some effort if it becomes a priority. 7.6. Summary of Night Algorithm Work In conclusion, we made considerable progress under this contract in developing and assessing the capabilities of the night algorithm, and we have further plans for significant improvements. Partially with funding from another contract, we developed a concept for ground-truthing the cloud algorithm results using bright stars, and we further tested and developed concepts for a high resolution algorithm. Fully under funding from the current contract, DO #11, we updated the inputs to the SOR site contrast-based night algorithm, and processed a database. We tested these data using Program SORCloudAssess, and found that the results were good, although as we expected, they will benefit from planned algorithm improvements. 8. Wavelength Options to use for Optical Cloud Imaging One of the requirements of this contract was to analyze the pros and cons of using Infrared (IR) systems. As discussed earlier, the funding increment that was sent at the time this priority was set was used partially to begin funding this contract, and was used partially to fund options in the previous contract. As a result, we were able to complete this work under the early contract. The results were reported in a talk in July 2004, and they have been reported in Shields et al. 2007 and in Memo AV07-026t. In this report, we will summarize the results presented in the earlier report. Because one of the end goals of this project is to be able to identify the presence of clouds that will impact transmittance at 1.6 (m in the Short Wave IR (SWIR), we ran an experiment to compare the cloud imagery from the WSI with that from a fisheye imager we built that operates at 1.6 um (Shields et al. 2003c). Figure 50 shows a comparison of the visible and SWIR during the day, and Figure 51 shows a comparison near sunset. Fig. 50. Thin clouds, imagery from a SWIR system at 1.6 um on the left, and the WSI in the visible at 650 nm 7 May 04 near 1230 Local 40 Fig. 51. Sunset, imagery from a SWIR system at 1.6 um on the left, and the WSI in the visible at 650 nm 11 May 04 near 1942 L As discussed in the previous report, we concluded that a visible system will detect all the clouds that impact the SWIR. We feel that the visible system would be superior, because it has much better sensitivity and can work at night. Even in the daytime, the visible system has superior imagery due to low noise and better uniformity and resolution. A short evaluation of Mid Wave IR (MWIR) system characteristics led us to conclude that use of a MWIR would be more complicated than use of a Long Wave IR (LWIR) system, and would have other disadvantages. We did an extensive evaluation of LWIR systems, evaluating imagery we had access to, as well as doing theoretical evaluations. We presented results for zenith angles of 0°, 60°, and 85°. As an example, the results for 60° are shown in Table 3. In this table, we show the effective temperature of the cloud signal, for low (1 km), mid (5 km), and high (10 km) clouds. In the first column, the entry “10” is for opaque clouds at 10 km, and “10 thin” is for thin clouds at 10 km. “Aerosol” indicates the calculations for the aerosol background at the same look angle. These calculations are derived and explained in Shields et al. 2007. The effective temperature shown for each altitude has been corrected for the impacts of beam transmittance and path radiance, as explained in Shields et al. 2007. The AT columns show the difference between the effective temperature of the clouds and the effective temperature of the background cloud-free sky. This is shown for a standard atmosphere, and two extremes: the winter at 60° N latitude, and the summer at 30° N latitude. In Table 3, the color pink identifies those cases where the temperature difference is less than 1/4 of the estimated 31° K range in the background. The table entries that are identified with pink are cases where the algorithm would have to be reasonably sophisticated, meaning that a fixed threshold would either miss the middle and high 41 clouds, or would identify much of the sky near the horizon as cloud. The difference between the cloud and the background is even smaller in comparison with the background variation from image to image. The need to take into account seasonal, diurnal, and other variations in the background signal which would make the algorithm more complicated. The blue square indicates a case where we expect that the cloud signal will be lower than the detection threshold of the instrument. Table 3 Sample IR Evaluation Results from Previous Report Computed Cloud and Background Results for 60° Zenith Angle (Colors explained in Text) Standard Atm Winter 60N Summer 30N Alt (km) 1 Zen Eff | AT Eff AT Eff) AT Temp Temp Temp | 60 305.7 22.7 269.1 22.0 | 320.3 29.3 290.1 7.1 254.3 7.3 299.3 8.3 10 2849 19 249.4 2.4 293.5 2.5 284.0 1.0 248.2 1.2 292.2 1.2 10 Thin |Aerosol 283.0 247.0 291.0 There are some LWIR systems in development, and at this point, the imagery we have seen do not appear to be as good for our purposes as the imagery obtained in the visible. A theoretical analysis of the cloud and background signals indicates that although low clouds at the zenith are easy to detect, sophisticated algorithms will probably be required to detect low clouds at angles away from the zenith. Also, a reasonably sophisticated algorithm will probably be required to detect middle and high clouds at most angles. Close to the horizon, middle and high clouds are expected to be buried in the noise. In cold environments, the middle and high clouds at the zenith may be offscale dark. (The data to support these comments are shown in Shields et al 2007.) In order to successfully identify the presence of clouds, two things are necessary. First, the raw imagery must have sufficient difference between the cloud and background signals. Second, algorithms must be able to sort out these differences and identify the presence of the clouds. Our initial analysis leads us to believe that for an IR system, the first requirement may not occur for many needed angles and cloud heights, and reasonably sophisticated algorithms will be required for most angles and cloud heights. By contrast, as illustrated in earlier sections, the visible sensors provide very high quality 42 imagery under all conditions we have encountered, and clouds are normally well detected down to the horizon, and at all cloud altitudes. Following this report, the sponsors decided that although they may want us to develop an IR fisheye system for test at a later time, at the present time it was deemed more important to continue to develop the capabilities of a visible system. 9. Hardware and Software Developments and System Preparation for Deployment As part of this contract, we evaluated upgrades that should be done prior to building the next instruments, we provided maintenance for systems in the field, and we began refurbishment of older WSI units for use in the SOR deployments. This section will provide an overview of the system evaluation, the upkeep of the other instruments in the field, and the hardware and software work required for the refurbishment. 9.1. Concepts for System Upgrades In designing the WSI systems, we strive to achieve a system that is very capable, reliable, and as cost-effective as possible within the constraints of meeting sponsor's technical needs and obtaining the required capability and reliability. The Day Night WSI systems were first designed in the early 1990's, and have been upgraded in several respects since that time. For example, the solar/lunar occultor is currently much more reliable than the first version designed in the early 1990's. Also since that time, the environmental housings have been upgraded to measure and report on the state of the instrument, and respond accordingly, for example turning the camera off if the environmental housing temperatures are too high. The computer and electronics, in the latest versions, were integrated into the environmental housing, so that the WSI could be separated from the user by a much longer distance, and be connected by a fiber optic communication. y Several upgrades are under consideration for the next generation of instrument. As discussed in Section 8, we feel that a visible system will provide better results than an IR system under most conditions. However, if it turns out that we are unable to provide good results for sunrise and sunset, and if this period of time is important enough to justify the incremental cost, we could evaluate building a hybrid system that takes advantage of both a visible and an IR sensor. One of the major changes under consideration is changing to a smaller and higher precision solar/lunar occultor that does not block so much of the sky. As reported in Shields et al. 2007, we designed a solar/lunar occultor with a smaller footprint (i.e. smaller obscured area in the sky), but we were not happy with the performance of the encoder used in that design. Under the previous contract, we tested a new encoder, and felt this would perform much better. A second major change is simplification of the control electronics. The current electronics allow the user to control each of the peripherals either via either computer or manual control on the Accessory Control Panel. We feel this flexibility is no longer needed, and plan to design a much simpler interface, in which the user can control the components independently through an interactive computer interface. One possible computer interface design is shown in Figure 52. Virtual ACP TROLLEY DRIVE ARC DRIVE CURRENT POSITION CURRENT POSITION 90.00 90.00 BRAKE SET REVERSE STOP FORWARD REVERSE STOP FORWARD RELEASE PULSE REVERSE PULSE FORWARD PULSE REVERSE PULSE FORWARD DRIVE ARC TO: 90.00 DRIVE TROLLEY TO: 90.00 GO Go SENSORS FILTER CHANGER CCD TEMP -40.00 SPECTRAL WHEEL STEP FORWARD POSITION CAM HSG TEMP 30.00 STEP REVERSE ENV HSG TEMP 25.00 ND WHEEL POSITION FLOW RATE 0.400 STEP FORWARD N2 PRESSURE 5.000 STEP BEVERSE Fig. 52. Initial design for computer control to replace Accessory Control Panels With modern cameras, the same or better cooling of the CCD chip can now be obtained with air cooling of the camera. This would eliminate the need for pumping and monitoring liquid coolant in the environmental housing, and hopefully enable a reduction in the size of the housing itself. Additional changes are under consideration, and will be more fully evaluated if we are funded to build new systems. 9.2. System Maintenance During this period of this contract, there were several repairs to Unit 12 at Albuquerque. The filter changer photodiodes failed in October 2004, and were replaced successfully by site personnel with our support. However, they failed again in December, and the MPL team went in January 2005, and repaired the system, as documented in Memo AV05- 024t. The instrument had been badly damaged by a nearby lightning strike during the previous contract, and we had repaired it and gotten it running again. In April 2005, there was another lightning storm that caused odd streaks in the camera image. We were not able to repair this until July 2005, because preparing an instrument for fielding was higher priority. The July repair trip was successful, although we discovered other 44 problems that were not show-stoppers. This trip is documented in Memo AV05-027t. Later in July, the system was shut down during a lightning storm, and in the process of getting it restarted, on-site personnel found that the camera cable had been damaged by rodents. (It operated anyway during this time.) In August a video card failed, and was sent to the manufacturer for repair. On its return in October, it still failed to work. The PL team went out in November, and made a number of repairs, including replacing the cable and getting the video to work. This trip is documented in Memo AV05-039t. At that point, all systems were in good repair, except the air conditioner. The air conditioner should be replaced when feasible, but it was not causing problems with the data, even in the summer. There were also a couple of repairs to Unit 14 in Virginia. In October 2004, the shutter failed. Memo AV04-044t was written to document the shutter changing procedures, and a shutter was sent to Virginia. It was successfully changed by the site personnel. In March 2005, a new computer cooling fan was sent to the site and successfully replaced by site personnel. The shutter failed again in July, and this time site personnel were not able to get it running. A repair trip was made by the MPL team in August, as documented in memo AV05-026t. The camera failed in March 2006. We offered to swap the camera with another one here, but this was not acceptable to TASC because it would change the calibration. Unfortunately, we did not have sufficient funding for a more expensive solution. Under the next contract, we did not receive sufficient funds for a full time hardware person, and SOR felt that deployment of cameras to the new sites was a higher priority with the limited resources. 9.3. WSI Refurbishment - Hardware As discussed in Section 5, some older WSI systems became available early in the contract, and the decision was made to refurbish these systems for use at three new sites for this program. These older instruments were originally built for the Department of Energy (DOE) and used for many years at a variety of sites. A decision was made in 2004 to retire the systems (mostly, I believe, because the algorithms developed by the sponsors were not adequate). We proposed that the retired instruments be given to MPL for use in other programs, and this was done. As a result of this situation, our SOR sponsors asked us to spend more of the Task 3 funds on refurbishment of these systems, and also provided the optional funding in the contract to begin to adapt three of the systems so they could be fielded at sites for the SOR program. The first two units for refurbishment, Units 7 and 8, were received in-house in February 05, as documented in Technical Memo AV05-004t. The retired unit that was already in- house, Unit 4, is documented in Memo AV05-020t. Other units that can serve as test units and spare parts, or be used for other programs, are documented in Memos AV05- 025t, AV05-028t, and AV06-022t. The configuration of the Unit 7 sensor and the controller, upon completion of the refurbishment, are shown in Figure 53. The left side shows the sensor unit, with its environmental housing and the solar/lunar occultor. This unit has successfully operated for many years in the Arctic, and similar units have operated in other locations, including the tropics and the desert. The right side shows the controller unit, as reconfigured for the SOR project. The other units are quite similar, except for the size of the occultor shade, as discussed later in this section. Also, Units 7 and 8 have the glass domes, and Unit 4 has an acrylic dome. Fig. 53. Sensor and Controller Configuration for Unit 7 These systems were built in the mid-to-late 1990's, and as a result they needed a fair amount of refurbishment. Typical refurbishment tasks include disassembly and cleaning, replacing worn components such as the coolant tubing, replacing any failed components such as arc drive motors, and getting the cameras tested and purged at Photometrics. Unit 4 had significant electronics issues with the Accessory Control Panels and camera housings. We also replaced filters as necessary, replaced the shutter, replaced cables as required, cleaned the system, replaced the acrylic optical domes, and re-labeled connections. These repairs were completed on the first system (Unit 7), and partially completed on two other systems (Unit 4 and 8). The solar/lunar occultor uses an arc drive to provide East-to-West motion. Normally, the trolley drive moves a small disk along a North-South axis, in order to obscure the sun or moon at its current position. Systems 7 and 8 used an occultor shade with no trolley, and 46 the trolley drive for Unit 4 had previously been cannibalized. The fixed shade is a piece of metal the width of the trolley, but it covers the full north-south extent of the solar and lunar motion, so that it doesn't need to move. This blocks a larger part of the sky than a trolley would, but it avoids the use of the trolley drive, which can fail particularly in difficult environments, so it was used for the systems fielded in locations such as the Arctic. The fixed shade also has a window in the middle to enable imaging the solar disk. The shade size depends on latitude, and is thus specific to the location. The shade sizes were calculated for the SOR locations and rebuilt accordingly. The occultor shades are documented in Memos AV05-15t and -016t, and the software used in the calculations is documented in Memo AV05-011t. Units 7 and 8 also have a glass dome with a heater, in place of the more standard acrylic dome. As part of the refurbishment process, the systems are refocused and radiometrically calibrated. The electronics such as the meters and the occultor arc drive control are recalibrated. The sensor and control sub-systems are thoroughly tested prior to deployment, and the assembled system is then tested. These older systems use a Photometrics Series 200 camera system, which must run on a DOS computer system. Upgrading the cameras to a Series 300 configuration that can run on more modern computers was considered, but the sponsors decided it would be too costly. As a result, we kept the DOS systems for the WSI control. Under the previous DOE program, the control computers had used a split backplane with two CPUs, to enable a networking system to come in and pull data files without disturbing the data acquisition. Because the SOR program planned to use a better networking system, and the second CPU was not sufficiently modern to use for the real time processing, the control computer was reconfigured to disconnect the second CPU and its associated cards. The computers were also refurbished by replacing and/or repairing any cards that were not operational on arrival. A WSI Processing computer running under Windows XP was added to enable processing the algorithm in real time. The appropriate networking cards were added to the processing computer. A Masterview switch was used, to allow the use of either computer with the same keyboard, mouse, and monitor. An updated GPS was added because the older one was no longer able to get a signal, and Garmin no longer supported that model. Also, a remote power switch was installed to monitor the status of the control computer via the processing computer. Software was written so that if the processing computer stops receiving data from the control computer, the processing computer will reboot the control computer via this remote power switch. Then the control computer will automatically restart the data acquisition program upon reboot and initiate a connection with the processing computer. (The WSI is also installed in such a way that if the processing computer hangs, the personnel at SOR can detect this and reboot it.) We decided to house the WSI controller in the small rack-mount shown in Fig. 53, which is shippable without an external crate. We mounted the processing computer on top of the rack mount. In this way, the controller has a smaller profile than when it is mounted in the more standard full rack mount. This was done primarily because the controller rack will be placed in a small and very crowded shed, and it minimizes the required space, as well as minimizing the heat created by the controller. With the more standard enclosed rack we use, the rack itself requires a cooler if the environment temperature control is marginal, as it may be in the sheds. With the smaller and more open rack configuration used for this fielding, no additional cooler is required, although it is still necessary that the shed be cooled. Following receipt of the first unit in February 2005, we began working intensively on the system refurbishment, because the sponsors were hoping to deploy the first one soon. Due to peculiarities in the funding process, the processing computers were provided directly by the sponsor, and were received in late April or early May of 2005. Following the installation of the processing computer into Unit 7, Unit 7 was tested outside successfully starting May 12. At that time we had not yet calibrated the system, nor done miscellaneous clean-up such as labeling cables. This was done, along with numerous tests, in May and June. The full automated hands-off test began June 27, although the new occultor shade was added after this date. As it turns out, our sponsors had to request that we delay the deployment due to issues outside of the control of MPL. As a result, Unit 7 was not actually deployed until 1 May 2006, under the next contract. However, during this time, Unit 7 was allowed to run continuously, and ran well. We ran the control computer and WSI system in hands-off mode, but we also frequently stopped the processing computer in order to test ongoing updates to the processing code that will be discussed in the next section. The memo AV06-007t documenting the hardware was written later, but is also referenced here. During this time, we worked on repair of the systems in the field, as documented in Section 9.2, and worked on refurbishment of Units 4 and 8, which are the other two units for deployment. By October 2005, Unit 8 was running indoors, although it still needed a significant amount of work, such as calibration of the electronics. By the end of the contract in April 2006, Unit 8 was running outdoors. In summary, Unit 7 was completely refurbished and ready for deployment when the sponsors were ready. It ran very well through the extensive test period. The other two units were partially refurbished, and this work was continued under the next contract. Because the sponsors were not ready for deployment under this contract, the deployment was delayed until the follow-on contracts. 9.4. WSI Refurbishment - Software Along with the hardware changes, significant software changes were required to the systems under refurbishment in order to enable fielding them for the SOR deployments. Some of these changes were due to the differing needs of the older project and the SOR project, and some were due to the hardware upgrades. Once these initial changes were 48 made, significant effort was put into making the programs more robust and convenient for the remote user. There are two primary programs on the system: the WSI control program, RunWSI, and the program that processes the algorithm on the processing computer, ProcWSID. The control program was updated to change the acquisition interval. The previous DOE project required acquisition of a full data set every 6 minutes, and red or open hole every 2 minutes. The SOR task required a full image set every 1 minute during the daytime, and 2 minutes at night. The program was changed so that the acquisition interval is user- selected, and it is currently set for 1 minute intervals during the day and 2 minutes intervals at night. The program was updated so that it no longer has the option to acquire either red or open hole imagery at night; it only acquires open hole, except under full moon. During the full moon night, the system acquires both open hole and spectral data in all filters, so that we can evaluate which are best for the algorithms. Once this decision is made, we anticipate acquiring either open hole or spectral data, but not both under full moon. The control program was modified to communicate with the processing computer via ftp. Several features of the earlier program that are not in the other SOR programs were retained. These include giving priority to grabbing images, even if the occultor is not quite in the right place or the filter is not yet in position, and delaying a minute if the 0- second mark is missed so that image sets always start on the O mark. The image archive format was changed to be compatible with the other SOR programs. The user hot key option was removed, because it interfered with the timing at these short acquisition intervals. The ability to either overwrite or append to diagnostic files was added. Various system diagnostics were added. For example, timing information related to the length of the image grabs was added to the header, to enable diagnosing problems related to hardware. A feature that had been in some versions, but not in the SOR systems, is a check to determine if the shutter is not opening properly. These diagnostics and others are documented in Memos AV05-023t and AV06-005. In addition, we added information to the headers, to be used by the processing program in creating the QC files, including the user-defined night acquisition interval and the time source. The code was speeded up, to enable reliable 1-minute data acquisition. Another change involved updating the program to accept the new GPS output strings. The new processing computers for Units 4, 7, and 8 run under WindowsXP. Previously, these WSI units did not include real-time algorithm processing. As part of this refurbishment, we took the processing program from Units 13 and 14, designed to run under WindowsNT, and updated it for this environment. This program included the day cloud algorithm. Units 13 and 14 also include the contrast-based night algorithm; however it is called separately as a system call. A couple of reboot options were added to ProcWSID. 1) Processing computer reboot - To free memory and refresh the operating system an option was added to reboot the processing computer at any user specified time and period. For example, Unit 7 is setup to reboot every day at 1000Z (added 24 Oct 2005). 2) Control computer reboot - if no data is received for any user specified period the processing computer can reboot the control computer. For example, the user can specify that if no data is received from the control computer for 30 minutes, the program can be set to either note the event in the reboot log (C:\Program Files\ProcWSI\WSILiveCheck.txt) or to reboot the computer (added 11 May 2005). One of the major improvements to the processing program was the development of a new user interface, shown in Figure 54. This user interface shows the status of each of the peripheral systems such as environmental housing temperature. It shows a green, yellow, or red light beside each peripheral subsystem depending on the status of the system. Technical Memo AV05-022t was written for the support personnel in the field, to explain how to check the WSI system. S ProcWSID Fie Options Help Red Image Date: 07/10/2005 Time:19:47G Exposure: 100 Arc Actual: 80.1 Arc Dest: 88.3 Tric Actual:Fixed Tric Dest: Fixed ND: 3 (3 log) SP: 3 (Red) CCD chip tcmp: -34,0 Env. Housing temp:16.0 Cam. Housing temp:24,0 Flow: 0.38 GPM N2:0.0 PSI . OLIS Flux Level Status WSI Hard Disk Status Min.: 768 Max.: 12600 Next grab time: 19:48Z Figure 54. New User Interface, to enable inexperienced users to assess system status Other features that were added to ProcWSID, some prior to June 05 and some later in the year, include the following: 1) All I/O paths are specified. Various input files are used for the occultor mask, day and night algorithms. Rather than determining which input file to use based on header information, we state which input files to use in ProcWSID.inp. Also, different output paths are specified for the different archives. 2) All I/O paths are verified during initialization - The program alerts the user early in the program if there are any missing input files. Also, the program creates the output directories if they don't exist. 3) The occultor mask routine was updated to handle the fixed occultor shade. e were some features in the Unit 13/14 code having to do with another system that is no longer in use. Since these features were no longer needed, they have been commented out. 5) Files are converted to a long file format – This was done to make it easier for SOR to identify WSI data on their systems. The new filename format is WSIuuummddyyyyhhmm.nnn; where uuu is unit number mmddyyyyhhmm is the date and time of the image grab and nnn is the image type. 6) QC files have been added for the different WSI components. See Tech. Memo AV05-023t for information on the QC files. 7) An option to write to external 300 GB disk was added. Archiving to an external disk provides a backup in case of ftp failure, and also makes transfer of a copy of the data to MPL easier. These disks need to be replaced approximately every 6 months. 8) A 12-character error string is written to headers of the ratio and cloud decision images to let the user know about any problems with the imager that would affect cloud decision results. The error string details are documented in Memo AV06-006t. 9) The program sorts the files before processing the algorithm. This was done so that if there is a backlog of files to process, the files will be processed in order by date and time, with the earliest files processed first. 1 During May and June 05, the code was tested extensively, particularly in failure mode. For example, we tested how the software handles the situation if the Accesssory Control Panel (ACP) is accidentally set in “local” rather than "computer" mode. We also turned off the ftp server program on the processing computer (that communicates with the control computer) to verify that RunWSI still runs if it can't ftp data and ProcWSID reboots the control computer after it doesn't receive data after 30 minutes. We also tested the program under full moon conditions and we turned off the CEU to verify that Run WSI properly flags the case where we're not receiving a camera signal. Following these tests, we started the Unit 7 control computer running in hands-off mode on June 27. The Run WSI program ran without failures, and it was run essentially continuously until we began final preparations for the deployment in May 2006 after the new funding was received. Similarly, the hardware performed flawlessly during this period except for occasional but rare spectral filter wheel skips. In the meantime, we continued to make extensive upgrades and tests of the processing program, while running the control program. While testing the processing program we found that we were having memory leak problems. We traced the problem to running the night algorithm as an independent program under DOS. So we decided to incorporate the 51 night algorithm into the ProcWSID program. This was an extensive job, but well worth the effort, and it solved the memory leak problem. Although originally the programs to ftp data from the site to SOR were to be written by another sponsor, this had not been done yet, so we wrote and tested several versions of the ftp programs. Other features were added. For example, we added much more sophisticated data QC files to the output of the processing program. The first version of these QC files is documented in Memo AV05-023t. In September 2005, the SOR team wrote a program to automatically evaluate these QC files and identify cases that require attention. In order to avoid confusion between units, ProcWSI was adapted to add the WSI unit number to both the raw and the processed filenames. A program ProcWSIDInput was written, to allow the user to update the program inputs in a safer way, and the first version was installed in March 2006. 7 A considerable amount of supporting software work was also done under this contract. Some of it is mentioned in Sections 6 and 7 that discuss the algorithms. Along with the instruments, we received custody of a field calibration device, designed for easy and convenient radiometric calibrations of the WSI. The field cal acquisition software was updated to provide the flexibility needed for the SOR program. In addition, a second version of the Fieldcal acquisition software was created to operate under the WindowsNT operating system used on Units 13 and 14. WSITest, a program designed to enable convenient test of the WSI hardware components, had been written for Units 13 and 14, and was adapted to work with Unit 12, and installed there in July 2005. (There is similar code that works for the DOS Units 4, 7, and 8.) Program WSIView was written to allow a user in the field to evaluate the imagery. Other support programs include several programs written to test WSI system components such as the remote power switch, and several programs written to test various versions of the ftp logic. Several other support programs to help with data analysis were written. These include image looping programs (one for Windows, and several to operate in an image processing V++ environment), a variety of programs written to enable easier handling and sorting of these large data bases, and programs for easier evaluation of the results of algorithm processing. As discussed in Section 9.3, Units 7 and 4 were fielded under the next contract, which will be documented in a later report. Prior to these deployments, the upgrades which had been tested on Unit 7 were also installed and tested on Unit 4. Technical Memo AV06- 002t discusses the directories and files on the control computer. Memos AV06-003t and -004t provide an operations overview for the control computer and the processing computer, respectively. Memos AV06-005t and -006t more fully document the Run WSI and ProcWSID software. In the future, in addition to continuing research on algorithm upgrades and evaluation and supporting programs, we also need to update the data acquisition and processing code for the SOR Unit 12 so that it will have these new features. The Unit 12 control code runs under Windows 95, so it will require significant changes to provide the upgrades inherent 52 in the DOS control code used on Units 4, 7 and 8. Installing the new processing code should be relatively straight-forward. A new version was installed in November 2005, but it does not include all the current features. 9.5. Deployment Logistics Support The three sites are being prepared by other contractors for SOR. As part of the work on the hardware and software, we had to coordinate numerous details, such as how much power is needed, and how many cables, the physical dimensions of the WSI installation, the size of shipping crates, and so on. These details were mostly worked out in early 2005. In addition, we do quite a bit of documentation prior to and just after deployments. Most of this was done under the next contract, since the site was finalized for deployment during the period of the next contract. However, the software documentation memos AV06-002 through 006 were written under this contract. Also, the initial version of the Units 7 and 8 setup memos, Memos AV05-030t and AV06-001t, were written under this contract. The setup memos were originally written for the DOE program, because the program initially required that the instruments be set up by untrained personnel. Although the level of detail included in these memos is beyond what is required by either SOR or MPL personnel, we find that much of the information in the memos can be useful, so we have continued providing them. Memo AV06-007t documents the general instrument overview for Unit 7. 10. Summary Under this contract, we had the opportunity to make significant upgrades to algorithms and our state of knowledge of the algorithm results. In particular, we upgraded the day algorithm to use the NIR data at hazy sites, and we wrote programs to assess the accuracy of the results. We developed concepts for ground-truthing the night algorithm automatically, and further developed concepts for the high resolution night algorithm. We processed and evaluated a test bed data set of both day and night data, and were able to assess the strengths and weaknesses of the algorithms. We made significant progress toward a transmittance-based night cloud algorithm. A considerable amount of support software for use in algorithm processing and development was written. Infrared systems were evaluated, and found to have significant problems for our application. The decision was made to postpone further evaluation of these systems. Potential WSI system upgrades were evaluated. Older WSI systems became available to MPL during this program, and the decision was made to begin refurbishing them for use at SOR deployments. One unit was refurbished successfully, with very significant updates to the software, and two units were partly refurbished. The software was updated not only to enable real time algorithm processing on these older systems, but the night algorithm was integrated into the actual processing code for the first time. The code was 53 adapted in several ways for current program needs, especially including features to make it more robust and features to enable more efficient QC of the data. We believe that we have completed all contract requirements. We very much appreciate having had the opportunity to do this work. 11. Acknowledgements We would like to express our appreciation to the personnel of Starfire Optical Range and their contractors from Boeing. Dr. Earl Spillar, the head of this program, and Ann Slavin, our acting contract monitor, were very helpful in providing guidance, and have been a pleasure to work with. We also have found our contacts with Maj. John Lewis and the Air Force personnel to be professional and productive. The newer members of the Boeing team, Marjorie Shoemake and Darielle Dexheimer, supported us with careful work and enthusiasm, and were very helpful in diagnosing hardware issues and working with us on other aspects of the deployments. The SOR team is always a pleasure to work with. We feel this WSI work is valuable, and very much appreciate having had the chance to advance the state of the art, as well as meet our sponsor's specific needs. We also would like to express our appreciation to ONR, which provided the funding vehicle for this work. 12. References 12.1. In-house Technical Memoranda, in order by memo number; available to sponsor on request Shields, J., “Research Approach for High Resolution Night Cloud Algorithms”, Atmospheric Optics Group Technical Memorandum AV01-069t, 3 July 2001 Baker, J., “Procedure for Changing WSI Shutter in Units 13 & 14”, Atmospheric Optics Group Technical Memorandum AV04-044t, 28 September 2004 Shields, J., “Initial Virginia night data analysis", Atmospheric Optics Group Technical Memorandum AV04-054t, 31 December 2004 Shields, J., “Return of D/N WSI Units 7 and 8 from NSA Sites”, Atmospheric Optics Group Technical Memorandum AV05-004t, 15 February 2005 Shields, J. and M. Karr, “Ceilometer Study Results”, Atmospheric Optics Group Technical Memorandum AV05-010t, 24 February 2005 Karr, M., “OccInfo for Windows Version 1.0”, Atmospheric Optics Group Technical Memorandum AV05-011t, 4 March 2005 Burden, A., “Cloud-free Background Imagery for Version 3 Night Cloud Algorithm”, Atmospheric Optics Group Technical Memorandum AV05-012t, 24 February 2005 54 Shields, J., "SOR Occultor Shade Size Considerations", Atmospheric Optics Group Technical Memorandum AV05-015t, 12 May 2005 Shields, J., "SOR Occultor Shade Computed Sizes”, Atmospheric Optics Group Technical Memorandum AV05-016t, 12 May 2005 Shields, J., “Unit 4 Parts changes during 2002 - 2005”, Atmospheric Optics Group Technical Memorandum AV05-020t, 1 July 2005 Shields, J., “Ground-truthing the WSI algorithms”, Atmospheric Optics Group Technical Memorandum AV05-021t, 15 July 2005 Baker, J., “Quick Checklist for Whole Sky Imager (WSI)”, Atmospheric Optics Group Technical Memorandum AV05-022t, 29 July 2005 Karr, M., “ProcWSID QC Files”, Atmospheric Optics Group Technical Memorandum AV05-023t, 2 August 2005 Baker, J., “Trip Report SOR January 2005 – Filter Changer Repair”, Atmospheric Optics Group Technical Memorandum AV05-024t, 24 August 2005 Baker, J., “Return of D/N WSI Unit 6 from TWP Site", Atmospheric Optics Group Technical Memorandum AV05-025t, 24 August 2005 Baker, J., “Trip Report TASC August 2005 - Shutter, Sensor Repair”, Atmospheric Optics Group Technical Memorandum AV05-026t, 30 August 2005 Baker, J., “Trip Report SOR July 2005”, Atmospheric Optics Group Technical Memorandum AV05-027t, 30 August 2005 Baker, J., “Return of D/N WSI Unit 9 from TWP Site”, Atmospheric Optics Group Technical Memorandum AV05-028t, 12 September 2005 Baker, J., “Set-Up Instructions for WSI Unit 7”, Atmospheric Optics Group Technical Memorandum AV05-030t, 22 September 2005 Shields, J., “Initial WSI Test Data from SOR Jul – Aug 05”, Atmospheric Optics Group Technical Memorandum AV05-031t, 7 October 2005 ocessing of the SOR Jul - Aug 05 1-min resolution Data Set”, Atmospheric Optics Group Technical Memorandum AV05-032t, 17 October 2005 Shields, J. and M. Karr, "New Programs and Program Upgrades for Algorithm Processing”, Atmospheric Optics Group Technical Memorandum AV05-033t, 18 October 2005 55 Burden, A., “Unit 12 Star-Based Geometric Calibration (Applies 14 Sept 2002 – 7 July 2005)”, Atmospheric Optics Group Technical Memorandum AV05-034t, 15 October 2005 Burden, A., “Unit 12 Star-Based Geometric Calibration (Applies 8 July 2005 Until Next WSI Move)”, Atmospheric Optics Group Technical Memorandum AV05-035t, 15 October 2005 Shields, J., "Cloud Free Line of Sight Forecast Test-bed Program”, Atmospheric Optics Group Technical Memorandum AV05-036t, 1 November 2005 Shields, J., “Night Algorithm Results from SOR Test Data Set”, Atmospheric Optics Group Technical Memorandum AV05-037t, 2 November 2005 Shields, J., “Day Algorithm Results from SOR Test Data Set”, Atmospheric Optics Group Technical Memorandum AV05-038t, 2 November 2005 Baker, J., “Trip Report SOR November 2005”, Atmospheric Optics Group Technical Memorandum AV05-039t, 29 November 2005 Baker, J., “Set-Up Instructions for WSI Unit 8”, Atmospheric Optics Group Technical Memorandum AV06-001t, 14 March 2006 Karr, M., “WSI Control Computer Outline”, Atmospheric Optics Group Technical Memorandum AV06-002t, 29 March 2006 Karr, M., “Control Computer Operations Overview for WSI Unit 7”, Atmospheric Optics Group Technical Memorandum AV06-003t, 30 March 2006 Karr, M., “Processing Computer Operations Overview for WSI Unit 7”, Atmospheric Optics Group Technical Memorandum AV06-004t, 30 March 2006 Karr, M., “Run WSI for DOS based SOR Units, Version 1.0”, Atmospheric Optics Group Technical Memorandum AV06-005t, 5 June 2006 Karr, M., “ProcWSI for DOS based SOR Units, Version 1.0”, Atmospheric Optics Group Technical Memorandum AV06-006t, 5 June 2006 Baker, J., “Unit 7 General Instrument Overview and FAQ”, Atmospheric Optics Group Technical Memorandum AV06-007t, 5 April 2006, revised June 28 2006 Burden, A., “Night Algorithm Progress", Atmospheric Optics Group Technical Memorandum AV06-009t, 1 August 2006 56 Shields, J., “Processing of the VA Apr 05 Data Set”, Atmospheric Optics Group Technical Memorandum AV06-018t, 18 September 2006 [documents work done in early 2006] Shields, J., “Return of D/N WSI Unit 3 from Manus Site”, Atmospheric Optics Group Technical Memorandum AV06-022t, 20 September 2006 [documents instrument received in early 2006] Shields, J., “Wavelength Options in Cloud Imaging”, Atmospheric Optics Group Technical Memorandum AV07-026t, 3 May 2007 12.2. Power Point Files from Presentations to SOR December 04: Reviews WSI hardware upgrade concepts, day and night algorithm upgrades, results with stand-alone day algorithm. April 05: Reviews hardware and software refurbishments, presents improvements in night beam transmittance determination, new night ground truthing, and tests of high resolution night algorithm concepts. June 05: Overview of concepts behind day and night cloud algorithms July 05: Review of deployment status and how WSI fits into program August 05: Update on unit readiness, including software updates, new user interface, and deployment logistics. Reported processing of test-bed data. Commented on CFLOS forecasting ideas. In separate file, presented Cloud Free Line of Sight statistics. December 05: New results from setting up day and night algorithm for SOR, processing test-bed data, writing a program to assess accuracy, and results of algorithm accuracy tests, day and night. February 06: New results with NIR algorithm, processing data from very hazy site in Virginia, assessment of algorithm accuracy for this site. Also presented data comparing visible and Short Wave IR cloud imaging, and new ideas on CFLOS forecasting. 12.3. Published References and Technical Notes in order by date Johnson, R. W., W. S. Hering and J. E. Shields (1989), “Automated Visibility and Cloud Cover Measurements with a Solid-State Imaging System”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, SIO 89-7, GL- TR-89-0061, NTIS No. ADA216906 Johnson, R. W., J. E. Shields, and T. L. Koehler (1991), “Analysis and Interpretation of Simultaneous Multi-Station Whole Sky Imagery”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, SIO 91-3, PL-TR-91- 2214 Shields, J. E., R. W. Johnson, and T. L. Koehler, (1993), “Automated Whole Sky Imaging Systems for Cloud Field Assessment”, Fourth Symposium on Global Change Studies, 17 - 22 January 1993, American Meteorological Society, Boston, MA Shields, J. E., R. W. Johnson, and M. E. Karr, (1994), “Upgrading the Day Night Whole Sky Imager from Manual/Interactive to Full Automatic Control, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Report MPL-U-140/94 Shields, J. E., R. W. Johnson, M. E. Karr, R. A. Weymouth, and D. S. Sauer, (1997a), “Delivery and Development of a Day/Night Whole Sky Imager with Enhanced Angular Alignment for Full 24 Hour Cloud Distribution Assessment”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Report MPL-U-8/97 Shields, J. E., M. E. Karr, and R. W. Johnson, (1997b), “Service Support for the Phillips Laboratory Whole Sky Imager”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Report MPL-U-10/97 Shields, J. E., R. W. Johnson, M. E. Karr, and J. L. Wertz, (1998), “Automated Day/Night Whole Sky Imagers for Field Assessment of Cloud Cover Distributions and Radiance Distributions”, Tenth Symposium on Meteorological Observations and Instrumentation, 11 – 16 January 1998, American Meteorological Society, Boston, MA S ster, U., Shields, J., Karr, M., Johnson, R., Dehne, K. and Woldt, M, (2000), “Ground- Based Cloud Images and Sky Radiances in the Visible and Near Infrared Region from Whole Sky Imager Measurements”, Proceedings of Climate Monitoring - Satellite Application Facility Training Workshop sponsored by DWD, EUMETSAT and WMO, Dresden 2000. Shields, J. E., M. E. Karr, A.R. Burden, R.W. Johnson, and J. G. Baker, (2002), “Analytic Support for the Phillips Laboratory Whole Sky Imager, 1997 - 2001”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego. Shields, J. E., R.W. Johnson, M. E. Karr, A.R. Burden, and J. G. Baker, (2003a), “WSI Field Calibration System Operations Manual", Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Technical Note 252, February 2003. Shields, J. E., M. E. Karr, A.R. Burden, R.W. Johnson, and J. G. Baker, (2003b), “Analysis and Measurement of Cloud Free Line of Sight and Related Cloud Statistical Behavior – Published as Final Report for ONR Contract N00014-97-D-0350 DO #2”, H 58 Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Technical Note 262, June 2003. Shields, J. E., R. W. Johnson, M. E. Karr, A. R. Burden, and J. G. Baker (2003c), Calibrated Fisheye Imaging Systems for Determination of Cloud Top Radiances from a UAV, International Symposium on Optical Science and Technology, SPIE the International Society for Optical Engineering, 2003. Shields J. E., R. W. Johnson, M. E. Karr, A. R. Burden, and J. G. Baker, (2003d), Daylight Visible/NIR Whole Sky Imagers for Cloud and Radiance Monitoring in Support of UV Research Programs, International Symposium on Optical Science and Technology, SPIE the International Society for Optical Engineering, 2003. Shields, J. E., R. W. Johnson, M. E. Karr, A. R. Burden, and J. G. Baker (2003e), Whole Sky Imagers for Real-time Cloud Assessment, Cloud Free Line of Sight Determinations and Potential Tactical Applications, The Battlespace Atmospheric and Cloud Impacts on Military Operations (BACIMO) Conference, Monterey, CA. http://www.nrlmry.navy.mil/bacimo.html, 2003. Shields, J. E., A.R. Burden, M. E. Karr, R.W. Johnson, and J. G. Baker, (2004a), “Development of Techniques for Determination of Nighttime Atmospheric Transmittance and Related Analytic Support for the Whole Sky Imager - Published as Final Report for ONR Contract N00014-01-D-0043 DO #5", Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Technical Note 263, April 2004. Shields, J. E., M. E. Karr, A.R. Burden, R.W. Johnson, and J. G. Baker, (2004), “Project Report for Providing Two Day/Night Whole Sky Imagers and Related Development Work for Starfire Optical Range - Published as Final Report for ONR Contract N00014- 97-D-0350 DO #6”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Technical Note 265, May 2004. Shields, J. E., J. G. Baker, M. E. Karr, R. W. Johnson, and A. R. Burden, (2005a), Visibility measurements along extended paths over the ocean surface, International Symposium on Optical Science and Technology, SPIE the International Society for Optical Engineering, August 2005. Shields, J. E., A.R. Burden, R.W. Johnson, M. E. Karr, and J. G. Baker, (2005b), “Cloud Free Line of Sight Probabilities and persistence Probabilities from Whole Sky Imager Data”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Technical Note 266, August 2005. Shields, J. E., A. R. Burden, R. W. Johnson, M. E. Karr, and J. G. Baker (2005c), Measurement and Evaluation of Cloud Free Line of Sight with Digital Whole Sky Imagers, The Battlespace Atmospheric and Cloud Impacts on Military Operations (BACIMO) Conference, Monterey, CA. http://www.nrlmry.navy.mil/bacimo.html, 2005. 59 Shields, J. E., R. W. Johnson, J. G. Baker, M. E. Karr, and A. R. Burden, (2006), Multispectral scattering measurements along extended paths using an imaging system, International Symposium on Optical Science and Technology, SPIE the International Society for Optical Engineering, August 2006. Shields, J. E., M. E. Karr, A.R. Burden, R.W. Johnson, and W. S. Hodgkiss, (2007), “Enhancement of Near-Real-Time Cloud Analysis and Related Analytic Support for Whole Sky Imagers, Final Report for ONR Contract N00014-01-D-0043 DO #4”, Marine Physical Laboratory, Scripps Institution of Oceanography, University of California San Diego, Technical Note 271, May 2007. 60