DATABASE DESCRIPTION

NEW YORK

Hi-Rez Data(TM) Climatological Series

Copyright(c) 1990-95 by ZedX, Inc.

INTRODUCTION

The databases comprising the Hi-Rez Data(TM), Climatological Series, consist of geographically-addressed integer values in a compressed, raster form. The databases are identified by state, variable, and month. The high resolution data in each monthly database were generated by applying a mathematical algorithm to the 30-year (1951-80) and (1961-90) climatological station records, which were compiled as part of the "Climatographies of the United States," by the National Climatic Data Center. The high resolution data in each annual database were derived by either averaging or summing the twelve interpolated monthly databases. The data are representative of a standard station setting at a spatial resolution of approximately one square kilometer. State and county political boundaries are included as a separate database. The high resolution boundary data were derived from the "1980 Census Digital Boundary File," which was created by the Geography Division of the Bureau of Census. The boundary database, which can be overlaid on the climatological data, has been provided for reference purposes. This printout provides a description of the database structure, a statement concerning the accuracy of a Hi-Rez Data(TM), and reference station data for checking the installation of the Hi-Rez Data(TM) into a geographic information system (GIS) or similar program.

DATA ACCURACY

The "accuracy" of the Hi-Rez Data(TM), or the extent to which an interpolated value agrees with a station record, depends on a number of factors. These factors include the quality and protocol of the original tation observations, the density of stations, the spatial scale represented by a datum, the variation and height of the surface topography across a region, the proximity of large water bodies to station locations, and, most importantly, how the data are to be integrated int o an application or interpreted for a particular decision. Before any evaluation is made to determine the accuracy of the interpolated values, a number of assumptions must be stated to clarify the physical setting represented by a datum. The first assumption is that the climatological records derived from station observations are the true values of any potential site within a geographic area. The second assumption is that the observations at a site were made with current instrument placement practices, and observation protocols. The third assumption is that all station sites are over a gently sloping, clipped grass surface that has no nearby obstructions to air flow. With these assumptions, no accounting is made for a non-standard station design, poor instrument placement, damaged or miscalibrated instruments, poor observing practices, poor exposure, or a non-grass surface cover, when arriving at a number to represent the accuracy of a datum.

The accuracy figures presented in this printout are limited to their inherent spatial variability in a data set for a region defined by state boundaries. There are a number of statistical measures that can be used to judge the collective or "average" accuracy of a data sample across a state, but few reveal the risk of using an interpolated datum in place of a station record. However, this risk can be quantified with three arithmetic measures that can computed by comparing the Hi-Rez Data(TM) to matched samples of the original station records. In order to match an interpolated datum to a station record, it was assumed that both data sets satisfied the assumptions above about the physical setting, and that the climatological data in both sets represent average values for the approximate one-square-kilometer areas (pixels) that define the spatial scale of the databases.

The three arithmetic measures are the difference between means of the two matched data sets, the mean absolute difference between the two data sets, and the absolute differences for five progressive percentages of the two matched data samples. The difference between means can be interpreted as the "bias" in the Hi-Rez Data(TM) relative to the station records serving as a truth set. The bias gives a measure of how far off, as a state average, the interpolated values are from the original station values. The mean absolute difference is a measure of how much an interpolated value differs from its station counterpart regardless of whether the difference is positive or negative. As a mean, the absolute difference, like the bias, is an average for all locations within a state. The third measure, which is also based on the absolute difference, accounts for the variability in the differences between interpolated and station values for different sites across a state. This measure consists of five absolute differences that correspond to progressive percentages of the sample size of the matched data sets.

The absolute differences as a percentage of sample size were derived in three steps. First the absolute difference was calculated for each matched pair of interpolated and station values for all station locations within a state. Second, the absolute differences were sorted in an ascending order. That is, the station with the lowest absolute difference, regardless of its geographic location, was the first entry and the station with the largest difference was the last entry. Third, an absolute difference was assigned to a percentage of the sample size based on its sorted position. Five percentages were selected: 50, 75, 90, 95, and 99%. Since the absolute difference is being presented in an unconventional form, an example may be useful to illustrate its derivation and its usefulness for quantifying the accuracy of the Hi-Rez Data(TM). Suppose a small state had matched temperature data sets of ten values each and that the absolute differences computed for the two sets were 2.8, 1.5, 0.8, 3.5, 2.1, 0.4, 1.9, 1.0, 2.5, and 1.2 F. After being sorted without regard to location, the values were stored as 0.4, 0.8, 1.0, 1.2, 1.5, 1.9, 2.1, 2.5, 2.8, and 3.5 F. The absolute difference corresponding to 50% of the sample size would be 1.5 F. The absolute difference corresponding to 90% of the sample would be 2.8 F. The absolute differences at these set percentages can be interpreted as a measure of Hi-Rez Data(TM) accuracy. A user could infer that 50% of all locations represented by the interpolated data had an absolute difference of 1.5 F or less. Similarly, the same user could infer that 90% of all locations had an absolute difference of 2.8 F or less. The user understands that the numbers represented by the absolute differences can be positive or negative. The user also understands that the progressive percentages represent a decrease in the risk of using an interpolated value as a substitute for a station value. That is, an absolute difference of 2.8 F for 90% of a sample has less risk than the same absolute difference at 50% of the same sample.

The choice of percentage for judging data accuracy depends on the data requirements and the acceptable level of risk for a particular application. It is important to note that the user is basing the above inferences on a limited sample size within a state. However, the inferences are based on the original station records, and the pattern displayed by the absolute differences is, for most states, representative of most locations within a state. As a final note, the user should be aware that the greatest absolute differences are generally found at higher elevations, along the borders of large water bodies, and along the slopes of large mountains. The three arithmetic measures chosen to evaluate the accuracy of the Hi-Rez Data(TM) are presented in Table 3 for the listed state. It should be noted that the sample size given in the table represents a subset of the total number of stations used to generate the Hi-Rez Data. The larger sample, used by the interpolation algorithm, defines a climatological zone that extends over several states including the one listed in the table. However, the sample used to compute the measures in Table 3 does include nearly all the stations in the listed state. Table 1. The difference between means (bias), the mean absolute difference, and the absolute difference as a percentage of sample size are presented for the monthly maximum temperature, minimum temperature, and precipitation totals for the listed state. The three measures were computed from matched data sets of Hi-Rez Data(TM) and station records. ACCURACY FILES

DATA SET: 1961-90 STATE: NEW YORK

Hi-Rez Data(TM) 6/95

Hi-Rez Data(TM) 6/95