At Xweather, we understand the importance of having accurate forecasts - which is why we offer our unique, proprietary forecast data to our clients. The importance and specific degree of forecast accuracy will depend heavily on use case and industry needs. Errors within forecast accuracy can come from a number of sources, including data sources available at the location, geographic features of or around the location, time of year, lead time of interest, and the complexity of typical weather in the region.
In the event that you’d like to check the accuracy of our forecast data, this blog post will act as a guide to creating your own forecast accuracy report. Provided you already have your weather data available, we estimate the working time of this report to be about five hours. Note that as we walk through this, we’ll be using JavaScript, but it is possible to run forecast accuracy reports using other languages as well.
Gather the following
Before implementing a forecast accuracy querying schedule, you will need to collect the following data:
The locations you want to test: We recommend locations that are relevant to your business application.
The locations of the ICAO stations that will be used for observational data: These should be as close as possible to your forecast accuracy locations.
Weather data for a range of at least 3 to 6 months: This is the most important factor for an accurate report. A minimum of 2 months’ data may be sufficient if the area of focus hasn’t seen severe weather within that time range. However, severe or abnormal weather in your location of concern can skew the forecast accuracy, which is why we recommend a longer period of data gathering.
Start querying
After all the necessary information is gathered, you can start making two types of queries: Forecast and observational. Each will contain the same information, but the difference in the values returned will be the forecast accuracy report data needed.
Here’s an example of a forecast query response:
First, a forecast query will be made for the chosen location (as seen above). We recommend a lead time of 1 to 3 hours. This means a forecast will be made for anywhere from 1 to 3 hours from now, based on your input. Then, when that time occurs, an observational query will be made (see below).
An example of an observations archive query response:
Second, the two above queries will be consistently run for a period - preferably of 3 to 6 months - as detailed earlier. We recommend storing the data received during this timeframe in a database. The comparison between forecasted values and observed values can now be made, such as standard deviation or boxplots.
Note: For the purpose of forecast accuracy reports, observations is used as it reports the raw data from the closest weather station. This is the data most relevant for calculating forecast accuracy. For applications requiring current conditions outside of forecast validation, we strongly recommend using the conditions endpoint, as it interpolates observations with forecast data to provide spatial and temporal data.
Things to consider
The biggest factor in preventing forecasts from being used in the report is observational data that has unexpected gaps. This can occur for any number of reasons but is most commonly caused when a station stops reporting data. The below pseudocode will check for any data gaps that may have occurred.
Lastly, we recommend converting a few forecast and observations values to make comparison easier:
Wind direction: Transform these values to the traditional 45-degree cardinal directions.
Sky cover: Sky values within our forecast are 0-100; most observation stations report okta values (0-8). Convert our value to the nearest okta to make data comparison easier.
In conclusion
Generating a forecast accuracy report may seem like a daunting task, but we hope this guide serves as a helpful starting point and gives you confidence in Xweather's forecast data accuracy. Reach out to us if you have any questions about implementing additional systems to automatically query data.