Skip to content

GIS Integration / Overview

Introduction

NovaStar data web services provide data in formats that can be consumed by Geographic Information System (GIS), web mapping, and other software that uses spatial data. This documentation describes multiple options for integrating web services with GIS software.

NovaStar web services focus on easy to use open data formats, in particular GeoJSON, which has the benefits of being in text format, can be compressed, is an open standard, is a simple format, and is a single file that can be used with many consuming applications.

Although it is possible to publish other formats such as KML or OGC web feature service, NovaStar will likely focus on GeoJSON for the foreseeable future. NovaStar uses the PostgreSQL database and the use of PostGIS is being evaluated. However, PostGIS export features are not currently used by data web services.

See the next section for general strategies for spatial data integration. Specific documentation pages are are provided for the following software solutions:

Strategies for Using Web Services Spatial Data

The general strategy for using NovaStar data web services is to request GeoJSON data that can be displayed in a map. Although conceptually this is simple, implementation is complicated by the following technical considerations, which may need to be addressed through iterative prototyping of a solution.

  • Software may not consume GeoJSON - Not all GIS software consumes GeoJSON in a simple process.
    • Challenge 1: The software solution must consume GeoJSON.
  • Data available at stations varies - Multiple stations are available in a system; however, the data available at stations can vary greatly. For example, most stations may measure precipitation; however, a smaller number of stations may measure water level, and fewer stations may measure discharge.
    • Challenge 2: Showing all stations in a map layer regardless of available data types can be confusing when data are not available for a data type.
    • Challenge 3: Showing multiple layers by "station type" (e.g., climate station) or data type requires configuring symbols that are unique and display without obscuring each other.
  • Display of time series data requires additional data handling - Time series data involve integrating the station identifier, data type (sensor parameter), and data interval in order to retrieve time series data.
    • Challenge 4: Time series that are available at a station vary and require logic to display time series in appropriate groups and with appropriate symbols. For example, one graph may show precipitation with units of inches, another graph may show water level with units of feet, and another graph may show discharge with units of cfs.
  • Display of alarm conditions requires additional data handling - Alarm conditions may not directly apply to available time series. Alarm conditions in NovaStar are defined as triggers on the scaled data reports and can be configured as instantaneous values or changes over a time interval. For example, alarm triggers can be defined for precipitation totals over a time duration, such as inches over 1 hour compared to a recurrance interval resulting in a "20 year event".
    • Challenge 5: Time series that are associated with a station data type may not correspond to alarm triggers and consequently the integration of alarms with map layers and time series requires additional logic to ensure proper interpretation.
  • Display of other data requires additional data handling - For example, stream channel cross sections and elevation profiles are specific to each station's location.
    • Challenge 6: Managing other data, returning the data in web services, and integrating with applications may require system enhancements and custom coding.
  • Missing Values must be Handled - Time series values and resulting statistics as map layer properties (attributes) may have null values due to a lack of data reports in the requested interval.
    • Challenge 7: ArcGIS Online and other software displays must handle missing data such as null values. Null values should NOT be treated as zeros in most cases but should instead be treated as "unknown" or "no report".
  • Data Must Refresh on a Reasonable Frequency - Data that are displayed in map layers, dashboard graphs, etc. reflect recent data reports. Data views must be updated on a regular frequency to ensure that the data can be acted on.
    • Challenge 8: Data must be refreshed on a regular frequency that results in good system performance while minimizing latency of data available for information products.

The above challenges are addressed in the NovaStar Operator by utilizing a combination of web services and integrated displays. For example, the map display allows selecting data display layers by data type and interval and the station dashboard automatically determines how to display time series and alarms for the station.

Creating similar map-based displays and dashboards in ArcGIS and other software can be implemented with varying levels of sophistication, depending on the level of custom configuration and coding. The following describes a strategy that leverages web services and addresses the above challenges. Enhancements to web services may be needed to address additional requirements, for example:

  • providing symbology hints
  • improving integration of data such as alarms
  • improving performance, for example by enhancing caching
  • ensuring that returned GeoJSON properties are sufficient to allow a calling application to implement an integrated solution

Query GeoJSON Map Layer With One or More Data Types

The stationStatistics service can be used to return a GeoJSON response with one or more data types, statistics, and intervals. The intervals (relative to the current time) are used to determine the sample to compute the statistic. The result is a layer with geometries matching the station coordinates and properties (attributes) that include station properties and calculated statistics. The following example returns only stations that include hourly precipitation totals. Note that the statsticFormat=flat results in a flat data representation, whereas the default is more detailed statistic data, but may not be easily used in the calling application.

https://larimerco-ns5.trilynx-novastar.systems/novastar/data/api/v1/stationStatistics?format=geojson&statisticFormat=flat&dataType=Precip&statistic=Total&interval=1Hour,1Day

See the full result, with an excerpt shown below. Note that the property (JSON elements) Precip-Total.1Hour has a name that is a combination of the web service query parameters dataType, statistic, and interval. The web service will automatically list all available data types at a station if the dataType query parameter is omitted, which will cause the number of properties to vary between stations. However, many applications require that the element names are consistent for all JSON objects. Therefore, specifying a list of data types ensures that the returned data are consistent. The NovaStar data types can be determined by using the dataTypes service and the tscatalog service can be used to list available time series.

    {
      "type" : "Feature",
      "geometry" : {
        "type" : "Point",
        "coordinates" : [
          -104.944179,
          40.473224
        ]
      },
      "properties" : {
        "numId" : 10404,
        "name" : "Weld - Poudre at WCR13",
        "description" : "CS475A Radar",
        "descriptionText" : "CS475A Radar",
        "elevation" : 4790.0,
        "id" : 422,
        "lastTimePolled" : null,
        "latitude" : 40.473224,
        "longitude" : -104.944179,
        "dashboardUrl" : "https://larimerco-ns5.trilynx-novastar.systems/novastar/operator/#/stationDashboard/10404",
        "outOfService" : false,
        "remoteTag" : "10404",
        "tagName" : "",
        "stationTypeDescription" : null,
        "stationTypeId" : 16,
        "stationTypeLine" : 0,
        "stationTypeName" : "Campbell A2",
        "stationTypeProtocol" : "ALERT2",
        "Precip-Total.1Hour" : 0.0,
        "Precip-Total.1Day" : 0.0
      }
    },

If the consuming application can only be configured to pre-select a single property (attribute), then either one statistic can be requested, or the application can be configured to select the desired element name (and others will be ignored). In this case, it may be necessary to query multiple layers using different web service URLs.

If the consuming application can allow the property (attribute) to be selected from a list, then multiple statistics can be requested and made available to the user.

In either case, appropriate symbology must be configured for the layers. The NovaStar Operator legend can be consulted for symbology examples.

Create Station Dashboard

One option for displaying a station's data is to use the stationStatistics GeoJSON service response dashboardUrl property value. This will display the NovaStar Operator station dashboard.

Another option is to build a custom dashboard using available data for a station and its points (sensors). For example, the following web services can be called to retrieve data for a station:

Performance Considerations

Some NovaStar web services have built-in caching to increase performance. However, many web services are not cached by default and result in a database query. This introduces a load on the system that needs to be managed. To address performance issues, web services should be run in a web browser as a test to ensure that a refresh frequency is appropriate. Each JSON web service response includes a responseInfo section that includes the queryMs property indicating the time to query the data, which does not consider the time to format the response or network communication time. To ensure good performance, the ratio of this time and the map refresh interval should not approach 1 and should ideally be much less than one. Additional web service enhancements may be possible to increase performance, and server capabilities can also be upgraded.

  "responseInfo" : {
    "uri" : "http://localhost/novastar-data/api/v1/stationStatistics?format=geojson",
    "uriEncoded" : "http://localhost/novastar-data/api/v1/stationStatistics?format=geojson",
    "periodStart" : null,
    "periodEnd" : "2021-12-28T12:41:34.16-07:00",
    "size" : 159,
    "queryStart" : "2021-12-28T12:41:34.16-07:00",
    "queryMs" : 5754,
    "queryTargetMs" : -999,
    "queryTargetMet" : null,
    "isCached" : false,
    "cacheExpirationTime" : null
  },

If web services are called by a limited number of applications, for example to ingest data into an ArcGIS component, then load on the NovaStar server will be minimal. However, if many applications access web services directly, it may be necessary to optimize the system configuration, for example by enabling caching. There is a trade-off between caching and latency of information.

Contact TriLynx Systems to discuss system performance optimization.