Connecting Downstream Services
A common use case for exploring and analyzing collected device data is to send data to external services and tools for analytics, alerts, machine learning models, application development, and more. The following is a high level look at the APIs available for storage access and how to use them to send data downstream to other services.
API Storage Access
While our APIs allow connections directly with data sources, there is still often a need to massage the data into the correct format for other integrations (Azure Machine Learning's automated ML feature needs data in json lines or CSV format).
When working with IoT storage data, how it is stored and what interval it is stored at is extremely important to the overall cost of the system. We break our storage into three categories that support a cost-efficient way to handle data storage and access. Cold storage contains historic data, warm storage contains near-term queryable data, and hot storage provides a way to stream individual messages to other services in real time. The following high-level walk-through outlines APIs for accessing these storage types.
This storage access documentation focuses on our pre-built APIs available to all OpenBiotech users.
Access Keys
The API keys will be located within the API tab in the dashboard.
To quickly copy storage access keys use the button
Cold Storage
For many use cases, cold storage historic data can be formatted in an efficient way to support service integrations. The APIs provided to access this data are geared at helping grab a time period of data and format it in a number of ways (JSON, CSV, JSON Lines, etc). The following is a simple example that could be used to retrieve device telemetry data for Microsoft Power BI:
curl -X GET "https://dashboard.openbiotech.co/api/data/cold/execute" -H "Authorization: Bearer {token}"
There are values to replace and adjust the parameters as desired.
The API Access Token can be located in the API section as described above.
Warm Storage
A queryable storage location, warm storage offers a way to work with data in a dynamic, no-sql way. This storage type comes with a larger cost, and for that reason, controlling the amount of data in this storage is important. The retention period can be tuned to meet application needs and cost concerns. The following is a simple example that could be used to retrieve device telemetry data for use in an application:
curl -X GET "https://dashboard.openbiotech.co/api/data/warm/explorer" -H "Authorization: Bearer {token}"
You can replace values and adjust the arguments as desired.
The API Access Token can be located in the API section as described above.
Hot Storage
Hot Storage refers to data that must be accessible immediately and reliably (referred to as “data streams"). To connect to this API, use the Hot Storage URL, API Access Token and a SignalR client in any language. SignalR is an open-source library that facilitates real-time functionality in web applications. Using this, the server application can push real-time data to the client. You can see this API in action in the 'Streaming' tab of the device data dashboard.
https://dashboard.openbiotech.co/api/data/hot/connect
Authorization: Bearer {token}
The URL and API Access Token can also be located in the API section as described above.
Downstream Services
The purpose of the Storage Access APIs is to give rapid access to device data in a number of different formats that is ready to be used with other systems. In the next sections, we will guide you on how to combine these APIs to send device data downstream to various third-party services for dashboards (Power BI, Grafana, Tableau, SQL Server), alerts (Azure Logic Apps, Grafana) and AI/ML (Azure ML).
Need help installing Git and cURL for Windows 11/10? Here's a helpful tutorial and Git download link.