Introduction
- Collect log and telemetry data with the support of multiple data streams created by advances software.
- Handle the large amounts of data that can be used for troubleshooting, diagnostics, reporting capabilities.
- Efficient query execution for analysis.
Features
- Store a huge amount of data.
- Support of query language to perform analysis.
- Support of the various types of data: structured, semi-structured, and unstructured.
- Support of multiple Azure services to provide end-to-end data integration solutions.
Working with Azure Data Explorer
- Create database
- Ingest Data
- Query Stored data
Sample Query to execute in Data Explorer
StormEvents
| where StartTime >= datetime(2021-01-01) and StartTime < datetime(2021-07-01)
| where State == "GUJARAT"
- You can also programmatically ingest data into data explorer using PowerShell and C#.
- We can also use power BI with a Data explorer connector to create visualization reports.
Performance of Azure Data Explorer
- Azure Data Explorer uses Kusto Engine V3 which provides advanced features for storage and build a query engine. It provides huge performance features wrt to storing logs and telemetry data plus advances query execution capabilities.
- Azure Data Explorer support various types of file formats including avro,csv,json,multijson,orc etc.
Visualization with Data Explorer
- Apart from providing storage and query capabilities, Azure data explorer also provides visualization capabilities.
- Azure Data Explorer dashboards is a web-based application that runs the query and builds dashboards.
- In addition to the dashboard, it also has native support of integration with other tools,
- Power Bi
- Excel
- Grafana
- Kibana
- ODBC Connector
- Qlik
Conclusion
We can use data explorer to store various types of data and perform analytics and ingest data efficiently.