DIPS Technology Stack

Distributed and Intelligence Power System (DIPS)
Partner: MeRC, Net Zero Initiative, Monash Power Engineering Advanced Research Lab (PEARL), Monash Data Future Institute

DIPS platform aims to provide a secure, reliable and scalable data processing tool to process, visualise and analyse the sheer volume of data coming from the high-frequency smart devices in a power grid network. These capabilities have been achieved by employing the technologies well known in the world of IoT and big data such as openPDC, Apache Kafka, MQTT, MongoDB and Grafana. 

OpenPDC is an open-source phasor data concentrator software system designed by Grid Protection Alliance (GPA), to stream synchrophasor time-series data in real-time from many hundreds of input devices such as PMUs and distribute data (both real-time and historical) to consumers (applications or people) of this data.

Confluent Kafka
Confluent Kafka is one of the most widely used distributed data streaming platforms. It is a framework for storing, reading and analysing streaming data. It is a publish-subscribe based durable messaging system exchanging data between processes, applications, and servers.

MQTT is a standard used for connecting remote/edge devices and for messaging between device to cloud and cloud to device with a small code footprint and minimal network bandwidth. MQTT today is used in a wide variety of industries, such as automotive, manufacturing, telecommunications, oil and gas etc.

MongoDB is a NoSQL database used a cluster for persistence data storage over the research cloud. It is an open source NoSQL database optimised for data ingestion, indexing, fast query processing with great ease, speed, scalability and at a lower cost.

Grafana, a data visualisation tool will be added in the DIPS workflow to visualise, explore and analyse the waveform of the PMU data before downloading it. An open source integration between MongoDB and Grafana has been used for this purpose. The integration supports any valid MongoDB query and is able to extract statistical information like histograms and ranges of the data. Further analysis can always be done by the researcher on the other side after downloading.

PyTorch is a deep learning framework with both Python and C APIs, supporting scalable and distributed training. It has a robust ecosystem, supporting applications in vision, NLP, audio and more.

NVIDIA Triton is an AI inference server that streamlines deployment of models on GPU or CPU infrastructure. It supports all major machine learning frameworks and scales well, supporting multi-GPU, multi-node inference on large models that cannot fit in a single GPU.