{"id":22460,"date":"2024-09-04T18:46:47","date_gmt":"2024-09-05T01:46:47","guid":{"rendered":"https:\/\/tdengine.com\/?p=22460"},"modified":"2025-03-30T23:04:43","modified_gmt":"2025-03-31T06:04:43","slug":"using-streams-and-subscriptions-for-alerting","status":"publish","type":"post","link":"https:\/\/tdengine.com\/using-streams-and-subscriptions-for-alerting\/","title":{"rendered":"Using Streams and Subscriptions for Alerting"},"content":{"rendered":"\n<p>One of the many ways in which TDengine simplifies application development is by offering <a href=\"https:\/\/docs.tdengine.com\/advanced-features\/stream-processing\/\">stream processing<\/a> and <a href=\"https:\/\/docs.tdengine.com\/advanced-features\/data-subscription\/\">publish\/subscribe<\/a> features that are built into TDengine. This also reduces the complexity of deploying a time-series application which would usually involve setting up a stream processing platform like Kafka or using Kafka or a message broker as a pub\/sub platform.<\/p>\n\n\n\n<p>TDengine offers this functionality in the platform itself and setting up streams and topics is as simple as writing SQL or using the UI in TDengine Cloud or TDengine Enterprise to write the SQL for you.<\/p>\n\n\n<div class=\"gb-container gb-container-9a3c1f82 toc\">\n\n<div class=\"gb-headline gb-headline-89b93e98 gb-headline-text\">Table of Contents<\/div>\n\n\n\n<style>\n.toc ol {\ncounter-reset: item\n}\n.toc li {\ndisplay: block\n}\n.toc li:before {\ncontent: counters(item, \".\") \" \";\ncounter-increment: item\n}\n<\/style>\n\n\n<div class=\"gb-container gb-container-bf352232\">\n\n<ol class=\"wp-block-list nopad-li\">\n<li><a href=\"#t1\">Steps<\/a><\/li>\n\n\n\n<li><a href=\"#t3\">Data Sharing<\/a><\/li>\n\n\n\n<li><a href=\"#t4\">Conclusion<\/a><\/li>\n<\/ol>\n\n<\/div>\n<\/div>\n\n\n<h2 class=\"gb-headline gb-headline-91abf170 gb-headline-text\">Video Tutorial<\/h2>\n\n\n<div class=\"gb-container gb-container-0e862247\">\n\n<iframe width=\"100%\" height=\"100%\" style=\"position: absolute; top: 0; left: 0; width: 100%; height: 100%\" src=\"https:\/\/www.youtube-nocookie.com\/embed\/040RQ-MYR8g?si=kFtzeK8c_IZ8tcQ8\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n\n<\/div>\n\n\n<h2 class=\"gb-headline gb-headline-ef7658c0 gb-headline-text\" id=\"t1\">Steps<\/h2>\n\n\n\n<p>In the video, we demonstrate how to setup an alerting system for a solar farm, which sends an SMS when the power output of a solar panel falls below a certain threshold.<\/p>\n\n\n\n<p>In order for you to go through this video, you will have watch and follow the steps in <a href=\"https:\/\/tdengine.com\/solar-panel-monitoring-opc-ua-ingestion\/\">the previous blog<\/a> in which we set up an OPC connector to read data from a solar farm.<\/p>\n\n\n\n<p>At a high level in this video we do the following:<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><a href=\"https:\/\/docs.tdengine.com\/tdengine-reference\/sql-manual\/manage-streams\/\">Set up a stream<\/a> which aggregates the power output from each panel for the prior 5 minutes.<\/li>\n\n\n\n<li>Streams always write the aggregated results to a different supertable and, in our case, we write the aggregated data to a supertable called &#8220;p5&#8221;<\/li>\n\n\n\n<li>We then <a href=\"https:\/\/docs.tdengine.com\/tdengine-reference\/sql-manual\/manage-topics-and-consumer-groups\/\">set up a topic<\/a> which looks at the &#8220;p5&#8221; supertable and writes data to a topic when the output power falls below a threshold that we have set to be 10.<\/li>\n\n\n\n<li>We then use the sample code provided by TDengine to create a Python application to subscribe to the topic.<\/li>\n\n\n\n<li>We use the Twilio API to send an SMS to a number when data is published on this topic.<\/li>\n<\/ol>\n\n\n\n<p>Note that, you can design any kind of alerting system you want and are not limited to using Twilio APIs.<\/p>\n\n\n\n<p>Also note that in a real production system for a solar farm, your application would need to be more robust and, for example, lookup the sunset and sunrise times to ensure you don&#8217;t send alerts between those hours.<\/p>\n\n\n\n<h2 class=\"gb-headline gb-headline-164ee67c gb-headline-text\" id=\"t3\">Data Sharing<\/h2>\n\n\n\n<p>While we have not shown this in the video, data subscriptions can also be used to share data with 3rd party stakeholders. TDengine Cloud does not have a limit on the number of users for your organization. Since data subscriptions are created using SQL, you can get very granular in terms of the data you share. Of course, with data subscriptions, 3rd parties would have to consume the data in the subscription using TDengine APIs and then do something with the data in their application. But sharing the data is very simple as illustrated below.<\/p>\n\n\n\n<p>In Fig 1 below, you can see the topic that was created in the video.<\/p>\n\n\n\n<figure class=\"gb-block-image gb-block-image-dbfa183c\"><img decoding=\"async\" width=\"2674\" height=\"480\" class=\"gb-image gb-image-dbfa183c\" src=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01.png?strip=all&sharp=1\" alt=\"\" title=\"streams-01\" srcset=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01.png?strip=all&amp;sharp=1 2674w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01-300x54.png?strip=all&amp;sharp=1 300w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01-1024x184.png?strip=all&amp;sharp=1 1024w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01-768x138.png?strip=all&amp;sharp=1 768w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01-1536x276.png?strip=all&amp;sharp=1 1536w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01-2048x368.png?strip=all&amp;sharp=1 2048w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01.png?strip=all&amp;sharp=1&amp;w=534 534w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01.png?strip=all&amp;sharp=1&amp;w=1604 1604w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01.png?strip=all&amp;sharp=1&amp;w=2139 2139w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01.png?strip=all&amp;sharp=1&amp;w=450 450w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-01.png?strip=all&amp;sharp=1&amp;w=1920 1920w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/figure>\n\n\n\n<figure class=\"gb-block-image gb-block-image-1344dd43\"><img decoding=\"async\" width=\"2682\" height=\"516\" class=\"gb-image gb-image-1344dd43\" src=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02.png?strip=all&sharp=1\" alt=\"\" title=\"streams-02\" srcset=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02.png?strip=all&amp;sharp=1 2682w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02-300x58.png?strip=all&amp;sharp=1 300w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02-1024x197.png?strip=all&amp;sharp=1 1024w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02-768x148.png?strip=all&amp;sharp=1 768w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02-1536x296.png?strip=all&amp;sharp=1 1536w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02-2048x394.png?strip=all&amp;sharp=1 2048w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02.png?strip=all&amp;sharp=1&amp;w=536 536w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02.png?strip=all&amp;sharp=1&amp;w=1609 1609w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02.png?strip=all&amp;sharp=1&amp;w=2145 2145w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02.png?strip=all&amp;sharp=1&amp;w=450 450w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-02.png?strip=all&amp;sharp=1&amp;w=1920 1920w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/figure>\n\n\n\n<p>This takes you to the next screen. Make sure the right topic is showing in the drop down and then click on the &#8220;Add user to consume this topic&#8221; button.<\/p>\n\n\n\n<figure class=\"gb-block-image gb-block-image-ef2df9b7\"><img decoding=\"async\" width=\"535\" height=\"657\" class=\"gb-image gb-image-ef2df9b7\" src=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-03.png?strip=all&sharp=1\" alt=\"\" title=\"streams-03\" srcset=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-03.png?strip=all&amp;sharp=1 535w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-03-244x300.png?strip=all&amp;sharp=1 244w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-03.png?strip=all&amp;sharp=1&amp;w=107 107w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-03.png?strip=all&amp;sharp=1&amp;w=321 321w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-03.png?strip=all&amp;sharp=1&amp;w=428 428w\" sizes=\"(max-width: 535px) 100vw, 535px\" \/><\/figure>\n\n\n\n<figure class=\"gb-block-image gb-block-image-16f07c5f\"><img decoding=\"async\" width=\"1766\" height=\"215\" class=\"gb-image gb-image-16f07c5f\" src=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04.png?strip=all&sharp=1\" alt=\"\" title=\"streams-04\" srcset=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04.png?strip=all&amp;sharp=1 1766w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04-300x37.png?strip=all&amp;sharp=1 300w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04-1024x125.png?strip=all&amp;sharp=1 1024w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04-768x93.png?strip=all&amp;sharp=1 768w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04-1536x187.png?strip=all&amp;sharp=1 1536w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04.png?strip=all&amp;sharp=1&amp;w=353 353w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04.png?strip=all&amp;sharp=1&amp;w=706 706w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04.png?strip=all&amp;sharp=1&amp;w=1412 1412w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-04.png?strip=all&amp;sharp=1&amp;w=450 450w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/figure>\n\n\n\n<p>This user also needs to be given &#8220;read&#8221; permission on the &#8220;p5&#8221; supertable on which this topic is created.<\/p>\n\n\n\n<figure class=\"gb-block-image gb-block-image-02ed1f91\"><img decoding=\"async\" width=\"1814\" height=\"589\" class=\"gb-image gb-image-02ed1f91\" src=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05.png?strip=all&sharp=1\" alt=\"\" title=\"streams-05\" srcset=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05.png?strip=all&amp;sharp=1 1814w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05-300x97.png?strip=all&amp;sharp=1 300w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05-1024x332.png?strip=all&amp;sharp=1 1024w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05-768x249.png?strip=all&amp;sharp=1 768w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05-1536x499.png?strip=all&amp;sharp=1 1536w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05.png?strip=all&amp;sharp=1&amp;w=362 362w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05.png?strip=all&amp;sharp=1&amp;w=1088 1088w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05.png?strip=all&amp;sharp=1&amp;w=1451 1451w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-05.png?strip=all&amp;sharp=1&amp;w=450 450w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><\/figure>\n\n\n\n<p>Then, when this user logs into their account, they will see the following in the &#8220;Topics&#8221; screen.<\/p>\n\n\n\n<figure class=\"gb-block-image gb-block-image-bc26b2d1\"><img decoding=\"async\" width=\"1190\" height=\"342\" class=\"gb-image gb-image-bc26b2d1\" src=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06.png?strip=all&sharp=1\" alt=\"\" title=\"streams-06\" srcset=\"https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06.png?strip=all&amp;sharp=1 1190w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06-300x86.png?strip=all&amp;sharp=1 300w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06-1024x294.png?strip=all&amp;sharp=1 1024w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06-768x221.png?strip=all&amp;sharp=1 768w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06.png?strip=all&amp;sharp=1&amp;w=238 238w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06.png?strip=all&amp;sharp=1&amp;w=476 476w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06.png?strip=all&amp;sharp=1&amp;w=714 714w, https:\/\/eujqw4hwudm.exactdn.com\/wp-content\/uploads\/streams-06.png?strip=all&amp;sharp=1&amp;w=952 952w\" sizes=\"(max-width: 1190px) 100vw, 1190px\" \/><\/figure>\n\n\n\n<h2 class=\"gb-headline gb-headline-66640e9e gb-headline-text\" id=\"t4\">Conclusion<\/h2>\n\n\n\n<p>To conclude, using the Stream Processing and Subscriptions functionality in TDengine allows you to create fairly powerful applications and allows you to share data at a very granular level with many stakeholders with just an email address and role-based access control.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Stream processing and subscriptions allow you to create powerful applications and share data at a very granular level with just an email address and RBAC.<\/p>\n","protected":false},"author":81,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"content-type":"","footnotes":""},"categories":[21],"tags":[],"ppma_author":[167],"class_list":["post-22460","post","type-post","status-publish","format-standard","hentry","category-engineering"],"authors":[{"term_id":167,"user_id":81,"is_guest":0,"slug":"chait","display_name":"Chait Diwadkar","avatar_url":{"url":"https:\/\/tdengine.com\/wp-content\/uploads\/29.03-05-cdiwadkar.jpg","url2x":"https:\/\/tdengine.com\/wp-content\/uploads\/29.03-05-cdiwadkar.jpg"},"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/posts\/22460","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/users\/81"}],"replies":[{"embeddable":true,"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/comments?post=22460"}],"version-history":[{"count":7,"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/posts\/22460\/revisions"}],"predecessor-version":[{"id":23324,"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/posts\/22460\/revisions\/23324"}],"wp:attachment":[{"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/media?parent=22460"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/categories?post=22460"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/tags?post=22460"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/tdengine.com\/wp-json\/wp\/v2\/ppma_author?post=22460"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}