Tuesday, January 12, 2016

Adding Statistics to Siddhi 3

A new statistics feature is added to Siddhi 3.0.5.

This will help users to find how their queries are behaving and to understand what query to optimise to get better performance and to optimise memory consumption.

This is very easy to enable and you can enable this by just adding

"@plan:statistics(reporter = 'console', interval = '5' )"

to the execution plan will start statistics reporting on console every 5 seconds.

You will be able to and output something like
1/12/16 12:01:39 AM ============================================================

-- Gauges ----------------------------------------------------------------------
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query1.memory
             value = 548804
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query2.memory
             value = 556686

-- Meters ----------------------------------------------------------------------
org.wso2.siddhi.stream.cseEventStream.throughput
             count = 2
         mean rate = 0.37 events/second
     1-minute rate = 0.40 events/second
     5-minute rate = 0.40 events/second
    15-minute rate = 0.40 events/second
org.wso2.siddhi.stream.cseEventStream2.throughput
             count = 0
         mean rate = 0.00 events/second
     1-minute rate = 0.00 events/second
     5-minute rate = 0.00 events/second
    15-minute rate = 0.00 events/second
org.wso2.siddhi.stream.outputStream.throughput
             count = 3
         mean rate = 0.57 events/second
     1-minute rate = 0.60 events/second
     5-minute rate = 0.60 events/second
    15-minute rate = 0.60 events/second

-- Timers ----------------------------------------------------------------------
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query1.latency
             count = 2
         mean rate = 0.38 calls/second
     1-minute rate = 0.40 calls/second
     5-minute rate = 0.40 calls/second
    15-minute rate = 0.40 calls/second
               min = 0.01 milliseconds
               max = 0.08 milliseconds
              mean = 0.05 milliseconds
            stddev = 0.04 milliseconds
            median = 0.08 milliseconds
              75% <= 0.08 milliseconds
              95% <= 0.08 milliseconds
              98% <= 0.08 milliseconds
              99% <= 0.08 milliseconds
            99.9% <= 0.08 milliseconds
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query2.latency
             count = 2
         mean rate = 0.38 calls/second
     1-minute rate = 0.40 calls/second
     5-minute rate = 0.40 calls/second
    15-minute rate = 0.40 calls/second
               min = 0.01 milliseconds
               max = 0.01 milliseconds
              mean = 0.01 milliseconds
            stddev = 0.00 milliseconds
            median = 0.01 milliseconds
              75% <= 0.01 milliseconds
              95% <= 0.01 milliseconds
              98% <= 0.01 milliseconds
              99% <= 0.01 milliseconds
            99.9% <= 0.01 milliseconds

You can also add 

"@plan:statistics(reporter = 'jmx')"

to report stats via jmx.

You can find a sample implementations here.
https://github.com/wso2/siddhi/blob/master/modules/siddhi-core/src/test/java/org/wso2/siddhi/core/managment/StatisticsTestCase.java

Siddhi also allows users to add custom reporting tools. To do so.
Just implement the StatisticsTrackerFactory interface

package org.wso2.siddhi.core.util.statistics;
import org.wso2.siddhi.query.api.annotation.Element;
import java.util.List;

public interface StatisticsTrackerFactory {
    LatencyTracker createLatencyTracker(String name, StatisticsManager statisticsManager);
    ThroughputTracker createThroughputTracker(String name, StatisticsManager statisticsManager);
    MemoryUsageTracker createMemoryUsageTracker(StatisticsManager statisticsManager);
    StatisticsManager createStatisticsManager(List<Element> elements);
}

And add that to the Siddhi Manager
siddhiManager.setStatisticsConfiguration(new StatisticsConfiguration(new MyMetricsFactory()));




Happy Coding :) 

9 comments:

  1. Thanks for sample, but you don't that it will be overload your script? I not pro in this, but so much object will overload client system!
    http://mypaperhelper

    ReplyDelete
  2. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command.
    Hadoop Training in Chennai

    ReplyDelete
  3. I like to get up early to go out and breathe fresh air. I feel that it is good for health and a good habit
    http://19216811ll.com

    ReplyDelete
  4. Very Well Written Article on Hadoop Technology. Please Post More Post of this Technology To grab latest Updates and Information.
    Hadoop Training in Bangalore

    ReplyDelete
  5. Thanks for your post. This is excellent information. The list of your blogs is very helpful for those who want to learn, It is amazing!!! You have been helping many application.
    best selenium training in chennai | best selenium training institute in chennai selenium training in chennai | best selenium training in chennai | selenium training in Velachery

    ReplyDelete
  6. In the beginning, I would like to thank you much about this great post. Its very useful and helpful for anyone looking for tips. I like your writing style and I hope you will keep doing this good working.
    Data Science Course in Chennai
    Data Science Training in Chennai
    Blue Prism Training Chennai
    AWS Training in Chennai
    Digital Marketing Course in Chennai
    Data Science Training in Porur
    Data Science Training in Adyar
    Data Science Training in Velachery

    ReplyDelete