Monday, January 11, 2016

Adding Statistics to Siddhi 3

A new statistics feature is added to Siddhi 3.0.5.

This will help users to find how their queries are behaving and to understand what query to optimise to get better performance and to optimise memory consumption.

This is very easy to enable and you can enable this by just adding

"@plan:statistics(reporter = 'console', interval = '5' )"

to the execution plan will start statistics reporting on console every 5 seconds.

You will be able to and output something like
1/12/16 12:01:39 AM ============================================================

-- Gauges ----------------------------------------------------------------------
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query1.memory
             value = 548804
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query2.memory
             value = 556686

-- Meters ----------------------------------------------------------------------
org.wso2.siddhi.stream.cseEventStream.throughput
             count = 2
         mean rate = 0.37 events/second
     1-minute rate = 0.40 events/second
     5-minute rate = 0.40 events/second
    15-minute rate = 0.40 events/second
org.wso2.siddhi.stream.cseEventStream2.throughput
             count = 0
         mean rate = 0.00 events/second
     1-minute rate = 0.00 events/second
     5-minute rate = 0.00 events/second
    15-minute rate = 0.00 events/second
org.wso2.siddhi.stream.outputStream.throughput
             count = 3
         mean rate = 0.57 events/second
     1-minute rate = 0.60 events/second
     5-minute rate = 0.60 events/second
    15-minute rate = 0.60 events/second

-- Timers ----------------------------------------------------------------------
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query1.latency
             count = 2
         mean rate = 0.38 calls/second
     1-minute rate = 0.40 calls/second
     5-minute rate = 0.40 calls/second
    15-minute rate = 0.40 calls/second
               min = 0.01 milliseconds
               max = 0.08 milliseconds
              mean = 0.05 milliseconds
            stddev = 0.04 milliseconds
            median = 0.08 milliseconds
              75% <= 0.08 milliseconds
              95% <= 0.08 milliseconds
              98% <= 0.08 milliseconds
              99% <= 0.08 milliseconds
            99.9% <= 0.08 milliseconds
org.wso2.siddhi.executionplan.5910c3f3-1af3-43bd-ae6e-06b1a959eaf3.query2.latency
             count = 2
         mean rate = 0.38 calls/second
     1-minute rate = 0.40 calls/second
     5-minute rate = 0.40 calls/second
    15-minute rate = 0.40 calls/second
               min = 0.01 milliseconds
               max = 0.01 milliseconds
              mean = 0.01 milliseconds
            stddev = 0.00 milliseconds
            median = 0.01 milliseconds
              75% <= 0.01 milliseconds
              95% <= 0.01 milliseconds
              98% <= 0.01 milliseconds
              99% <= 0.01 milliseconds
            99.9% <= 0.01 milliseconds

You can also add 

"@plan:statistics(reporter = 'jmx')"

to report stats via jmx.

You can find a sample implementations here.
https://github.com/wso2/siddhi/blob/master/modules/siddhi-core/src/test/java/org/wso2/siddhi/core/managment/StatisticsTestCase.java

Siddhi also allows users to add custom reporting tools. To do so.
Just implement the StatisticsTrackerFactory interface

package org.wso2.siddhi.core.util.statistics;
import org.wso2.siddhi.query.api.annotation.Element;
import java.util.List;

public interface StatisticsTrackerFactory {
    LatencyTracker createLatencyTracker(String name, StatisticsManager statisticsManager);
    ThroughputTracker createThroughputTracker(String name, StatisticsManager statisticsManager);
    MemoryUsageTracker createMemoryUsageTracker(StatisticsManager statisticsManager);
    StatisticsManager createStatisticsManager(List<Element> elements);
}

And add that to the Siddhi Manager
siddhiManager.setStatisticsConfiguration(new StatisticsConfiguration(new MyMetricsFactory()));




Happy Coding :) 

19 comments:

  1. Thanks for sample, but you don't that it will be overload your script? I not pro in this, but so much object will overload client system!
    http://mypaperhelper

    ReplyDelete
  2. It's interesting that many of the bloggers to helped clarify a few things for me as well as giving.Most of ideas can be nice content.The people to give them a good shake to get your point and across the command.
    Hadoop Training in Chennai

    ReplyDelete
  3. Very Well Written Article on Hadoop Technology. Please Post More Post of this Technology To grab latest Updates and Information.
    Hadoop Training in Bangalore

    ReplyDelete
  4. Excellent blog I visit this blog it's really awesome.
    Yaaron Studios is one of the rapidly growing editing studios in Hyderabad. We are the best Video Editing services in Hyderabad. We provides best graphic works like logo reveals, corporate presentation Etc. And also we gives the best Outdoor/Indoor shoots and Ad Making services.

    video editors studio in hyderabad
    short film editors in hyderabad
    corporate video editing studio in hyderabad
    ad making company in hyderabad

    ReplyDelete
  5. Thank you for excellent article.You made an article that is interesting.
    AWS Solutions Architect courses in Bangalore with certifications.
    https://onlineidealab.com/aws-training-in-bangalore/

    ReplyDelete
  6. I rarely meet the good blog like this one, I so in love with this site...

    Judi Ceme Online
    Agen Ceme Online
    IDN Poker Online

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. I really love your website.. Excellent colors & theme. Did you develop this site yourself? Please reply back as I’m trying to create my own blog and would like to know where you got this from or what the theme is named. Many thanks!

    idn poker

    bonus poker terbaru
    download game poker
    daftar texas poker
    domino qiu qiu
    daftar pokerqiu

    ReplyDelete
  9. You have some really great articles and I believe I would be a good asset. If you ever want to take some of the load off, I’d love to write some articles for your blog in exchange for a link back to mine. Please blast me an email if interested. Many
    thanks!

    Sabung ayam bangkok

    sbobet
    sabung ayam
    tarung ayam
    s128
    website ayam
    judi ayam

    ReplyDelete
  10. I have just over and done with understanding this article and I must confess it is really a thought-provoking one. I must say it is a real outstanding work done by the well-organized writer.
    SAP training in Kolkata
    SAP course in kolkata

    ReplyDelete
  11. Thank you for writing down such a wonderful piece of content writing. I really eulogize your insights. I have come across a lot of appealing piece of information in this article that is bold.
    SAP training in Mumbai
    SAP course in Mumbai

    ReplyDelete
  12. It's late discovering this demonstration. At any rate, it's a thing to be acquainted with that there are such occasions exist. I concur with your Blog and I will have returned to investigate it more later on so please keep up your demonstration. business analytics course in surat

    ReplyDelete
  13. Some us know all relating to the compelling medium you present
    powerful steps on this blog and therefore strongly encourage
    contribution from other ones on this subject while our own child is
    truly discovering a great deal. Have fun with the remaining portion of
    the year.
    oracle training in chennai
    ASP.NET Training in Chennai
    C#.NET Training In Chennai

    ReplyDelete
  14. I appreciate your hard work. Keep posting new updates with us. This is really a wonderful post. Nice Blog Very interesting and useful information on your website. Thanks for sharing the blog and this great information which is definitely going to help us.
    ba 3rd year result name wise

    ReplyDelete