What to expect in WSO2 CEP 4.0

WSO2 Complex Event Processor 4.0 is released! A long waited CEP release from WSO2. This is a major break through among current CEP products.

Here are some notable features it has introduced :

Scale and Performance 

1. Support for distributed processing with Apache Storm - WSO2 CEP runs Siddhi CEP on Apache Storm in a distributed mode while providing an SQL like Siddhi Query Language to define queries on Storm. 
2. Faster CEP engine with WSO2 Siddhi 3.0 - With LMAX Disruptor and improved memory management, Siddhi was able to run even faster than its predecessors doing ~ 4M Events/Sec
3. HA Support - Supporting Hot-Worm High Availability with zero downtime
4. More effective native thrift/binary transports 

Usability 

1. Realtime Dashboard support - Support for gadget generation wizard and customisable realtime dashboard
2. Easy to use management console for business users - Ability to tweak queries with a form based UI without knowing the implementation and complexities of SiddhiQL, Storm and  Java 
3. Event simulation capability - Try sending events easily 
4. Siddhi query try-it - Try writing queries 
5. Event flow visualisation - See how you the event flow pipeline will look like

Enhancements  

1. New protocols introduced such as MQTT, Kafka, Websocket - Support for many new and improved event receiver and publisher capabilities
2. Enhanced query language with more extensions -  Such as Function, Stream Processor, Window and Aggregate Function Processor
3. Support for machine learning via ml, pmml (Predictive Model Markup Language) extensions - To run machine learning model on realtime
4. Support for Natural Language processing via nlp (Natural Language Processing) extension - Supported via Stanford NLP
5. Geo processing capabilities via geo extension and Geo Dashboard - Dashboard having the capability to monitor and apply queries on geo spaces. 
6. Support for scripting using JavaScript, Scala and R - Can use scripts to write CEP logic!

This release not only overcomes the scaling bottleneck of previous WSO2 CEP versions, but it also provides a seamless query language that allows uses to run the same queries both on single node and on distributed environment, making WSO2 CEP 4.0 as a unique product in the market. 

You can download WSO2 CEP 4.0 from here 

and a quick getting started guide can be found here 

Try it out, and for questions please post on StackOverflow with tag 'wso2cep'. The community and the WSO2 team will be very happy to answer :)

You can contribute your code e.g. extensions and adopters, such that it will be maintained and released on future product releases:

For Extensions and Siddhi improvements please send the pull request to Siddhi repo and 

For adopters and other contributions please send to carbon-analytics-common carbon-event-processing and product-cep repos :) 

Be Realtime !!

Make it fast for everyone

I spoked at SLASSCOM Tech talk 2015 on performance. I have attached the slides below. This mainly covers how we need to think in terms of performance from product design, choosing technologies and upto testing. Here I have taken examples form middleware products. 

The topic "Make it fast for everyone" simply means that when you implement a solution that can be used by many other solutions and then when you make your solution faster and better performing then you are basically making all the dependent components to do their task faster. 

Make it fast for everyone - performance and middleware design from Sriskandarajah Suhothayan

Becoming a Master in Apache Maven 3

Writing programs in Java is cool, its a language thats very powerful which have right amount of flexibility that makes a developers life a hell easy. But when it comes to compiling, building and managing releases of a project its not that easy, it also has the same issues encountered by other programming languages.

To solve this problem, build tools like Apache ANT and Apache Maven have emerged. ANT is very flexible tool which allows users to do almost any thing when it comes to build, maintenance and releases. Having said that since its so flexible its quite hard to configure and manage, every project using ANT uses it in their own way and hence the projects using ANT looses their consistency. At the same time when we look at Apache Maven which is not flexible as ANT by default, but it follows an amazing concept "Convention over configuration" which give that right mix of convention and configuration for you to easily create, build, deploy and even manage releases at an enterprise level.

For examples Maven always works with defaults, and you can easily create and build a maven project just with the follow snippet in the pom.xml file of your project.

<project>
     <modelVersion>4.0.0</modelVersion>
     <groupId>org.example</groupId>
     <artifactId>sample-one</artifactId>
     <version>1.0.0</version>
</project>

And this little configuration is tied up with many conventions

• The Java source code is available at {base-dir}/src/main/java
• Test cases are available at {base-dir}/src/test/java
• A JAR file type of artifact is produced
• Compiled class files are copied into {base-dir}/target/classes
• The final artifact is copied into {base-dir}/target

But there are cases where we need to go a step ahead and break the rules with reasons ! And in any case if we need to change the above defaults its just a matter of adding the Maven Build Plugin and the artifact type to the project tag as below.

<project>
     <modelVersion>4.0.0</modelVersion>
     <groupId>org.example</groupId>
     <artifactId>sample-one</artifactId>
     <version>1.0.0</version>
     <packaging>jar</packaging>
     <build>
         <sourceDirectory>${basedir}/src/main/java</sourceDirectory>
         <testSourceDirectory>${basedir}/src/test/java</testSourceDirectory>
         <outputDirectory>${basedir}/target/classes</outputDirectory>
     </build>
</project>

I came across this great book "Mastering Apache Maven 3" by Prabath Siriwardena that give you all the bits and pieces from getting started to eventually becoming a master in Maven. From this you will get to know the fundamentals and when to break the conventions with reasons. This helps you to develop and manage large, complex projects with confidence by providing an enterprise level knowledge to manage the whole Maven infrastructure.

This book covers Maven Configuration from the basics, discussing how to construct and build a  Maven project, manage Build Lifecycles, introduce useful functionalities through Maven Plugins and helps you to write your own custom plugins when needed. It also provide steps on building distributable archives using Maven Assemblies which adheres to a user-defined layout and structure,   demonstrate the usage of  Maven Archetypes for easily construct Maven projects and steps to create new Archetypes to help your developers and customers to quickly start on your project type without any configurations and replicated work. Further it also helps you to host and manage your Maven artifacts in repositories using Maven Repository Management, and most importantly explains you the Best Practices to keep your projects in line with enterprise standards.