Role of a CEP System in Cloud Computing

Cloud Computing is a concept in where various services are made available through a cloud ( a network of resources which is a part of the internet), to users, without any knowledge, control, or privileged access to the underlying infrastructure that provides services. The services are necessarily IT related business applications that are more or less extensions to the traditional computing systems.

Clouds that provide services are backed by huge infrastructures of resources and handle large number of clients or users to which it provides services. The number of clients serviced by a particular cloud at any given instance can be in the magnitude of millions to billions depending on the type of service provided. This gives rise to the importance of a proper event management system in order to track usage and assist workflows.

Complex Event Processing Systems are vested with the capability of handling large streams of events and identifying useful patterns and trigger events based on them. Thus, it is clearly evident that a CEP system is perhaps the ultimate event manager or synchronizer demanded by a cloud. But, the biggest question is how to integrate a CEP system to the cloud with minimal effort.

SCI-Flex is a good example of how one could leverage the capabilities of a CEP system in event management in cloud computing. The infrastructure provided by SCI-Flex easily blends to the infrastructure of a cloud that provides services. The system can in fact be positioned as a central hub or a cluster of nodes and still have the same experience due to the way in which it being implemented.

We are hoping to add more information on integrating SCI-Flex to your cloud. Join the SCI-Flex developer list to learn more.

Internet Relay Chat Discussion Channel

SCI-Flex being an Open Source project from its inception works primarily around a developer mailing list. However, in order to speed up the development activities and to have real time chat between developers, we decided that it would be better to have a public discussion channel on our behalf. Do to the immense popularity and guaranteed stability and 24/7 service found in the freenode IRC network we decided that it would be great to host the SCI-Flex discussion channel on freenode.

Since SCI-Flex is an Open Source project and since we need to reveal all forms of communication and important project related discussions to all the members of the community, it would invariably be good to have the channel publicly logged and also available to anyone interested in seeing it. Due to limitations we experience, so far there isn't a public chat log in place. We are at present working on this feature and expect to have this in place by the end of August 2008.

We invite all those who are interested in SCI-Flex to join ##sci-flex @ chat.freenode.net. More information on freenode can be found here. You can learn about IRC in here. Pidgin and Mibbit (web based) are some of the IRC clients I use.

SCI-Flex introduced in webinar By Paul

SCI-Flex was featured on "Complex Event Processing with Esper and WSO2 ESB" - Webinar by Paul Fremantle which was held on 29th July 2008. Paul Fremantle, Co-Founder and CTO of WSO2, spoke about Complex Event processing capabilities of the WSO2 ESB, in the fourth and final part of the webinar series on the WSO2 ESB. WSO2 ESB is a enterprise grade lightweight ESB based on the popular Apache Synapse ESB, and is presently at version 1.7.

Paul discussed about how the WSO2 ESB can easily be configured to listen and monitor various events flowing through the WSO2 ESB using the Esper open source complex event processing system, and identify patterns and useful events. As a mechanism of integrating Esper into the WSO2 ESB, Paul introduced SCI-Flex which is project focused on maximizing the use of Complex Event Processing in the field of Service Oriented Architecture through a flexible integration.

The SCI-Flex team is now working on further improving the Synapse-Esper mediator which serves as the primary medium for connecting Synapse and Esper which can infact be used with the WSO2 ESB as well. We are very much thankful to Paul for mentioning about SCI-Flex in his discussion. More information on the webinar and subject matter can be found here.

SCI-Flex Milestone 1 Plan

The SCI-Flex team is at present working on the 1st milestone of SCI-Flex due in early September 2008. The development of SCI-Flex was broken down in to several parts so that basic functionality will be made available from early stages of development. It was decided that we would primarily focus on the development of the Synapse-Esper mediator at the beginning and focus on other aspects afterwards.

The requirements for the 1st Milestone as agreed by our team can be expressed as,

1. Make it possible for Synapse to talk to Esper. All messages that go through Synapse will be routed to Esper during this stage of our research. By achieving this, we need not worry about what messages would go to the CEP and what wouldn't.
2. Attach registry support to our SOA+CEP infrastructure. We will be making it possible for someone to store a CEP query, EQL/EPL in a back-end registry, which will then be fetched by Synapse by default and then applied upon the event stream that will be flushed through the CEP. We will not worry about how we will be fetching various queries, or fetching a particular query on demand for this milestone.
3. Make it possible for someone to specify a Endpoint Reference (EPR) along with the query, so that Synapse will redirect the response obtained through the CEP to the specified EPR. The EPR will also be made available through the registry.

The development activities related to Milestone 1, is carried on the SCI-Flex M1 branch. We'll be merging it to the trunk once we've finished work, most probably in early September.

CEP Enhances SOA's Eventing Capabilities

Eventing in a SOA environment is crucial for enterprise applications. According to the WS-Eventing specification, web services are capable of monitoring and notifying status and also performing a defined operation when a certain event is triggered. However, like any other specification that explains web based extensions to event driven architectures, this lacks the true power of making decisions on events based on a stream of events itself.

The combination of CEP and SOA unleashes the true potential of a complete event driven architecture implemented over a web based or any similar network communication protocol based architecture that is supported by existing SOA networks. Thus, one could describe CEP over SOA as a universal replacement to existing Publisher-Subscriber systems such as,

1. JMS Topics
2. XMPP
3. WS-Eventing
4. Atom Pub/Sub

Decisions made through CEP systems involve identifying meaningful events from a number of events, extracting the product of a reaction between several events, understanding event patterns etc. The possibilities in handling bulky event sources are immense. Thus, the positive addition to an SOA environment through a CEP system is limitless.

It is clearly understood that CEP enhances SOA's eventing capabilities. In addition to these marginal gains, a complete CEP installation over an SOA network makes you more productive. We are looking forward to introduce such other benefits.