Multiple Mediator Example

SCI-Flex provides an extension that connects the Open Source CEP System, Esper with the Apache Synapse ESB. Due to various possible formats of messages being exchanged between the two components, SCI-Flex extensions for Synapse-Esper mediation comes in several flavors. As of now, we have two mediators, namely the XMLMediator, which operates on XML DOM information, and the AxiomMediator, which operates on Axiom based Tree representations of XML.

Necessarily, XML and Axiom are two different ways one could look into the same problem, but, depending on the scenario, one is always better than the other. There can also be situations where the ESB and CEP communicate with information flowing from various sources. This leads to situation which would perhaps reqire both the mediators in action rather than one.

Each SCI-Flex mediator communicates with a specific instance of Esper CEP system, and adding two mediators would necessarily add two instances of Esper unless otherwise explicitly stated. This can be challenging as there can be queries that span across data handled by both the mediators. The solution is simple. It requires us to share a single instance of the CEP engine.

How it is done is what is important. Each instance of the CEP engine needs to be configured before it being used. According to how the mediator is being implemented, each mediator has the capability of configuring the CEP engine. Thus, it is important to decide on who configures the CEP engine. We have used a standard way of configuring the CEP engine in the first mediator lying in the mediation path.

Once configured, a unique instance URI can be used to access the shared CEP instance. A more detailed example that uses two event sources, a web service and a timer, is found in here.

Role of a CEP System in Cloud Computing

Cloud Computing is a concept in where various services are made available through a cloud ( a network of resources which is a part of the internet), to users, without any knowledge, control, or privileged access to the underlying infrastructure that provides services. The services are necessarily IT related business applications that are more or less extensions to the traditional computing systems.

Clouds that provide services are backed by huge infrastructures of resources and handle large number of clients or users to which it provides services. The number of clients serviced by a particular cloud at any given instance can be in the magnitude of millions to billions depending on the type of service provided. This gives rise to the importance of a proper event management system in order to track usage and assist workflows.

Complex Event Processing Systems are vested with the capability of handling large streams of events and identifying useful patterns and trigger events based on them. Thus, it is clearly evident that a CEP system is perhaps the ultimate event manager or synchronizer demanded by a cloud. But, the biggest question is how to integrate a CEP system to the cloud with minimal effort.

SCI-Flex is a good example of how one could leverage the capabilities of a CEP system in event management in cloud computing. The infrastructure provided by SCI-Flex easily blends to the infrastructure of a cloud that provides services. The system can in fact be positioned as a central hub or a cluster of nodes and still have the same experience due to the way in which it being implemented.

We are hoping to add more information on integrating SCI-Flex to your cloud. Join the SCI-Flex developer list to learn more.

Internet Relay Chat Discussion Channel

SCI-Flex being an Open Source project from its inception works primarily around a developer mailing list. However, in order to speed up the development activities and to have real time chat between developers, we decided that it would be better to have a public discussion channel on our behalf. Do to the immense popularity and guaranteed stability and 24/7 service found in the freenode IRC network we decided that it would be great to host the SCI-Flex discussion channel on freenode.

Since SCI-Flex is an Open Source project and since we need to reveal all forms of communication and important project related discussions to all the members of the community, it would invariably be good to have the channel publicly logged and also available to anyone interested in seeing it. Due to limitations we experience, so far there isn't a public chat log in place. We are at present working on this feature and expect to have this in place by the end of August 2008.

We invite all those who are interested in SCI-Flex to join ##sci-flex @ chat.freenode.net. More information on freenode can be found here. You can learn about IRC in here. Pidgin and Mibbit (web based) are some of the IRC clients I use.

SCI-Flex introduced in webinar By Paul

SCI-Flex was featured on "Complex Event Processing with Esper and WSO2 ESB" - Webinar by Paul Fremantle which was held on 29th July 2008. Paul Fremantle, Co-Founder and CTO of WSO2, spoke about Complex Event processing capabilities of the WSO2 ESB, in the fourth and final part of the webinar series on the WSO2 ESB. WSO2 ESB is a enterprise grade lightweight ESB based on the popular Apache Synapse ESB, and is presently at version 1.7.

Paul discussed about how the WSO2 ESB can easily be configured to listen and monitor various events flowing through the WSO2 ESB using the Esper open source complex event processing system, and identify patterns and useful events. As a mechanism of integrating Esper into the WSO2 ESB, Paul introduced SCI-Flex which is project focused on maximizing the use of Complex Event Processing in the field of Service Oriented Architecture through a flexible integration.

The SCI-Flex team is now working on further improving the Synapse-Esper mediator which serves as the primary medium for connecting Synapse and Esper which can infact be used with the WSO2 ESB as well. We are very much thankful to Paul for mentioning about SCI-Flex in his discussion. More information on the webinar and subject matter can be found here.

SCI-Flex Milestone 1 Plan

The SCI-Flex team is at present working on the 1st milestone of SCI-Flex due in early September 2008. The development of SCI-Flex was broken down in to several parts so that basic functionality will be made available from early stages of development. It was decided that we would primarily focus on the development of the Synapse-Esper mediator at the beginning and focus on other aspects afterwards.

The requirements for the 1st Milestone as agreed by our team can be expressed as,

1. Make it possible for Synapse to talk to Esper. All messages that go through Synapse will be routed to Esper during this stage of our research. By achieving this, we need not worry about what messages would go to the CEP and what wouldn't.
2. Attach registry support to our SOA+CEP infrastructure. We will be making it possible for someone to store a CEP query, EQL/EPL in a back-end registry, which will then be fetched by Synapse by default and then applied upon the event stream that will be flushed through the CEP. We will not worry about how we will be fetching various queries, or fetching a particular query on demand for this milestone.
3. Make it possible for someone to specify a Endpoint Reference (EPR) along with the query, so that Synapse will redirect the response obtained through the CEP to the specified EPR. The EPR will also be made available through the registry.

The development activities related to Milestone 1, is carried on the SCI-Flex M1 branch. We'll be merging it to the trunk once we've finished work, most probably in early September.