Ask an Expert: Catching up with Srinath Perera

Srinath Perera is vice president of research at WSO2. He is a scientist, software architect, author, and speaker. He is also a key architect behind Apache Axis2 and WSO2 Stream Processor. We caught up with Srinath recently to get his take on the significance of Streaming SQL, the future of open source stream processing solutions, and why we must learn to think, question, and see beyond the obvious.

1. What has your journey at WSO2 been like?

This is my ninth year at WSO2, but I have been working with Sanjiva Weerawarana on similar technologies since 2003. Yes, it’s been close to 15 years, and it’s been a lot of fun. I have worked on a wide variety of challenging problems, and have worked with many brilliant individuals who will make good stories for one’s grandchildren one day. I have done a lot more than I imagined years ago.

2. For agile digital businesses, the availability of business insights is a significant factor in gaining a competitive advantage. How does WSO2 Stream Processor help?

Our product can easily plug-in to a user’s system and collect data. You could then write queries using Streaming SQL to detect important conditions. Streaming SQL is similar to SQL, but works on data streams instead of data tables. The former is flowing, while the latter is stored on a disk.

Compared to what our competitors offer, we have very powerful Streaming SQL with operators most others do not have. We enable you to use machine learning models within Streaming SQL itself. Also, if you are looking for a small deployment, our server can run a HA deployment with only two nodes and process about 100,000 events/second. If you are looking for a large deployment, we can run on top of Kafka. In the event you are unsure or undecided, you can always start small and later switch to Kafka without changing any code.

3. What does the future hold for open source stream processing solutions?

In my opinion, stream processing has not become mainstream yet. People are still figuring out analytics. It’s not easy to find developers who excel in analytics. Stream processing has to wait for that adoption to play out. No one will try to do real-time before they figure out basic analytics; that is unless you have specialized use cases such as for stock markets, surveillance, and anomaly detection.

4. What are the benefits of an open source stream processing solution?

I think there’s a growing trend for middleware as an open source model. They use complex code, support a wide variety of use cases, and are used by many. We are increasingly made aware that products are best built using the open source model. I think there’s no better testament than Microsoft, a company that hated open-source, but has now embraced it.

5. How did you start working in stream processing?

A long time ago, in 2007, while I was doing a Ph.D, we worked on a paper comparing Complex Event Processors (or CEPs, which is an older name for stream processing) and rule-based systems. I was fascinated by the technology, and after I joined WSO2, I supervised an undergraduate thesis project to build an open-source CEP engine. This was in 2011 - well before stream processing became cool! It was called WSO2 Complex Event Processor back then and was later renamed WSO2 Stream Processor.

6. What is your proudest accomplishment in recent times?

In general, it is the role I have played with Apache Axis2. However, if you want me to choose something recent, I suppose my work with the WSO2 Research Team stands out. Some good work will be made public soon. I have also worked with Paul Fremantle, WSO2’s CTO, to build a framework to evaluate different emerging technologies. You will hear more about this too soon.

7. What advice would you like to give a budding developer or an architect to better their career?

There is this quote that I love, "Wisdom is tolerance of cognitive dissonance.” It took me awhile to understand what it meant. We all interpret how the world works, but when we discover things that do not match our way of thinking, we ignore them. However, the world is more complicated than that. By understanding those mismatches and by learning through struggle and discomfort, we achieve true wisdom. That is what that quote conveys.

I would say learn to think, question, and see beyond the obvious. I refuse to tell people I work with how to solve something. Instead, I tell them, “Tell me how you will solve it and then I will complain.” I think they are used to it now. That way, we all use put our critical thinking skills to good use and one day, they will not need me for guidance.

To learn more about Srinath's work, follow him on Twitter and read his blog.