Have You Checked Out the WSO2 Machine Learner Yet?

WSO2 Machine Learner is the latest addition to the WSO2 product stack. Or, to be precise, one of the latest additions: Machine Learner only held the crown for a few weeks before WSO2 Data Analytics Server become the new kid on the block.

Nevertheless, Machine Learner is something we’re particularly proud of. As you no doubt guessed, it’s a product meant for predictive business analytics: by building models and running data through them, it allows one to quite literally peek into possible futures.

Why is this important? Well, machine learning is rapidly becoming the new frontier for businesses. We’ve gone from educated guesses to dedicated analytics: now we’re trying to predict the future.

“I keep saying the sexy job in the next ten years will be statisticians. People think I’m joking, but who would’ve guessed that computer engineers would’ve been the sexy job of the 1990s?” – Hal Varian, Chief Economist at Google

Hal Varian expects statisticians to be the next sexy job because of the sheer amounts of data and intelligence that companies need to chew through these days.  At WSO2, we expect machine learning to be an integral part of what makes that job sexy. And useful.

Done right, machine learning can easily replace a small army of statisticians – especially when it comes to enterprise big data. The world isn’t (yet) at the level where we can recreate a Minority Report for businesses, but there’s a lot of work that’s gone into this space. If you don’t know the field, you should: we’d recommend this short and beautifully designed Visual Introduction to Machine Learning, courtesy of R2D3.

If you know the field, you can skip the above and go directly to what Nirmal Fernando has to say. Nirmal leads the team working on the project, and his blogpost on ML gives a quick, concise breakdown of what WSO2 Machine Learner is capable of. Part of the challenge of predictive analytics is presenting the data intelligently, something we’ve paid careful attention to with WSO2 ML.

After that, you should probably check out what Srinath Perera, our Vice President of Research, has to say: he’s written a post called “WSO2 Machine Learner: Why Would You Care?” on his personal blog. This talks about how ML is built, and the uses you can put it to.

As you can see, ML completes our analytics offerings – we now offer batch, real-time, interactive and predictive analytics, and all of these components tie into each other beautifully for creating end-to-end analytics solutions for your business.

Of course, we have plans for making Machine Learner more powerful: we plan to add neural networks, algorithmic recommendation, anomaly detection, and more – as the product evolves, we’ll keep pushing out updates.

You can jump in and check this all out for yourself, of course! Machine Learner is free and open source. The Getting Started guide in our documentation will tell you everything you need to know to get a Machine Learner instance off the ground. Supun Setunga’s cooked up a detailed post on how you can set up that very first predictive model.

Download Machine Learner here, and you’re good to go. Set up your models right and you can peek into the future without actually having to go at 88 miles an hour in an old DeLorean. What’s not to like?

PS: If you’d like to call for support, or suggest features, don’t hesitate to contact us. We welcome bug reports, too!

Mastering the Art and Science of Capacity Planning

Imagine an e-commerce provider running out of memory on Black Friday or a search engine provider paying 20 times the cost of their optimal server capacity. Both these scenarios are great examples of what makes a solutions architect cringe. That’s why the ability to forecast the capacity of a system is an important activity in enterprise system design and solution architecture.

Capacity planning is an art as much as it is a science. Along with certain parameters, it also involves experience, knowledge of the domain itself and insight into the system. In some instances, it goes as far as analyzing the psychology of the system’s expected users and their usage patterns.

Mifan Careem from the WSO2 Solutions Architecture team recently wrote a white paper that looks at factors affecting the capacity of a system and how you can calculate your system’s capacity using these factors.

Here are some insights from this white paper.

There are multiple methodologies for carrying out capacity planning. A few parameters that will help include:

  • Transactions per second: number of actions per unit time
  • Work done per transaction: level of operations a transaction triggers
  • Think time: delay between user requests
  • Active users: users who use the system at a given time
  • Concurrent users: a subset of active users that perform actions at the same time
  • Message size: size of the message passed across the ‘wire’
  • Latency: additional time spent due to the introduction of a system
  • Other non-functional QoS requirements such as guaranteed message delivery, transmission of secure messages, throttling and uptime

After taking the above parameters into consideration you’ll also need to decide what the forecasting period should be, either focusing on only year 1 or whether your requirements will double in year 2. and if so would there be a significant downtime at the end of year one to accommodate this?

Mifan points out that the design of the application or software plays a big role in capacity planning. For each operation factors such as database connections, the number of objects stored in memory, and the amount of processing that takes place determine the amount of memory and processing capacity required. You need to keep these numbers low or share resources effectively in order to create a well-designed and efficient system with lower capacity requirements.

There are some things you’ll need to consider when designing your architecture:

  • Profiling and load testing your application
  • Caching to improve your performance and latency
  • Having buffer capacity when allocating server specifications
  • Server profiling via monitoring and profiling tools

You will also need to consider the type of hardware that will be used. This makes a difference as well, Mifan explains. The ideal way to calculate capacity across different forms of hardware is to have benchmarks on these distinct environments.

Here are a few more things Mifan notes that you need to keep in mind for a well-designed architecture:

  • Scalability: the ability to handle requests in proportion to available hardware resources
  • High availability: a system that is continuously operational for a long period of time
  • Disaster recovery: the replication of the primary site onto a geographically separate site
  • Backup and recovery: the replication of system state and system data onto a backup medium
  • Cloud: allows servers to be deployed in different geographically separated locations providing accessible means of achieving full-scale, high availability

The above-mentioned factors can be used for your capacity planning. The importance of these factors varies based on your environment type. It’s also important to have an accurate business architecture that can be converted to a high-level solution architecture. Based on that your team can start gathering capacity data in order to create the most accurate capacity plan. In addition to forecasting capacity, it’s important to test the environment to identify its peak capacity.

In part two of this white paper, Mifan will show how you can apply these concepts to determine the capacity of an actual use case using the WSO2 middleware platform.

You can download this white paper; Capacity Planning for Application Design – Part 1 and other whitepapers by visiting http://wso2.com/whitepapers/

New Products and More Unveiled at WSO2Con US 2015 – On Right Now

Lights, music, dance and of course technology magic took center stage as we kicked off WSO2Con US 2015 today.

In his keynote, Sanjiva Weerawarana, CEO, Founder and Chief Architect  of WSO2 spoke about how WSO2 helps you connect your world. Touching on the lessons learnt over the last ten years, he reviewed how new and upcoming products designed to work together within WSO2’s holistic platform are harnessing the power of analytics, Internet of Things (IoT), mobile, cloud, containers, microservices, and business processes, among others.

Here’s a summary of the new products announced…

Analytics Everywhere WSO2 Data Analytics Server (WSO2 DAS) 3.0 addresses the demand for enhancing customer and partner interactions, identifying new business models, anticipating and preventing security breaches and technology failures, and improving operations, all through big data analytics. It combines into one integrated, high-performance platform real-time and batch analysis of data with predictive analytics via machine learning to support the multiple demands of mobile, Web, cloud, and IoT-based apps. Significantly, WSO2 DAS also is being incorporated into the next generation of WSO2’s products, including API management, IoT and mobility management, business process management (BPM), and governance, among others.

Mobility and the Internet of Things –  WSO2 DAS 3.0 and the WSO2 integration platform, including WSO2 Message Broker 3.0, WSO2 Enterprise Service Bus (WSO2 ESB) 4.9, WSO2 Business Process Server (WSO2 BPS 3.5), and WSO2 Data Services Server (WSO2 DSS) 3.5. has been optimized for IoT. There are also two products to specifically address IoT and mobility demands, which are scheduled to launch in late Q4 2015/early Q1 2016.

  • WSO2 IoT Server is a comprehensive server for implementing an IoT architecture, including the management of IoT devices, APIs and applications.
  • WSO2 Enterprise Mobility Manager is being re-architected to use WSO2 CDMF and incorporates mobile device plugins along with functionality for mobile application and API management derived from WSO2 App Manager 2.0 and WSO2 API Manager 1.9, respectively, as well as analytics from WSO2 DAS 3.0.

Containers and the Cloud – WSO2 is migrating its cloud platforms from the Apache Stratos platform-as-a-service (PaaS) framework to a Kubernetes framework to achieve greater agility and scalability, as well as better resource utilization.

The Move to Leaner IT and Microservices –  IT organizations require software that can support today’s demands for high scalability while also remaining lean enough to run in highly distributed and diverse environments. WSO2 is addressing these architectural shifts with three product developments that are scheduled to launch in Q4 2015.

  • WSO2 Carbon Version 5.0 features a composable server architecture with a leaner footprint, and it removes tight coupling with legacy libraries, such as Axis2 and XML.
  • WSO2 Gateway is a new ultra high performance, lightweight and configuration-driven message gateway based on standard gateway patterns.
  • WSO2 Microservices Server is a fast, lightweight runtime engine which provides a simple way to develop, deploy and monitor microservices.

The “Consumerization” of IT – The online consumer experience is raising the bar on users’ expectations for enterprise systems. WSO2 is helping  IT organizations address this demand by providing the ability to provision and manage enterprise assets—such as APIs, applications, services, among others—through socially enabled consumer store-like capabilities. Among new and forthcoming products:

  • WSO2 Enterprise Store 2.0 is the first complete platform for managing and provisioning any type of asset—applications, APIs, gadgets, e-books and other resources—across the entire enterprise asset life cycle.
  • WSO2 Governance Registry 5.0 provides comprehensive, out-of-the-box support for a range of governance and life-cycle management scenarios.
  • WSO2 Dashboard Server 2.0, the new release of the previously named WSO2 User Engagement Server is scheduled to launch in Q1 2016. It will further enhance the ability of users to build dashboards, microsites and gadgets using internal or third-party data resources, as well as share them among authenticated or anonymous users across the enterprise.
  • WSO2 Process Center 1.0, scheduled to launch in Q2 2016, is powered by WSO2 Enterprise Store functionality, and makes it easy for anyone in the enterprise to document and discover processes—from simple Word documents to sophisticated Business Process Modeling Notation (BPMN) or Business Process Execution Language (BPEL) flows.