[Article] Introducing WSO2 Identity Server Authentication Analytics

  • By Hasintha Indrajee
  • 26 Sep, 2016


WSO2 Identity Server (WSO2 IS) is an open source identity and entitlement management server. It supports a wide array of authentication protocols, such as SAML 2.0 Web SSO, OpenID, OAuth 2.0/1.0a, OpenID Connect, and WS-Federation Passive. It has a wide range of interesting features that make it the industry’s first enterprise identity bus (EIB).

WSO2 IS 5.2.0 comes with a host of new features of which analytics is a key highlight. In the first version of identity server analytics, we include authentication analytics, i.e. login and session data analytics. This article will provide an introduction to WSO2 IS analytics and explain the benefits and uses of this feature.

Quick start

Before going into a detailed description of how WSO2 Identity Analytics works, it’s better to have some hands on experience. In order to play around with IS analytics, you need to download WSO2 Identity Server 5.2.0 and WSO2 Analytics IS 5.2.0 distributions from here1.

In order to enable data publishing from the identity server side, you need to enable required data publishers. The following two entries can be seen in the Identity.xml file that resides in IS_HOME/repository/conf/identity/

<EventListener type="org.wso2.carbon.identity.core.handler.AbstractIdentityMessageHandler"                       name="org.wso2.carbon.identity.data.publisher.application.authentication.impl.DASLoginDataPublisherImpl"
          orderId="10" enable="true"/>
<EventListener type="org.wso2.carbon.identity.core.handler.AbstractIdentityMessageHandler"
                       orderId="11" enable="true"/>

These two publishers are used to publish two different types of events from the Identity Server to the Analytics Server. As the name suggests, DASLoginDataPublisherImpl is used to publish authentication data to the analytics server. DASSessionDataPublisherImpl is used to publish session related data.

If you observe these two configurations in an out-of-the-box pack, the enable attribute will be false. You need to make it true as shown in the above configuration in order to get data published.

Start WSO2 Identity Server and the Analytics IS server. For the Analytics server, you don’t need to add any port offset. The Analytics server comes with a port offset out of the box. Required configurations are done in the Identity Server assuming the user runs on default port offsets that come out of the box. If you prefer to change default configurations you can follow this2.

As a prerequisite, set up a travelocity sample by following instructions given here3. This is nothing more than deploying a web app and doing a few configurations on the WSO2 Identity Server side.

Once all of the above steps are done, login to the travelocity application. You can try a few scenarios like authentication failures, successes, logins, and logouts. Once you play around with this app a few times go to the analytics server https://localhost:9444/carbon/admin/login.jsp. Once you login to the server you can see Analytics Dashboard on the left side of the admin console. Click on it and go to the analytics dashboard (this may ask you to allow popups from the browser).

Once you login to the analytics dashboard you can click on View and view events that are presented in a descriptive way. The login attempts page will look the image depicted in Figure 1.

Figure 1: Login attempts view

In the following sections we will be describing what each section looks like and how we use data to show each section.

Understanding authentication analytics

Before going into authentication analytics, we need to have a basic understanding of how an authentication flow works in WSO2 Identity Server.
An authentication flow may consist of multiple authentication steps. When an authentication request comes to WSO2 Identity Server through an inbound authentication protocol, the IS will go through configured steps and authenticate the user. Once all steps are successfully completed the IS will send out an authentication response back to the relying party based on the incoming protocol. All these are handled by Authentication Framework in WSO2 Identity Server.

Figure 2: Overall flow of IS Analytics

Publishing authentication related events from WSO2 Identity Server

Authentication events are published from the identity server’s authentication framework that’s responsible for handling authentication. There are five types of events we currently publish from authentication framework.

  1. Authentication step events - WSO2 Identity Server works as an identity bus. It can have multiple steps for a single authentication flow. Multiple authenticators can be involved in a single authentication sequence, e.g. you may have multi-factor authentication; in such cases, you may need to authenticate using basic authentication and SMSOTP as well (in two steps), or else you may need to log in with Google as well (this is a federated step where IS delegates authentication to an external IDP). For each of these steps, the authentication framework fires an event saying authentication status and some other important metadata of the authentication step.
  2. Overall authentication event - Once all the steps in an authentication sequence have been completed successfully, the identity server will successfully authenticate the user to the IS. Here, there will be an event sent with all other required information about authentication flow.
  3. Session create event - To notify a session create event. A session creation happens at the first login of a user.
  4. Session update event - To notify session update event, i.e. updating an existing session.
  5. Session delete event - To notify session delete event, i.e. signout from the existing login, deleting an existing active session.

These events are published to the IS Analytics server and then processed based on different requirements. All processed data will be shown in the Analytics dashboard in a user-friendly manner.

Let’s talk about data interpretation on WSO2 IS analytics dashboard.

There are two main sections in WSO2 IS analytics dashboard, namely Authentication and Session sections. The Authentication section is divided into three main sections named overall authentication, local authentication and federated Authentication. Let’s get some insights into each of these sections and talk about and how we define data that falls under each of these.

Overall authentication

Figure 3: Overall authentication view

The simplest definition for this section is that if an authentication request comes to the identity server and if it serves successfully with success, then it will be counted as an overall authentication event. For an overall authentication event, the steps involved and whether the user had a previous session or not aren't relevant.

  • Success count - overall authentications that were successful - if an authentication request comes to IS and the user authenticates successfully (completing all steps successfully), It will be considered as a login success event, i.e. it will be counted under overall authentication success.
  • If a user has a valid session (a cookie that’s obtained through a previous login) and tries to log in (passive login) it will still be counted as an overall authentication success. According to the definition, an authentication request comes to the IS and authentication successes. Therefore, an overall authentication event will be triggered.
  • In order to consider an event as a success, all the steps should be successfully completed without errors.
  • If an error occurs in a step, it will be considered as an overall authentication failure, i.e. all step failures will be considered as overall authentication failures as well.
  • First login is an attribute we use to keep track og the session creations for a user, e.g. there can be multiple authentication requests coming to the identity server, but a user may own a valid session as well. In such cases all requests will be served as authentication success events, but will not fall under first login.

Local authentication

Figure 4: Local authentication view

  • If an authentication sequence contains a local step, it will be considered a local authentication attempt.
  • Only a single local authentication success will be published irrespective of the number of local steps configured, i.e. if you have two local authentication steps and both are successful, it will only be calculated as a single local authentication effort. The rationale behind this is that even if you authenticate multiple times it’s the same identity provider you’re authenticated against (i.e. local identity provider). A key fact is that even though we fire two local success events, we will only calculate a single local authentication success.
  • If a user has a valid session from IS (a cookie that’s obtained through a previous login) and tries to log in (passive login) again then the behavior will depend on the following scenarios:
    • If the user is already authenticated from a local authenticator in his previous authentication effort (should be an overall success effort) - even though there is a local authentication step in the new effort, it will not be calculated as a local authentication success because user has logged in to local IDP in his previous success effort he took to get the valid session. However, it will be counted as an overall authentication success as described in the “overall authentication” section.
    • If the user doesn't have a local authenticator involved in his previous authentication success scenario - if the user has a local step in the new login effort, if it succeeds, it will be counted as a login success event under Local Authentication. Simply user authenticates through this local step and it will succeed. Following this, we will publish local steps and will be calculated under local successes.
  • All local step authentication failures will be counted under authentication failures in Local Authentication.
    • If the user fails to authenticate to a local step (basic authentication or FIDO, etc), those failures will be counted as authentication failures under Local Authentication.

Federated authentication

Figure 5: Federated authentication view

All federated steps fall under this section. If you log into a federated authenticator successfully it will be considered as a successful login effort under Federated Authentication.

All failures that happen in the federated steps will be counted as federated authentication failures under the Federated Authentication section. Most federated authenticators do not get an event back when an authentication failure happens from the federated IDP side. Hence, we will not be publishing authentication failures for such scenarios (since we do not get an authentication failure from a delegated IDP), e.g. if you have configured Facebook as an authenticator and you try to login through it, if an authentication failure happens Facebook will not send a failure response back to the identity server. Instead, Facebook will keep retrying authentication by continuously promoting the login page. In such scenarios, we do not send out an authentication step failure event for the federated authenticator.

Practical examples

The following is a concrete example that will elaborate some important aspects of the behavior of identity server analytics.

Example 1 - There are two applications as travelocity.com and avis.com. Travelocity.com has a local authenticator (basic authenticator) and a federated authenticator (Facebook) configured for its authentication sequence. Avis.com has a local authenticator and a federated authenticator (Google) configured within its sequence. Consider the following scenario.

A user logs in to travelocity.com after successfully entering a username and password through the local authenticator and then logging in through Facebook. Now this user has a valid session that’s obtained through IS.

The user goes to avis.com from the same browser. Now he/she has a valid session that can be used to authenticate to the identity server, but IS will prompt Google authentication since the user has not previously authenticated to Google. This is the behavior of the identity server. An important point to note here is that it will not prompt you to login to the local step because you have a valid session that you’ve previously obtained by logging in through a local step.

If we consider the above two incidents, you will see events in the analytics dashboard as follows:

  1. Logging in from travelocity.com - There will be an overall authentication success event, a local step success event, and a federated step success event for login through Facebook. This scenario will fall under the first login, i.e. the session creation happens at this login.
  2. Logging in from avis.com following travelocity.com - There will be an overall authentication success event, but there will not be a local authentication success event (ideally the user didn't have to enter username and password for the local step). This scenario will not fall under the first login, i.e. the session creation happens only at the time of login to travelocity.com and not at the time of avis.com. If you haven’t noticed the user had to authenticate to Google in this scenario. Therefore, there will be a federated step event for this authentication.

Example 2 - Suppose we have two applications as travelocity.com and avis.com. Travelocity.com is configured to have basic authentication and one federated step. Let’s say the federated step is Facebook.

Avis.com is configured to have three authentication steps. Basic authentication, and two federated steps. Let’s say Facebook and Google. For both of these scenarios the same Facebook IDP is used (i.e. we have only configured a single federated IDP for Facebook).

Suppose the user tries to log in to travelocity.com, he/she will be prompted basic authentication where the user needs to enter a username and password. Once that step has been passed successfully the second step will appear, i.e. Facebook authentication. Then the user has to successfully authenticate to Facebook and get back. Once these two steps are completed, the user is successfully authenticated to the identity server.

Now the user tries to authenticate to avis.com from the same browser where he has a valid cookie from the identity server. Once the request reaches the identity server, it will not prompt the user to login to Local IDP and Facebook since he/she already has logged in from those two IDPs previously. The user only has to login to Google and obtain access to avis.com.

If we have a look at events that are generated in this sequence of incidents, it will be as follows:

  1. Login to travelocity.com - This will trigger a two-step authentication event; the first one from the local authenticator, and the second one that’s triggered after a successful Facebook authentication. There will be an overall authentication success event as well that says this is a first login as well. There will also be a session creation event.
  2. Login to avis.com - This will only generate a single step event and an overall event. Apart from these two authentication events, a session update event will also be generated.

Understanding streams

Events are transferred to IS Analytics Server from the identity server as streams. Definitions of the event streams are stored in the filesystem as deployable artifacts in the <IS_ANALYTICS_HOME>/repository/deployment/server/eventstreams/ directory as .json files. This definition is used as an event template to transfer data between the identity server and Analytics IS. In Identity Server 5.2.0 we have two types of event streams that define two main event structures; these include authentication event stream (org.wso2.is.analytics.stream.OverallAuthentication) and session data stream (org.wso2.is.analytics.stream.OverallSession).

Authentication data stream is used to transfer two types of events. They are authentication step events and overall authentication events. The following is a description of those events and important attributes of those events.

Authentication step event

This event is used to transfer information about authentication steps. All authentication step-related event information is transferred through this stream. On the other hand, all step-related counts are calculated from this event except local authentication success. The following is the rationale for not using this event to calculate Local Authentication Success.

According to our definition, we consider authenticating to the local IDP as a single event of interest for a single authentication flow, i.e. no matter how many times you authenticate to the local IDP through different local authenticators (basic authenticator, FIDO, etc.), we only define it as a single authentication success event that falls under local authentication. The rationale is that no matter how many times you authenticate to the local IDP in a single flow, it will still be the same user that resides in the local store.

Following are some important attributes in a step event:

  • Event Type - This should be “step” in step events.
  • Authentication Success - Will always be false for step events since this attribute talks about overall authentication success.
  • Identity Provider - The identity provider associated with the step, i.e. LOCAL, Facebook or any other IDP.
  • Is First Login - Whether the login caused to create a new session or login was associated with an existing valid session.
  • Roles Comma Separated - For this field we are only sending out Non-Internal roles and non-Application roles.
  • Identity Provider Type - Type of identity provider, i.e. whether the identity provider associated with the step is LOCAL or Federated.

Overall authentication event

This event is used to communicate the overall authentication success of a sequence. At the same time, this will be used to calculate local authentication success events as well.

The following are some important attributes in an overall event:

  • Event Type - This should be “overall” in overall events
  • Authentication Success - Will always be true for overall events as per current implementation. We only send out overall success events (for failures we are only sending out step events)
  • Roles Comma Separated - For this field, we are only sending out non-internal roles and non-application roles

When it comes to an overall event an important attribute in a stream is LocalUserName. In the first part of this article, we stated that overall event is used to convey data about local authentication success step.

Session data analytics

Figure 6: Session analytics view

For all session creations, update and termination of identity server will trigger events to Analytics IS as per current implementation. Based on these, calculations are done from the Analytics IS side and session information will be maintained.

There are a few key attributes in session stream:

  • Session starting time stamp - The time the session started, i.e. a session will be created at the first login events
  • Renew timestamp - Session renewed timestamp
  • Termination time stamp - The time the session is supposed to terminate
  • Action - Action will be “1” from a session creation event and “0” for session termination event. “2” indicates a session update

The graph in Figure 6 indicates how Active, New and Terminated sessions vary with time (given periods).

Active Session - A session that’s active in the given time. This session does not necessarily require to be created within the observing period of time. Instead, this may be an old session that remains active.

New Session - A session that’s newly created. The creation of this session needs to be within the observing period of time.


WSO2 Identity Server Analytics is a very powerful tool to analyze and interpret data in a comprehensive manner. The 5.2.0 release comes with Authentication analytics out of the box. Data publishing is done from the identity server side and the published data is processed, stored, and represented on the Analytics server-side using state-of-the-art techniques. Data publishing from the identity server side is done in a comprehensive format and can be extended to publish or leverage any other use cases as well. At the same time, DAS publishers make it possible to publish through multiple methods (transport). These capabilities make WSO2 identity server analytics a powerful tool that can be used to extend and to be utilized and customized as per the user’s requirements.