The Making of TomatoTube, Featuring The WSO2 Mashup Server

  • By Tyrell Perera
  • 17 Oct, 2007


Making a TomatoTube movieThe WSO2 Mashup Server[1] lets its users create Mashups using different data sources found around the Web. Your newly created Mashup will also be exposed as a Web Service, so that it in turn becomes consumable by others. In this tutorial, we will look at how to create a useful Mashup by combining data exposed by two Web sites. You will also learn about some of the APIs provided by the WSO2 Mashup Server to harvest data, extract information useful to your Mashup and expose them to a third party consumer as Web Service Operations or Feeds.

Since we write Mashups using JavaScript in the WSO2 Mashup Server, it is assumed that you have at least an intermediate level of exposure to the language. A basic knowledge of Web Services[9] and how they work will also be helpful along with an idea of how RSS[2] works.

Applies To

 WSO2 Mashup Server   v0.2+ 


Table of Contents


When it comes to movies, the human population is divided into two categories; the movie-goer and the movie-buff. Although a detailed discussion of these two categories is not within the scope of this tutorial, it should be noted that no matter what category one belongs to, they will eventually like to get their moneys worth.

Sites such as Rotten Tomatoes[3], provide the necessary information to achieve this very goal. They are community driven and contain movie reviews done by average movie-goers and movie buffs. They also provide a set of RSS feeds[4], for which a user can subscribe to receive the latest movie ratings.

There are also community driven sites such as YouTube[5], where one can find many trailers for a movie. YouTube exposes its data through their GData API[6]. It is fairly easy to generate a query, to get a list of trailers, for a given movie, using this service in RSS format.

By now you should have a hint of what we are trying to do in this tutorial (if the word TomatoTube wasn't a good clue). We are going to use the data exposed by these two sites and create a Mashup, which will give us the Top 10 rated movies at Rotten Tomatoes along with their trailers from YouTube. In other words, TomatoTube will do your movie research for you, with just a click of a button.

Designing Our Mashup

The Requirements a.k.a What We Need ...

Let's take a look at the work flows and tasks required to create a Mashup of this nature.

  1. Retrieve the RSS feed from Rotten Tomatoes, which gives us a list of the Top 10 rated movies, a description and a link to their reviews in the site
  2. Extract the movie name from this feed, then construct the query to retrieve the relevant trailer from YouTube
  3. Create a new feed, which will contain the information from Rotten Tomatoes and an embedded YouTube video of the movie trailer
  4. Host the new feed so that TomatoTube users have the option of subscribing to it with their RSS aggregators
  5. Cache the newly created feed locally, since the above work flows are resource intensive. Serve requests using the cache in order to save resources
  6. Periodically do work flows 1 to 4 and refresh our local cache to make sure it is synchronized with Rotten Tomatoes and YouTube 

Implementation a.k.a What We Have ...

In the heart of the WSO2 Mashup Server, we can find a set of JavaScript objects known as Host Objects. These Host Objects, encapsulate useful functionality required to create Mashups. The Host Objects are native JavaScript Objects. There is also a set of Annotations, which allow us to expose a conventional JavaScript file and its functions as a Web Service and its Operations.

The single paragraph above pretty much sums up all you need to know in order to write a Mashup using the WSO2 Mashup Server. So let's start writing some code, shall we?

Let's Create A Mashup

Creating a Mashup Skeleton from the Admin UI

Assuming that you have downloaded and have a WSO2 Mashup Server instance running in your machine according to the instructions in the packaged documentation, it's a pretty easy task to create a functional, skeleton Mashup from the Admin UI. In a vanilla installation, you can access the Admin UI by typing https://localhost:7443/ (Note the https in this URL). Sign in using the credentials you gave at initial startup.

Once in the Admin UI, you will find the Create a New Service link under the Management Tasks list. Let's give our Mashup the name 'TomatoTube'. You will be presented with a new page where the skeleton code generated for the new Mashup is displayed. There are two tabs, which contains both the Mashup and Custom UI skeleton code. Note that the Mashup is not physically created yet and in order to do so you need to click the  'Apply Changes' buttons of both tabs. This will save your changes and will keep you in the current page, whereas 'Save Changes' will save and take you back to the previous page where you came from.


Once you save the changes, your new mashup will be created and deployed in the server. You can verify this by going to the home page and checking the My Mashups list. All your mashups will be listed here after a few seconds from the time they are created.

Note: The TomatoTube Mashup comes bundled in as a Sample from version 0.2 of the WSO2 Mashup Server. You might have to use a different name for your Mashup to successfully complete this step. The JavaScript files for the Mashups are stored in the 'scripts' sub directory found in the installation directory. You may un-deploy the original TomatoTube by deleting the tomatoTube.js file and the tomatoTube.resources directory found inside 'scripts'. 

When the Mashup gets listed, you can click on its name to view additional details about the skeleton generated. Most importantly, it will show you a Try Service link among the links displayed at the bottom (Fig 1.1).


The TomatoTube skeleton information

[Fig 1.1] The newly created skeleton's information

The Try Service link is important to us at this stage because it allows us to test our Web Service by invoking its operations. This comes in handy when we have some functional code and want to verify it works as planned. Click the Try service link and you will see a page as illustrated in Fig 1.2 with buttons to invoke each operation.

The Try service page

[Fig 1.2] The Try service page of our skeleton Mashup

Now let's take a look at the skeleton JavaScript code generated for us by the Mashup Server. There are two ways to view and work with the code. The first is to open it in the Admin UI itself. This will give you a minimalist editor where you can work and save the modifications. The second option is to open the TomatoTube.js file found in the scripts sub directory in your favorite JavaScript editor. I prefer the second option better since it gives me the maximum flexibility. It is also the safer option during the novice user stages, where we are likely to break the code often.

The auto generated code is listed below. There is a considerable amount of internal documentation in the form of comments, which will help you to understand what's going on. As we proceed further, you will better understand the value and use of the Annotations seen in this skeleton code.

this.serviceName = "TomatoTube";
this.documentation = "TODO: Add service level documentation here";

toString.documentation = "TODO: Add operation level documentation here";
toString.inputTypes = { /* TODO: Add input types of this operation */};
toString.outputType = "String"; /* TODO: Add output type here */
function toString()
//TODO: Add function code here
return "Hi, my name is TomatoTube";


Retrieving RSS feeds from Rotten Tomatoes and YouTube

The first thing we have to do is reading the RSS feed from Rotten Tomatoes. The WSO2 Mashup Server comes with a set of Host Objects, which allows us to do just that. There are actually three Host Objects, which makes this possible. They are Feed, Entry and FeedReader. In summary a Feed is a collection of Entry objects, while the FeedReader is capable of reading a published Feed from a given URL.

The following code shows how to read a published Feed using the FeedReader host Object.

//The URL of the Rotten Tomatoes RSS feed
var rssUrl = "";

//Instantiating a FeedReader Host Object and reading the feed
var feedReader = new FeedReader();
var rottenTomatoesFeed = feedReader.get(rssUrl);

//Obtaining all the entries of the Feed as an Array
var feedEntries = rottenTomatoesFeed.getEntries();

Once we have an Array of entries, we can go through each and every entry to obtain the name of the movie. Since the movie name we get as the title includes a percentage rating attached to it (ex: 26% Resident Evil: Extinction), we have to do a bit of cleanup to get the actual movie name. Hence the splitting of entry tittle using the '%' sign.

for (var x = 0; x < 10; x++) {
//Extracting the movie name from the title
var title = feedEntries[x].title;
var tempArray = title.split("%");

var movieName = "";
if (tempArray.length > 1) {
movieName = tempArray[1];
} else {
movieName = tempArray[0];


Next, we have to construct a YouTube GData query string to get a trailer for our movie. We are going to write a function named findTrailer for this purpose. We will also expose this function as a Web Service Operation, so that other Mashups can use it as well when we finally host TomatoTube.

Our findTrailer function will accept a movie name as a String parameter and will return a String containing the link of a trailer found in YouTube. The following code is the full function code. Please take a look at the annotations used before the function declaration. These will be used by the WSO2 Mashup Server when exposing findTrailer as a Web Service Operation.

We are creating the search query according to the YouTube GData documentation[7]. Our query will give us a collection of 5 trailers ordered by their relevance to the movie name we provide, in RSS 2.0 format. We then use the FeedReader Host Object to read the result and return the first link returned.

findTrailer.documentation = "Uses the YouTube GData API to search for the trailer of a given movie.";
findTrailer.inputTypes = { "moviename" : "string" };
findTrailer.outputType = "String";
function findTrailer(moviename)
//Formatting the movie name to escape special characters
moviename = escape(moviename);

//Creating the query string we are using RSS 2.0 to retrieve a list of trailers.
var query = "" + moviename + "%20trailer&start-index=1&max-results=5&orderby=relevance&alt=rss";
var trailerReader = new FeedReader();

try {
var youTubeFeed = trailerReader.get(query);
var feedEntries = youTubeFeed.getEntries();

if (feedEntries.length > 0) {
//Returning the first link found
return new String(feedEntries[0].link[0]);
} catch(err)
logMessage(err, "error");
return "Not Found...";

Remember the Try Service Link we mentioned earlier? It comes handy at this point to make sure our function is exposed properly to third parties as well as testing it without further delay. Figure 1.3 illustrates a successfully tested findTrailer operation.

Note: The WSO2 Mashup Server has a Hot Update mechanism, which will un-deploy and re-deploy your Mashup as soon as you save the code changes.

Testing the findTrailer operation

[Fig 1.3] Testing the findTrailer operation

Extracting the Information we want and Creating our own Feeds

Now that we have harvested the data we need, it's time to format it the way we want. In the case of TomatoTube, we are going to create an embeddable video from the YouTube link we received and inject it into the original description we received from Rotten Tomatoes.

We will use a utility function named convertLinkToEmbed to convert a given YouTube video link to an embeddable video. Since we do not want to expose this function to the outside world when our Mashup is hosted, we will also use the convertLinkToEmbed.visible = false annotation to tell the WSO2 Mashup Server about our intentions.

In summary, what we are doing here is splitting the YouTube link using '=' and obtaining the video code. We then use this video code to generate and return the HTML required to create the embedded video.

convertLinkToEmbed.visible = false;
function convertLinkToEmbed(link)
var tempArray = link.split("=");
var videoCode = tempArray[tempArray.length - 1];

return '<object width="650" height="535"><param name="movie" ' +
'value="' + videoCode +
'"></param><param name="wmode" ' +
'value="transparent"></param><embed src="' + videoCode +
'" type="application/x-shockwave-flash" wmode="transparent" width="650" height="535"></embed></object>';

Once we get the HTML for video embedding, we can start creating our own Feeds. We will create two feeds to implement TomatoTube. The reason for having two feeds instead of one is, that one feed will be created with XML/HTTP based communication in mind, while the other will be created as a regular RSS 2.0 feed. The difference between these two feeds is the former having its Entry Descriptions wrapped with CDATA[8] tags while the latter does not.

The following code fragment shows the initialization of the two Feed objects. Both objects seem identical at this stage.

//Creating the live feed. This feed will be cached in file to serve requests
var mashupFeed = new Feed();
mashupFeed.feedType = "rss_2.0";
mashupFeed.title = "TomatoTube - A Mashup of Rotten Tomatoes and YouTube";
mashupFeed.description = "The Top 20 tomato rated movies in theaters."; = "hosted-" + mode + "-mashup-feed.xml";

//Cloning the feed to be hosted as an RSS feed @ TomatoTube. This feed will have descriptions without CDATA wrapping
var hostedFeed = new Feed();
hostedFeed.feedType = "rss_2.0";
hostedFeed.title = "TomatoTube - A Mashup of Rotten Tomatoes and YouTube";
hostedFeed.description = "The Top 20 tomato rated movies in theaters."; = "hosted-" + mode + "-mashup-feed.xml";

When we have the Feed objects in place, we can loop through the Feed Entries retrieved from Rotten Tomatoes, extract the movie names and create embeddable YouTube trailers as shown in the code fragment below.

//Extracting the movie name from the title
var title = feedEntries[x].title;
var tempArray = title.split("%");

var movieName = "";

if (tempArray.length > 1) {
movieName = format(tempArray[1]);
} else {
movieName = format(tempArray[0]);

if (!(movieName == "undefined")) {
//Getting the You Tube trailer
var trailerLink = findTrailer(movieName);

//Checking whether a trailer is found
if (!(trailerLink == "Not Found...")) {
//Converting the link to an Embeddable You Tube video
var embedLink = convertLinkToEmbed(trailerLink);

//Attaching the Embeddable Link to the entry description
var description = feedEntries[x].description
description = description + "<br><br>Here's a teaser from YouTube;<br><br>" + embedLink;

//Adding description without CDATA to the hosted feed
var hostedFeedDescription = description;

//Wrapping with CDATA tags for transport
description = "<![CDATA[" + description + "]]>";

//Creating the new entry for the live feed
var newEntry = new Entry();
newEntry.title = feedEntries[x].title; = new String(feedEntries[x].link[0]);
newEntry.description = description;


//Cloning the new entry and adding non CDATA wrapped description to it. Adding this to the hosted feed.
var newHostedEntry = new Entry();
newHostedEntry.title = feedEntries[x].title; = new String(feedEntries[x].link[0]);
newHostedEntry.description = hostedFeedDescription;



//Writing the newly created live feed to a temporary file
mashupFeed.writeTo("temp-" + mode + "-mashup-feed.xml");

//Writing the newly created hosted feed to a temporary file
hostedFeed.writeTo("temp-hosted-" + mode + "-mashup-feed.xml");


Finally, as shown above, we will write the newly created feeds to disk. By caching the feeds locally and using the cached feeds to serve requests, we can save both server and bandwidth resources. As a precaution, we will first write to temporary files and after we are assured that everything went smoothly we will move both feeds to their live locations.

The WSO2 Mashup Server has a File Host Object, which allows the manipulation of files on disk. Using this you will be able to create, read, write, delete and move files. The default location for the files created is the .resources directory associated with your Mashup. In this case it is tomatoTube.resources found inside the scripts sub directory. Each and every Mashup created will have its associated resources directory, which can be used to store required resources as well as the Web UI for the Mashup.

The code fragment below moves the temporary Feed files to their live location.

//Deleting the live feed file if one exists
var currentLiveFeedFile = new File(mode + "-mashup-feed.xml");
if (currentLiveFeedFile.exists) {

//Moving the temp file to the live location
var tempFeedFile = new File("temp-" + mode + "-mashup-feed.xml");
tempFeedFile.move(mode + "-mashup-feed.xml");

//deleting the hosted feed file if it exists
var hostedFeedFile = new File("www/hosted-" + mode + "-mashup-feed.xml");
if (hostedFeedFile.exists) {

//Moving the temp file to the hosted location
var tempHostedFeedFile = new File("temp-hosted-" + mode + "-mashup-feed.xml");
tempHostedFeedFile.move("www/hosted-" + mode + "-mashup-feed.xml");



Refreshing the Local Cache Periodically

Up to this point, we have completed most of the tasks we originally planned. We now have a functional Mashup capable of retrieving RSS feeds from Rotten Tomatoes and YouTube, process them and create the mashed up feeds we need for TomatoeTube. We also have a functional caching mechanism to make sure we do not put unnecessary stress on the server. If you download the full source code for the Mashup, you will see this functionality encapsulated in a single, non exposed function named createTomatoeTubeFeed. This function accepts a single String parameter which indicates the 'mode' of Feed we want. Currently it can be either theater or dvd. Depending on the mode passed we choose different RSS Feed URLs from Rotten Tomatoes.

Now we have to think of this Mashup as a Long Running Service. Once initiated, it should be capable of maintaining the Feed cache by periodically performing the above tasks. We can give this capability to our Mashup with the use of two more Host Objects provided by the WSO2 Mashup Server. The System Host Object and the Session Host Object.

We will use the system.setInterval() static method provided to us by the WSO2 Mashup Server to schedule the periodic execution of the function createTomatoeTubeFeed. In its basic form you can call system.setInterval() by giving it some well formed JavaScript  to execute, along with the time interval (in milliseconds) between two consecutive executions. Optionally we can give a starting time for the initial execution and if required a time to stop (in JavaScript Date format).

The code listing below shows our non exposed scheduling function, which calls system.setInterval(). It schedules the execution of createTomatoeTubeFeed once every hour. it also uses the Session Host Object to store the state of the scheduler. The state is a boolean flag indicating whether createTomatoeTubeFeed function is scheduled for periodic execution.

One important thing to remember when using the Session Host Object is to set the Scope of the session at the beginning. The availability of data stored in a session depends on the scope declaration. For instance, we have to include this.scope = "application"; in the annotations for the TomatoTube service to make sure the state flag is available throughout the life of the service.

startPeriodicRefresh.visible = false;
function startPeriodicRefresh(mode, startNow)
//Scheduling the mashed up feed creation starting now and repeating every hour
logMessage("Starting periodic feed refreshing. Mode '" + mode + "'");

try {
var functionString = "createTomatoeTubeFeed('" + mode + "');";
var scheduler_id = "";

if (startNow) {
scheduler_id = system.setInterval(functionString, 1000 * 60 * 60);
} else {
//Setting start time to 60 minutes from now
var startTime = new Date();
startTime.setMinutes(startTime.getMinutes() + 60);
scheduler_id = system.setInterval(functionString, 1000 * 60 * 60, null, startTime);

//Flagging scheduler state in session
session.put("schedulerStarted_" + mode, new String("true"));

} catch(err) {
logMessage(err, "error");

Exposing an Operation to get the new TomatoTube feed

The final task on the server side is to code a Web Service Operation enabling the users of our Mashup to get the mashed up feed. Remember every Mashup created in the WSO2 Mashup Server is deployed and behaves as a Web Service. Just like we exposed the findTrailer operation, we will create another similar operation, which will return the XML/HTTP transport ready feed we created earlier.

Let's call this operation readTomatoTubeFeed. This operation will accept a String input, which is the mode of Feed required. It can be either theater or dvd. It will return the Feed as XML. By now you should be familiar with the annotations required to expose the operation. Once the code is written, you can use the Try Service link to test the operation as we did earlier with findTrailer.

readTomatoTubeFeed.documentation = "Obtains the top rated, movies in theaters and dvd from " +
"and embeds a You Tube trailer to the feed, creating a mashed up feed. " + "" +
"The current expected inputs are 'theater' and 'dvd'. " +
"The mashed up feed is written to a file in the workspace directory";
readTomatoTubeFeed.inputTypes = { "mode" : "string" };
readTomatoTubeFeed.outputType = "xml";
function readTomatoTubeFeed(mode) {

//Checking for supported input types
if (!((mode == "theater") | (mode == "dvd"))) {
return new XML("Invalid input. Currently supported modes are 'theater' and 'dvd'.");

//First check whether there is already a feed file in the live location
var liveFeedFile = new File(mode + "-mashup-feed.xml");

var mashedUpFeed;

if (liveFeedFile.exists) {
//Reading the file contents to a string and removing xml encoding info
mashedUpFeed = liveFeedFile.toString();
mashedUpFeed = mashedUpFeed.substring(mashedUpFeed.indexOf("?>") + 2);

//Check whether a scheduler exists for this feed mode
var schedulerState = session.get("schedulerStarted_" + mode);

logMessage("Current scheduler active status for " + mode + " is '" + schedulerState + "'");

if (!(schedulerState == "true")) {
startPeriodicRefresh(mode, true);

return new XML(mashedUpFeed);
} else {
//Creating the initial mashed up feed

//Reading the file contents to a string and removing xml encoding info
mashedUpFeed = liveFeedFile.toString();
mashedUpFeed = mashedUpFeed.substring(mashedUpFeed.indexOf("?>") + 2);

//Call the scheduler to create initial feed and periodic refresh hereafter
startPeriodicRefresh(mode, false);

return new XML(mashedUpFeed);

You can have a look at the complete source code to put all of the above code fragments in context, if you are feeling disoriented. In the next section, we will look at how to create the HTML necessary to create a client user interface to this service.

Creating a UI for our Mashup

When writing a client to any Mashup deployed in the WSO2 Mashup Server, two things are essential. First of all since all Mashups developed are also Web Services, we need a communication medium to talk with these Mashups. Secondly we need client stubs for a Mashup we need to communicate with. Since now we are writing ordinary client side JavaScript code, we don't have the luxury of the WSO2 Mashup Server Host Objects. But instead we have the WSRequest.js, which serves as the communication provider and the ?stubs facility which dynamically generates the JavaScript client stubs for any deployed Mashup. Therefore before going any further we need to import these two in our HTML page by including the following inside the <head> tag.

Please note that the location to create this HTML page is a sub directory titled www inside the tomatoTube.resources directory. The name of the file has to be either index.html or index.htm. Once the page is in place, the WSO2 Mashup Server will take over the hosting part.

<script type="text/javascript" src="/js/wso2/WSRequest.js"></script>
<script type="text/javascript" src="../TomatoTube?stub"></script>

Now we are ready to call the readTomatoTubeFeed operation of our Mashup and get a RSS feed to the client side. After invoking the operation and getting the Feed. it's just a matter of writing the DHTML to render the data received to the page. Three steps are involved in making a call to an operation.

  1. Set the end point address of the Mashup. In the case of TomatoTube this is "services/TomatoTube".
  2. Write a callback function for error handling and assign it as the error handler. In this instance the function name is handleError.
  3. Write the callback function, which will accept the data returned as the response to the call. in the following code the name of that function is fillData.
function getData() 

//Displaying the loading animation
var contentDiv = document.getElementById("content");
contentDiv.innerHTML = "<img src='../../images/loading.gif'/> Gathering reviews and trailers. Please wait ...";

var mode = document.getElementById("mode").value;

TomatoTube.setAddress(TomatoTube.endpoint, "services/TomatoTube");
TomatoTube.readTomatoTubeFeed.onError = handleError;
TomatoTube.readTomatoTubeFeed.callback = function (userdata) {

You can have a look at the complete source code to put the above code fragment in context, in case you are feeling disoriented. The full source for the HTML contains the DHTML required for rendering as well. The following is a screen shot of TomatoTube in action.

TomatoTube in action
[Fig 1.4] TomatoTube in action


In this tutorial you learned how to create a Mashup of RSS Feeds using the WSO2 Mashup Server. You also learned about some of the concepts related to creating Mashups with the WSO2 Mashup Server, while highlighting the functionality provided by;

  • Feed, Entry, FeedReader, System, Session Host Objects to Mashup authors,
  • WSRequest, Dynamic Stub Generation to Mashup UI authors and
  • The WSO2 Mashup Server Admin UI in creating a Mashup skeleton, exploring its meta-data and testing its service operations

You can always keep up to date about the latest version of the WSO2 Mashup Server by visiting its home page[1] and links there. Please refer the user guide packaged with the installation for more information on Host Objects and the Admin UI.











[10] - An online demo of the TomatoTube service

Source: The complete source code for the TomatoTube Mashup developed during the tutorial can be found here


Tyrell Perera, Senior Software Engineer, WSO2 Inc. tyrell at wso2 dot com

About Author

  • Tyrell Perera
  • Technical Lead and Product Manager
  • WSO2 Inc.