2011/11/21
21 Nov, 2011

Writing Google Gadgets – A Tutorial – Part 02

  • Samisa Abeysinghe
  • VP, Training - WSO2

Writing Google Gadgets – A Tutorial – Part 02

In the first part of this tutorial, we explored the basics gadgets, URL and HTML content types,
pulling information from the Web and displaying. In this second part of this tutorial, we will learn how to
process content fetched and present those in more appealing format.


Processing Fetched HTML with JavaScript

In the MarkMail gadget, we studied in part 01 of this tutorial, we displayed the content fetched with makeRequest, as it is.
But now, we need to process that to pick information that we want to display in the gadget.

The first step in processing received response data is to understand what is in the response, and the formation.

For our example, if you visit the MarkMail URL we used: https://markmail.org/browse/org.wso2.carbon-commits
and view the source of that page, you will notice that there is a div with the ID "browse" in there and it is this div, that has the main content of the page.

    <div id="browse">

Inside this <div> you will find a table with the monthly svn activity data. So from our gadget,
we would like to pic this <div>, and then the table with the data, and use that as the main content as opposed to the whole page.

<?xml version="1.0" encoding="UTF-8" ?>
<Module>
    <ModulePrefs title="WSO2 Open Source Development Data from MarkMail">
        <Require feature="dynamic-height" />
    </ModulePrefs>
    <Content type="html">
	  <![CDATA[
	  <a href="https://markmail.org/"><img src="https://markmail.org/images/logo_white.gif" alt="MarkMail"/></a>
	  <div id="contentDiv"></div>
	  <span id="hiddenSpan" style="display:none">Hidden</span>
	  <script type="text/javascript">
    
		  function getHtml() {    
			  var parameters = {};  
			  parameters[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.TEXT;  
			  var url = "https://markmail.org/browse/org.wso2.carbon-commits";  
			  gadgets.io.makeRequest(url, processResponse, parameters);
		  };
		
		  function processResponse(response) {  
			   document.getElementById('hiddenSpan').innerHTML = response.text;
			   
			   document.getElementById('contentDiv').innerHTML = "<table>" + document.getElementById('browse').getElementsByTagName('table')[0].innerHTML + "</table>";
			   gadgets.window.adjustHeight();
		  };
		
		  gadgets.util.registerOnLoadHandler(getHtml);

	 </script>
	  ]]>
    </Content>
</Module>

This gadget extends the previous sample gadget we discussed, in the dynamic height section, in part 01 of this tutorial.
The key difference here is that, instead of using the response text as it is, we use some logic to pick the table with the content, and display.

Since we get the response as text, in order to use some DOM logic to pick the table, we need to attach the text response data that came,
into an element on the current HTML document that the Web browser is aware of.

For this, we use the hidden <span> element declared on line 10. Note the 'style="display:none;"' hence, the span is not visible on the gadget.

Then on line 21, we assign the inner HTML of that hidden <span>
element to be the response text, from inside the process response callback. Once this assignment is done, we can treat the HTML inside the response
element as part of the main HTML document and use JavaScript logic to process it.

On line 23, we locate the <div> with ID "browse" and pick the table element:

document.getElementById('browse').getElementsByTagName('table')[0].innerHTML

We also wrap this content with a

tag, as the above logic will strip
tag:
"<table>" + document.getElementById('browse').getElementsByTagName('table')[0].innerHTML + "</table>"

And this processed content is assigned to be the inner HTML of "contentDiv".

To make sure we give due credit to MarkMail for the content, on line xx we provide the logo and link to the MarkMail gadget.
The rest of the logic on this gadget are the same as the previous gadget sample, and you are already familiar with this.

Here is the output of this new gadget:

MarkMail Gadget SVN Data Processed

Note that, even though the dates on this gadget are links, they will not work, as they were originally relative links to MarkMail site,
and we now display this as processed data on this gadget, and are now treated to be relative to the root of the Gadget Server hosting the gadget.
But with some simple JavaScript logic, we can point those back to MarkMail and correct the links.


Using TEXT vs DOM Content Type when Working with HTML

In this gadget, that we just discussed, we used the content type request parameter set to TEXT.

          parameters[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.TEXT;

There are four content types, TEXT, DOM, FEED and JASON, defined in Google gadget API [1].
Using TEXT content type, then setting the text content to inner HTML of a hidden element and them processing it as part of
HTML DOM document seems incorrect, when we have the opportunity to use content type DOM.

          parameters[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.DOM;

Well, this thinking is not entirely correct, given that our use case is to fetch HTML from a URL and process that and build the gadget content.
DOM content type is more appropriate when we want to deal with XML data fetched from SOAP or REST service endpoint.
An example on the use of DOM can be found in Google gadget documentation [2].
The pitfalls of working with HTML using DOM content type is also discussed on iGoogle developer forum [3].

The main fact to understand here is that the DOM returned by "makeRequest" is not part of the HTML document DOM, rather a separate DOM representation of the feted content.
Since the fetched content’s DOM is not part of the HTML document DOM tree, even though the fetched content is HTML, it cannot be easily attached to the gadget’s HTML document.

So when working with HTML, we are better off using TEXT content type and adopt the strategy of processing the response as text and attaching to HTML document as inner HTML.


Using External JavaScript Libraries

Now that we know how to fetch and process HTML within a gadget, let us see how we can use some external JavaScript library to bring in some visual appeal into a gadget.
In this step, we will use a JavaScript library to draw a graph based on the data contained in the HTML table.

There are many table-2-graphs libraries out there and let’s use the Bluff JavaScript graph library [4] for this.

When you deploy addition resources that the gadget users, such as JavaScript files, style sheets and images, you can use the registry based gadget deployment model
supported by WSO2 Gadget Server. Details on how to upload resources to registry is described in an article on WSO2 OxygenTank [5].

The three JavaScript files from Bluff library, required to draw graphs, can be uploaded into the registry, inside a collection named "js" relative to the location
where the gadget is located in registry, as shown in the image below.

Gadget Registry Resources

Note the path and the files appearing in the selected path, as highlighted in the above screenshot of the WSO2 Gadget Server’s registry.
Once the libraries are uploaded into the path where the gadget is located, we can refer to the JavaScript files, from within the HTML code of the gadget,
as we would do with the normal file system:

  <script type="text/javascript" src="js/js-class.js"></script>
  <script type="text/javascript" src="js/excanvas.js"></script>
  <script type="text/javascript" src="js/bluff-min.js"></script>

Now that we know the mechanism to make the required libraries available, let’s focus on the logic of the gadget.
Following is the source code for the complete gadget. On your left hand side, you find the gadget source and on your right hand side, the description.

If you want to clarify the logic on line number, say 28, look at the corresponding line on the right hand side description.

<?xml version="1.0" encoding="UTF-8" ?>
<Module>
    <ModulePrefs title="WSO2 Open Source Development Data from MarkMail">
        <Require feature="dynamic-height" />
    </ModulePrefs>
    <Content type="html">
        <![CDATA[
            <a href="https://markmail.org/"><img src="https://markmail.org/images/logo_white.gif" alt="MarkMail"/></a>  
            <div id="contentDiv"></div>
            <span id="hiddenSpan" style="display:none">Hidden</span>
            <canvas id="graph" width="600" height="625">Graph</canvas>
          
            <script type="text/javascript" src="js/js-class.js"></script>
            <script type="text/javascript" src="js/excanvas.js"></script>
            <script type="text/javascript" src="js/bluff-min.js"></script>
            
            <script type="text/javascript">
            
              function getHtml() {    
                  var parameters = {};  
                  parameters[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.TEXT;  
                  var url = "https://markmail.org/browse/org.wso2.carbon-commits";  
                  gadgets.io.makeRequest(url, processResponse, parameters);
              };
            
              function processResponse(response) {  
                    var textData = response.text;
                    textData = textData.replace(/,/g, '');
                    textData = textData.replace(/td><a/gi, 'th span="row"><a');
                    textData = textData.replace(/\/a><\/td/gi, '/a></th');
                   
                    document.getElementById('hiddenSpan').innerHTML = textData;
                    document.getElementById('contentDiv').innerHTML = 
                        "<table id='data'><caption>WSO2 Carbon SVN Commits</caption><thead><tr><th></th><th scope='col'>Commits</th></tr></thead>" + 
                        document.getElementById('browse').getElementsByTagName('table')[0].innerHTML + 
                        "</table>";
                   
                    var g = new Bluff.SideBar('graph', '600x625');
                    g.theme_keynote();
                    g.tooltips = true;
                    g.data_from_table('data');
                    g.draw();
               
                    document.getElementById('contentDiv').style.display = "none";
                 
                    gadgets.window.adjustHeight();
              };
            
              gadgets.util.registerOnLoadHandler(getHtml);
            
            </script>
        ]]>
    </Content>
</Module>

 
<- Start Module
    < - Module Preferences 
        <- Require the "dynamic-height" feature 
                used to adjust height 
    <- Start HTML content logic
	    
	    <- Link to MarkMail home page  
	    <- Content div where we will have the processed data table
	    <- Hidden span to hold fetched data
	    <- Graph canvas element used to hld the graph
	  
	    <- Third party JavaScript libraries 
	            used to draw graphs. These are 
	            stored in registry and referenced using relative links 
	    
	    <- Start JavaScript Logic 
	    
	      getHtml function     
	            
	          Set content type to be TEXT;  
	          Use MarkMail Carbon commits list archive as URL  
	          Make request with processResponse as callback
	      
	    
	      processResponse function   
	            Use response text
	            Replace ',' in numbers so that 1,200 become 1200, so that graphs library treat those as numbers
	            Replace td elements with th so that graph library treat them as axis
                  labels rather than data
	           
	            Make the processed text data to be inner HTML of hidden span element
	            Locate the table element in browse div of the received 
	                data, then wrap it with table caption and column 
	                headings and make that the inner HTML of 
	                content div
	           
	            Create a side bar graphs with Bluff library and use canvas element with "graph" ID to draw it
	            Set graph theme
	            Enable tool tips on graph
	            Provide the table with ID "data" as data source of the graph
	            Draw the graph
	       
	            Hide the content div, as that has only a table - we want to show only the graph
	         
	            Adjust the height of the window to show the full graph
	      
	    
	      Call getHtml function as the on load handler. This will trigger the fetiching of data and thus the processReponse callback
	    
	    <- End JavaScript Logic
	  
    <- End HTML content logic
<- End Module

You might wonder why we used two panes, one to present code and another to present logic, rather than source comments.
Note that, in cease of gadgets, even the comments get downloaded to the browser as content, increasing the download size. So you might want to keep those to minimum.

Here is the output of this gadget, with the graph in place.

MarkMail Gadget SVN Data Graph

Note: If you have trouble viewing this gadget with Internet Explorer (IE9), make sure that you switch from document mode Quirks to document mode IE9.
You can set the document mode using the developer tools. Press function key F12, while the IE window is active, and you can set the document mode using the right most Window menu.

Now we have studied how to make a fully-fledged and visually appealing Google gadget, fetching HTML from a URL, processing the received HTML,
and using JavaScript library to present the data in a visually appealing manner.

In the next part of this tutorial, the third, we will explore how to let the users set preferences, and make use of the same gadget to present different data in different formats.

References

[1] href="https://code.google.com/apis/gadgets/docs/remote-content.html#Content_Types">Google
gadget API documentation on remote content types

[2] href="https://code.google.com/apis/gadgets/docs/remote-content.html#Fetch_XML">DOM
content type example from Google gadget documentation

[3] href="https://markmail.org/message/fophexdfdakwnitr#query:+page:1+mid:d77dptoyzqenxdtu+state:results">Working
with HTML, iGoogle developer forum discussion archive on MarkMail

[4] Bluff
JavaScript graph library

[5] href="https://wso2.org/library/articles/authoring-deploying-using-xml-gadgets-wso2-gadget-server#UploadingToRegistry">Deploying
resources to WSO2 Gadget Server’s registry




This brings us to the end of this four part tutorial. Please feel free to provide us the feedback, and help us improve the content.

Please refer to the table of contents section to find out about the other parts of this tutorial.

 

About Author

  • Samisa Abeysinghe
  • VP, Training
  • WSO2 Inc.