Tuesday, May 24, 2011

BIRT: Using XLS as a data source

BIRT currently supports many data sources, including XML, Web Services, JDBC and flat files. This list can also be extended by implementing BIRT extension points. Using this method is called implementing an ODA(Open Data Access) extension.

An ODA allows the developer to add custom GUI elements to the BIRT designer to create and manipulate a data source. This method is described in a set of articles located on BIRT-Exchange. BIRT also supplies a scripted data source that allows a developer to write code in a simplified interface to retrieve data using Java or JavaScript. This post explains how a scripted data source can be implemented to retrieve XLS data using the Source Forge Java Excel API project. For more information on using this API see this blog post.


Adding the JAR to the BIRT Project


First download the Java Excel API and extract the jxl.jar file from the download. You can add this Jar to your BIRT project in several different ways. See this post for designer classpath options. One simple way to add the jar is to put the jar in your BIRT Project Resource folder and then add a reference to it in the report design. You can set the resource folder in the BIRT Designer Preferences (Window-Preferences-Report Design-Resource). The resource folder is also a web.xml setting in the deployed viewer.



In the above screen shot we have a resources folder under the BIRT reports project. Just import the jxl.jar into this folder. To create the reference to the jar in a report design, select the properties view for the report and click on resources. Click the add file button in the jar files section of the report resources and add the jar.



The jar file should now be accessible in the designer from script or expressions. When you deploy the report to the Viewer make sure to copy the jar to the resource folder of the deployed Viewer.

If this method of adding the jar is not desirable you can always create a folder in the project containing the lib and add the jar to the project classpath entry (Window-Preferences-Report Design-Classpath). This will allow the jar to be accessed at design time. When deploying the report you can then just copy the jar to the WEB-ONF/lib folder of the deployed viewer.


Implementing the Scripted Data Source/Set


BIRT supports implementing a scripted data source in Java or JavaScript. In this example we implement it using JavaScript. Although the code is written in JavaScript we can still call Java code by using the importPackage method. This example uses this approach.

Use the new data source wizard to add a scripted data source and use the new data set wizard to add a new scripted data set. In this example we are reading a three column xls file, so we define 3 columns in our dataset.





The final portion of this example requires that we add code to the script events to call the Java Excel API. Select the new data set and click on the script tab. From the drop down menu select the open event. Enter the following code.



importPackage( Packages.java.io);
importPackage( Packages.jxl );
try{
var inputWorkbook = new File(params["XLSLocation"].value);
myworkbook = Workbook.getWorkbook(inputWorkbook);
sheet = myworkbook.getSheet(0);
numrows = sheet.getRows();
curr = 0;
}catch(e){
curr=0;
numrows=0;
}



This code opens an xls file pointed to by the report parameter XLSLocation. You can hard code the file but this example is a bit more dynamic. The code then opens the first sheet in the workbook and sets a global JavaScript variable for the number of rows and initializes a current row counter to zero.



Select the drop down menu and select the fetch event and enter the following code.



if( curr < numrows ){

var cell1 = sheet.getCell(0, curr);
row["col1"] = cell1.getContents();
var cell2 = sheet.getCell(1, curr);
row["col2"] = cell2.getContents();
var cell3 = sheet.getCell(2, curr);
row["col3"] = cell3.getContents();

curr++
return true;
}else{
return false;
}



The fetch event is fired once per row of data and stops processing when the event returns false. In this code we are just retrieving the first three columns of the xls sheet and assigning them to the dataset row columns.



The final event to implement is the close event. Select close from the drop down box and enter the following code.


if( typeof myworkbook != 'undefined' ){
myworkbook.close();
}




This code closes the workbook.



You should now be able to preview the dataset.



Remember to add the parameter that defines the location of the xls file. Once the dataset is complete you can construct your BIRT report as usual. This attached example produces the following report.



This example is available at BIRT-Exchange.

Thursday, April 28, 2011

BIRT - Reducing large queries and custom navigation controls

Often times when writing a report that requires large amounts of data, the query times become burdensome. This will lead to dissatisfaction from customers of your report. To get around this issue certain companies like Actuate implement progressive viewing in their products that start returning formatted data before the report generation process is complete. This allows users of the report to start getting data long before the total report is completed.

Another option is to reduce the large queries into smaller queries and add navigation controls to your report to get the next smaller set of data. For example, if you have a customer listing report, the first query may return all customers whose name starts with the letter A. A set of navigation controls could have a next button that returns customers whose name starts with the letter B, etc. This approach is very simple in BIRT and this post details one way of doing it.

As an example we can use the classic models sample database that comes with BIRT. It contains a customers table. To return all customers starting with A we could use the query select * from customers where customername like ‘A%’. We want the report to be dynamic, so we will need script to modify the query on the fly. We can begin by just adding a new datasource that uses the classic models database. Next we create a dataset with the following query:

Select * from customers

The next step is to add a hidden report parameter that we use in script to add a where clause to the query. In this example the parameter is named “like”.



The parameter is hidden and has a default value of A. We can now use this parameter in the beforeOpen script of the data set we created earlier. This script should look like the following:

this.queryText = this.queryText + " where customername like '" + params["like"].value + "%'";

This script will change the query every time the report runs and will append the where clause with the hidden report parameter. The default value for the parameter is A which should list all customers starting with A. Before adding navigation controls to the report we need to create and calculate some global variables that our controls will need. Ideally we will add a previous and a next button to the report. In addition it would also be nice to have a list A-Z that the user could click on to go directly to a set of customers. So at the minimum when the report runs we need to get the parameter for the report and calculate what the previous and next customer names should start with. For example if the parameter is C, previous should be B and next should be D. This can be done with a simple beforeFactory script like:



switch( params['like'].value ){
case 'A':
prev='Z';
next='B';
break;

case 'B':
prev='A';
next='C';
break;

case 'C':
prev='B';
next='D';
break;
.
.
.
}


The beforeFactory script is used because it runs before data is retrieved and executes only once. The BIRT initialize script can be executed multiple times so it is not ideal for this script. The var keyword is also not used with the variables prev and next which will make them visible to downstream events. You could also use the reportContext.setGlobalVariable method to do store the variables as well.

After the table is added to the report you can then add a Text element that contains a script tag to implement your prev and next buttons.



Remember to set the type to HTML. The script tag contains a client side script that just reloads the report with a new value for the like parameter. This function is called nextData and takes the new like value as a parameter. The text element also contains two buttons that call the nextData function with the prev and next server side javascript variables we calculated in the beforeFactory. Notice this is done with the VALUE-OF tag. This tag allows us to combine server side scripting with client side scripting.



<script>
function nextData( a ){
var cl = new String(location.href);
var targetURL = new String();
var tst = cl.indexOf("&like");
if( tst == -1 ){
cl = cl + "&like=" + a;
}else{
var ch = cl.indexOf("&like=");
cl = cl.substring(0, ch) + "&like=" + a;

}
window.location = cl;
}
</script>
<input type="button" value="Previous"
onclick="nextData('<VALUE-OF>prev;</VALUE-OF>')" >
<input type="button" value="Next"
onclick="nextData('<VALUE-OF>next;</VALUE-OF>')" >


The last thing we add to the Text element is an Alpha index for all letters of the alphabet.


<br>
<a href="#" onclick="nextData('A');return false">A</href>
<a href="#" onclick="nextData('B');return false">B</href>
<a href="#" onclick="nextData('C');return false">C</href>
<a href="#" onclick="nextData('D');return false">D</href>
<a href="#" onclick="nextData('E');return false">E</href>
.
.


This allows the user to directly select a letter to use in the like clause. The output for the report looks like the following.


The report also handles the situation when no data is returned.



The no data label is placed in a grid below the table. The grid is bound to the table so it uses the same dataset. The dataset contains an aggregate computed column that counts the number of rows. This computed column is then used in the grid and the table visibility property.



For example the grid visibility is shown below.



The expression is:
(row["myrowcount"] > 0)
The expression returns true when rows exist. This will then hide the grid and label. The opposite condition is used on the table.
(row["myrowcount"] < 1)
This report is available at Birt-Exchange.

Wednesday, March 30, 2011

BIRT Roadshow coming to a city near you.

Actuate is continuing the very popular BIRT Road Shows that feature a free day of training on BIRT technologies. If you are interested, the dates and cities for April and May are listed below. Slots fill up fast so be sure to register quickly. More information is available here.

April
Seattle, WA Tuesday | 4/5
Minneapolis, MN Wednesday | 4/6
Indianapolis, IN Thursday | 4/7
Philadelphia, PA Wednesday | 4/13
May
Denver, CO Tuesday | 5/3
Toronto, ON Wednesday | 5/4
Princeton, NJ Tuesday | 5/10
Orlando, FL Wednesday | 5/11
Dallas, TX Thursday | 5/12
London, UK Thursday | 5/19
Louisville, KY Tuesday | 5/24
Washington, DC Wednesday | 5/25

Friday, March 04, 2011

BIRT 2.6.2 Enhancements

BIRT 2.6.2 is now available and is a maintenance release with a primary concern on bug fixes. This generally means not many new features are introduced. That said a couple of bug fixes merit special attention. The full list of available fixes can be found here.


BIRT Pagination and Page Scripts


BIRT 2.5 introduced page variables and page scripts. These features added the ability to repaginate based on things like a group change. So in one report output you could add a page variable that showed the page count and current group page number within a group. The full details for this feature are available in the BIRT 2.5 new and notable.

While using this feature is very nice, a particular bug caused the scripts not to fire when using a page break interval of 0. The page break interval is one of the many ways in BIRT to introduce a page break in the output. It essentially informs the engine to process a certain number of rows and then insert a page break. If the interval for a table is set to 0, the engine inserts page breaks when all the master page available real-estate is used. In BIRT 2.6.2 this bug is now fixed, allowing the designer to enter a page break interval of 0 in conjunction with page scripts.



The above example can be downloaded from BIRT-Exchange.

To see some of the pagination bugs that were addressed in 2.6.2 see this bugzilla query.


URL security bug



A security issue with the example Web Viewer was identified which posed a cross-site scripting vulnerability. The issue has to do with the __report URL parameter. This parameter specifies the report for the viewer to run. In previous versions of BIRT a fully qualified URL could be used to specify the report, which allowed the report to reside on a completely different server. Since BIRT reports can contain JavaScript which runs server side this poses a scripting risk. The full details for the bug are located here.

BIRT 2.6.2 fixes this issue by adding a web.xml entry that allows the developer to control what value can be used in the __report URL parameter. The entry is named URL_REPORT_PATH_POLICY and has three valid values (all, domain, and none). If the all value is specified no checks on the URL parameter are performed. This value should only be used when access to the viewer is restricted to a known user base. When the none option is used, only local reports to the viewer context for the server are allowed. When the domain option is used the report can contain a fully qualified URL as long as the domain for the server is the same.



If you cannot upgrade the viewer, the patch details are available in the bugzilla entry.


Formatter object


BIRT 2.6.2 now offers a formatter object in the expression builder to format dates, strings and numbers within a given expression. This function is detailed in the following post.




Radar Chart


The Radar Chart in BIRT was implemented in BIRT 2.6 to function as an example on how to implement your own chart types. The source for this chart type is available in two of the BIRT plug-ins (org.eclipse.birt.chart.examples and org.eclipse.birt.chart.examples.core). In BIRT 2.6.2 this chart type has been improved and it now supports rendering multiple series on the same radial within a radar chart.



The GUI for building the radar chart has also been improved to support a more optimal layout for configuring portions of the chart.

Friday, February 18, 2011

BIRT Formatting numbers and dates

BIRT offers many options when formatting data is required. This post discusses some of the common ways this can be done using the GUI, script and expressions.


Using the GUI to format data


When formatting the standard Data report item, customizing the format is very simple. Located within the properties view for the data item are three tabs that allow formatting a number, date, or string. Which format the data item uses will depend on the data type for the data item as specified in the binding.





Each of the format tabs provides a set of predefined patterns that can be applied to the data. In addition, the pattern can be set to custom allowing the developer to define very specific formatting codes. The formatting operation can be set to a specific locale or auto which will use the locale of the user running the report. For example, entering ¤###,##0.00 as the pattern for a data item that is going to contain a currency may produce €131,44 for the French locale, and $131.44 for a US locale. For an example of using positive:negative patterns you may want to have a look at this post.

Report parameters selections can also be formatted in similar fashion to a Data report item. Each parameter has a “Format As” selection within the Parameter Editor. By selecting the change button a standard or custom format can be applied to parameter.



Formatting labels within the chart builder is also possible. Most items in a chart that display data have an associated format that can be customized by using the format button. For example to format a date axis use the GUI as presented below.




Formatting Data using the Expression builder or Script



The above formatting solutions work very well, but there are times when you want to mix formatted data within a string. In this case you can write script, use and expression, or use a Text report item to format the data. For example assume a report has two dates that are passed as parameters and you want to generate text within the report that shows “Start Date – End Date”. This is not achievable using the GUI methods described above. To do this you can use a text item and the value of syntax:



Start Date: <VALUE-OF format="MMM-dd-yy">params["date1"].value;</VALUE-OF>
End Date: <VALUE-OF format="MMM-dd-yy">params["date2"].value;</VALUE-OF>




In this example the VALUE-OF syntax is evaluated at runtime of the report and the format attribute is applied to the output. This approach also works with numbers. For example, <VALUE-OF format="$#,##0.00">row[“MyData”];</VALUE-OF>.

This same functionality could have been achieved by using a data item with the following expression.


var mydt1 =params["date1"].value
var mydt2 =params["date2"].value

importPackage( Packages.java.text );
var sdf = new SimpleDateFormat("MMM d, yyyy", reportContext.getLocale());
var mydtf1 = sdf.format( mydt1 );
var mydtf2 = sdf.format( mydt2 );
"Start Date: " + mydtf1 + " End Date:" + mydtf2




In this example we are just using the imported Java SimpleDateFormat class to do the formatting.

If you are using BIRT 2.6.2 or higher, you can use the Formatter function within the expression builder to format localized numbers, dates and string as well. The Formatter function is available within the BIRT Functions Category of the expression builder. To build the example above, use the following expression in data item.



var mydtf1 = Formatter.format( params["date1"].value, "MMM d, yyyy");
var mydtf2 = Formatter.format( params["date2"].value, "MMM d, yyyy");
"Start Date: "+mydtf1 + " End Date " + mydtf2 + BirtDateTime.diffDay(params["date2"], params["date1"]) +" Days";




In this example we have combined the start and end dates with the BirtDateTime.diffDay function to build a string that not only shows the start and end date, but shows the duration of days.

The Formatter function takes the current locale into account when formatting the element. If you want to test different locales, either set the preview locale in the preferences or use the __locale parameter in your report URL.



As a side note, if you are interested in adding your own functions to the expression builder, see this post.

You can also import Java classes to manipulate data points within the BIRT or Chart event handlers. For example to set your own format on an axis label the following script could be used on a chart.


function beforeDrawAxisLabel( axis, label, context )
{
importPackage(Packages.org.eclipse.birt.chart.model.attribute);
importPackage(Packages.java.lang);
importPackage(Packages.java.text);
if( context.getExternalContext().getScriptable().getParameterValue("MyParameter") == "date"){
if (axis.getType() == AxisType.TEXT_LITERAL)
{
value = label.getCaption().getValue();
var dtf = new SimpleDateFormat("MM/dd/yy");
var dt = new Date(value);
var fn = dtf.format(dt);
label.getCaption().setValue(fn);
}
}


As another example, the code below could be placed in a report parameter’s getDefaultValueList event handler to return the current year.


importPackage( Packages.java.util );
cal = Calendar.getInstance();
tyear = cal.get(Calendar.YEAR);
tyear;


Thursday, January 27, 2011

Changing Eclipse BIRT Indigo Version Number to BIRT 3.7

For the BIRT project’s Indigo release in June of this year, our plan continues to be to match the BIRT version number with the Eclipse Platform version number used for Indigo. Going forward, this will make it easier for everyone to remember which version of BIRT is designed to work with which version of the Platform. And with Indigo being the 8th major release of the BIRT project, there is no downside with updating the release number to match the Platform release.


At the outset of the Indigo project, it was an open question whether Indigo would be Eclipse 3.7 or 4.0 – so as a short term measure, we adopted BIRT 4.0 as the working release number for BIRT Indigo. Now that the Indigo release will be designated Eclipse 3.7, we plan to update BIRT Indigo to use the BIRT 3.7 designation.

Friday, January 14, 2011

Building BIRT Based Dashboards

If you are using Actuate’s iServer technology, one of the new features for version 11 is the ability to build dashboards using nothing but the browser. These dashboards can be constructed using open source BIRT reports, portions of a BIRT report, Flash Objects, Google Gadgets, and other web content. Completed dashboards and gadgets can be shared with others and can be included in other web applications using the Actuate JSAPI.

To start building a dashboard, select the new tab icon in the Actuate Information Console.



This will create a dashboard canvas as shown below.



By selecting the down arrow icon on the newly created tab, you can share the dashboard, rename it, set the layout or define data that will be used in the dashboard. Using the Options field will also allow you to set the refresh time for a dashboard. This can be used to present things like live charts or tables. The layout option allows you to define how the dashboard is presented. You can set a columnar option or freeform which allows you to place the gadgets anywhere on the canvas.



The Dashboard and Gadget Gallery menu items contain previously constructed items that have been shared. The New Gadget menu contains the Gadgets that can be placed on the canvas. There are four categories depending on what type of gadget you wish to use: Data Visualization, Data Selection, Report and Extras.


Data Visualization Gadgets


The Data Visualization category contains a set of gadgets that can be used to analyze data. These include a crosstab, a table, and a set of Flash charts.

To use one of these gadgets, simply drag it to the canvas below the menu. For example, dragging the crosstab gadget onto the canvas will launch the Crosstab Builder.

The gadget retrieves its data from a data object or from an on-demand data design. The data design is essentially a set of BIRT data sources, data sets and data cubes. This allows you to leverage existing open source BIRT Reports as content for your dashboard. After assigning the design values for the crosstab you can move the gadget around the canvas to a suitable location.

Placing one of the chart types will launch the Chart Builder, which allows specifying the chart data.

Most data visualization gadgets support further customization for filtering and formatting. In addition the data can be further analyzed by drilling into the data by clicking on actionable fields or by using the Actuate Data Analyzer.



Data Selection Gadgets


The Data Selection category contains a set of gadgets that present selection able data that can be used to drive other gadgets. These gadgets include items like lists, combo boxes and calendar selectors. To use these, just drag the gadget to the canvas and set the data similar to process used for Data Visualization gadgets.

Once the gadget is placed you can link it to another gadget by selecting the target gadget and clicking on down arrow in the upper right corner of the gadget and select link.

This will launch the link editor where you can set field in the target you wish to link on.

In this particular example the selector updates the report parameter OrderDate for the Report Gadget. Once the linking is complete any time the selector is updated the target gadget will get updated with the new value. Selectors can be linked to multiple gadgets. For example, in the following screenshot, the product line list selector updates an exploded pie chart and a detail report.


Report Gadgets


The Report category contains a set of gadgets that handle presenting BIRT report content. This can include BIRT designs or generated documents, which allows for a cached or on demand design paradigm. In addition the design or document can be further processed to only display a given object in a report or document. Finally a parameter gadget allows the user to use a gadget that displays the parameters for a particular report. This gadget can then be linked to a report gadget, which will be updated when the parameter gadget is changed.

When dragging these gadgets to the canvas a New Report Gadget dialog will be displayed, this allows you to configure the location and parameters for a given report.

Using the refresh timer on the dashboard will also allow you to update how often the report is updated.

The Reportlet gadget allows the user to define a portion of a BIRT report or document to display. When using these one extra field is available in the New Report Gadget dialog. The field is the Bookmark field and corresponds with bookmark property for the report item in the BIRT report design.

In this example the bookmark Hist is located in the MasterSalesDashboard report design. The Hist bookmark in the BIRT report design corresponds to a grid report item that contains historic sales data.

The Parameter gadget functions similarly to the Report and Reportlet gadgets, but only displays the parameter entry dialogs. Once you have a Parameter gadget and a Report gadget on the same canvas you can link them by editing the Report gadget using the down arrow in the upper right corner of the gadget.

In the Edit Report Gadget dialog choose the Parameter Display Setting tab and in the first drop down select Link to Selectors. For each report parameter use the succeeding drop down menus to link them to the parameter gadget.

Once this is complete updating the parameter in the Parameter gadget will update the report in the Report gadget.


Extras Gadgets


The Extras category contains a set of gadgets that allow you to include images, HTML, customized text, video and Google gadgets in the dashboard.

Each gadget will have a custom dialog for configuring the gadget. For example the Import Gadget item dialog allows you to specify the location for a Google Gadget.


Conclusion


This post is an overview of the building BIRT based dashboards using Actuate technology. If you want more details see the help documentation. If you want to try out this functionality download it from BIRT-Exchange.