BIRT - Reducing large queries and custom navigation controls

Often times when writing a report that requires large amounts of data, the query times become burdensome. This will lead to dissatisfaction from customers of your report. To get around this issue certain companies like Actuate implement progressive viewing in their products that start returning formatted data before the report generation process is complete. This allows users of the report to start getting data long before the total report is completed.

Another option is to reduce the large queries into smaller queries and add navigation controls to your report to get the next smaller set of data. For example, if you have a customer listing report, the first query may return all customers whose name starts with the letter A. A set of navigation controls could have a next button that returns customers whose name starts with the letter B, etc. This approach is very simple in BIRT and this post details one way of doing it.

As an example we can use the classic models sample database that comes with BIRT. It contains a customers table. To return all customers starting with A we could use the query select * from customers where customername like ‘A%’. We want the report to be dynamic, so we will need script to modify the query on the fly. We can begin by just adding a new datasource that uses the classic models database. Next we create a dataset with the following query:

Select * from customers

The next step is to add a hidden report parameter that we use in script to add a where clause to the query. In this example the parameter is named “like”.



The parameter is hidden and has a default value of A. We can now use this parameter in the beforeOpen script of the data set we created earlier. This script should look like the following:

this.queryText = this.queryText + " where customername like '" + params["like"].value + "%'";

This script will change the query every time the report runs and will append the where clause with the hidden report parameter. The default value for the parameter is A which should list all customers starting with A. Before adding navigation controls to the report we need to create and calculate some global variables that our controls will need. Ideally we will add a previous and a next button to the report. In addition it would also be nice to have a list A-Z that the user could click on to go directly to a set of customers. So at the minimum when the report runs we need to get the parameter for the report and calculate what the previous and next customer names should start with. For example if the parameter is C, previous should be B and next should be D. This can be done with a simple beforeFactory script like:




switch( params['like'].value ){
case 'A':
prev='Z';
next='B';
break;

case 'B':
prev='A';
next='C';
break;

case 'C':
prev='B';
next='D';
break;
.
.
.
}



The beforeFactory script is used because it runs before data is retrieved and executes only once. The BIRT initialize script can be executed multiple times so it is not ideal for this script. The var keyword is also not used with the variables prev and next which will make them visible to downstream events. You could also use the reportContext.setGlobalVariable method to do store the variables as well.

After the table is added to the report you can then add a Text element that contains a script tag to implement your prev and next buttons.



Remember to set the type to HTML. The script tag contains a client side script that just reloads the report with a new value for the like parameter. This function is called nextData and takes the new like value as a parameter. The text element also contains two buttons that call the nextData function with the prev and next server side javascript variables we calculated in the beforeFactory. Notice this is done with the VALUE-OF tag. This tag allows us to combine server side scripting with client side scripting.

<script>
function nextData( a ){
var cl = new String(location.href);
var targetURL = new String();
var tst = cl.indexOf("&like");
if(  tst == -1 ){
 cl = cl + "&like=" + a;
}else{
 var ch = cl.indexOf("&like=");
 cl = cl.substring(0, ch) + "&like=" + a;
    
}
window.location = cl;
}
</script>
<input type="button" value="Previous"
onclick="nextData('<VALUE-OF>prev;</VALUE-OF>')" >
<input type="button" value="Next"
onclick="nextData('<VALUE-OF>next;</VALUE-OF>')" >

The last thing we add to the Text element is an Alpha index for all letters of the alphabet.

<br>
<a href="#" onclick="nextData('A');return false">A</href>
<a href="#" onclick="nextData('B');return false">B</href>
<a href="#" onclick="nextData('C');return false">C</href>
<a href="#" onclick="nextData('D');return false">D</href>
<a href="#" onclick="nextData('E');return false">E</href>
.
.

This allows the user to directly select a letter to use in the like clause. The output for the report looks like the following.


The report also handles the situation when no data is returned.



The no data label is placed in a grid below the table. The grid is bound to the table so it uses the same dataset. The dataset contains an aggregate computed column that counts the number of rows. This computed column is then used in the grid and the table visibility property.



For example the grid visibility is shown below.



The expression is:
(row["myrowcount"] > 0)
The expression returns true when rows exist. This will then hide the grid and label. The opposite condition is used on the table.
(row["myrowcount"] < 1)
This report is available at Birt-Exchange.

BIRT Roadshow coming to a city near you.

Actuate is continuing the very popular BIRT Road Shows that feature a free day of training on BIRT technologies. If you are interested, the dates and cities for April and May are listed below. Slots fill up fast so be sure to register quickly. More information is available here.

April
Seattle, WA Tuesday | 4/5
Minneapolis, MN Wednesday | 4/6
Indianapolis, IN Thursday | 4/7
Philadelphia, PA Wednesday | 4/13
May
Denver, CO Tuesday | 5/3
Toronto, ON Wednesday | 5/4
Princeton, NJ Tuesday | 5/10
Orlando, FL Wednesday | 5/11
Dallas, TX Thursday | 5/12
London, UK Thursday | 5/19
Louisville, KY Tuesday | 5/24
Washington, DC Wednesday | 5/25

BIRT 2.6.2 Enhancements

BIRT 2.6.2 is now available and is a maintenance release with a primary concern on bug fixes. This generally means not many new features are introduced. That said a couple of bug fixes merit special attention. The full list of available fixes can be found here.

BIRT Pagination and Page Scripts

BIRT 2.5 introduced page variables and page scripts. These features added the ability to repaginate based on things like a group change. So in one report output you could add a page variable that showed the page count and current group page number within a group. The full details for this feature are available in the BIRT 2.5 new and notable.

While using this feature is very nice, a particular bug caused the scripts not to fire when using a page break interval of 0. The page break interval is one of the many ways in BIRT to introduce a page break in the output. It essentially informs the engine to process a certain number of rows and then insert a page break. If the interval for a table is set to 0, the engine inserts page breaks when all the master page available real-estate is used. In BIRT 2.6.2 this bug is now fixed, allowing the designer to enter a page break interval of 0 in conjunction with page scripts.



The above example can be downloaded from BIRT-Exchange.

To see some of the pagination bugs that were addressed in 2.6.2 see this bugzilla query.

URL security bug

A security issue with the example Web Viewer was identified which posed a cross-site scripting vulnerability. The issue has to do with the __report URL parameter. This parameter specifies the report for the viewer to run. In previous versions of BIRT a fully qualified URL could be used to specify the report, which allowed the report to reside on a completely different server. Since BIRT reports can contain JavaScript which runs server side this poses a scripting risk. The full details for the bug are located here.

BIRT 2.6.2 fixes this issue by adding a web.xml entry that allows the developer to control what value can be used in the __report URL parameter. The entry is named URL_REPORT_PATH_POLICY and has three valid values (all, domain, and none). If the all value is specified no checks on the URL parameter are performed. This value should only be used when access to the viewer is restricted to a known user base. When the none option is used, only local reports to the viewer context for the server are allowed. When the domain option is used the report can contain a fully qualified URL as long as the domain for the server is the same.



If you cannot upgrade the viewer, the patch details are available in the bugzilla entry.

Formatter object

BIRT 2.6.2 now offers a formatter object in the expression builder to format dates, strings and numbers within a given expression. This function is detailed in the following post.

Radar Chart

The Radar Chart in BIRT was implemented in BIRT 2.6 to function as an example on how to implement your own chart types. The source for this chart type is available in two of the BIRT plug-ins (org.eclipse.birt.chart.examples and org.eclipse.birt.chart.examples.core). In BIRT 2.6.2 this chart type has been improved and it now supports rendering multiple series on the same radial within a radar chart.



The GUI for building the radar chart has also been improved to support a more optimal layout for configuring portions of the chart.

BIRT Formatting numbers and dates

BIRT offers many options when formatting data is required. This post discusses some of the common ways this can be done using the GUI, script and expressions.

Using the GUI to format data

When formatting the standard Data report item, customizing the format is very simple. Located within the properties view for the data item are three tabs that allow formatting a number, date, or string. Which format the data item uses will depend on the data type for the data item as specified in the binding.





Each of the format tabs provides a set of predefined patterns that can be applied to the data. In addition, the pattern can be set to custom allowing the developer to define very specific formatting codes. The formatting operation can be set to a specific locale or auto which will use the locale of the user running the report. For example, entering ¤###,##0.00 as the pattern for a data item that is going to contain a currency may produce €131,44 for the French locale, and $131.44 for a US locale. For an example of using positive:negative patterns you may want to have a look at this post.

Report parameters selections can also be formatted in similar fashion to a Data report item. Each parameter has a “Format As” selection within the Parameter Editor. By selecting the change button a standard or custom format can be applied to parameter.



Formatting labels within the chart builder is also possible. Most items in a chart that display data have an associated format that can be customized by using the format button. For example to format a date axis use the GUI as presented below.

Formatting Data using the Expression builder or Script

The above formatting solutions work very well, but there are times when you want to mix formatted data within a string. In this case you can write script, use and expression, or use a Text report item to format the data. For example assume a report has two dates that are passed as parameters and you want to generate text within the report that shows “Start Date – End Date”. This is not achievable using the GUI methods described above. To do this you can use a text item and the value of syntax:

Start Date: <VALUE-OF format="MMM-dd-yy">params["date1"].value;</VALUE-OF>
End Date: <VALUE-OF format="MMM-dd-yy">params["date2"].value;</VALUE-OF>

In this example the VALUE-OF syntax is evaluated at runtime of the report and the format attribute is applied to the output. This approach also works with numbers. For example, <VALUE-OF format="$#,##0.00">row[“MyData”];</VALUE-OF>.

This same functionality could have been achieved by using a data item with the following expression.

var mydt1 =params["date1"].value
var mydt2 =params["date2"].value

importPackage( Packages.java.text );
var sdf = new SimpleDateFormat("MMM d, yyyy", reportContext.getLocale());
var mydtf1 = sdf.format( mydt1 );
var mydtf2 = sdf.format( mydt2 );
"Start Date: " + mydtf1 + " End Date:" + mydtf2

In this example we are just using the imported Java SimpleDateFormat class to do the formatting.

If you are using BIRT 2.6.2 or higher, you can use the Formatter function within the expression builder to format localized numbers, dates and string as well. The Formatter function is available within the BIRT Functions Category of the expression builder. To build the example above, use the following expression in data item.

var mydtf1 = Formatter.format( params["date1"].value,  "MMM d, yyyy");
var mydtf2 = Formatter.format( params["date2"].value,  "MMM d, yyyy");
"Start Date: "+mydtf1 + " End Date " + mydtf2 + BirtDateTime.diffDay(params["date2"], params["date1"]) +" Days";

In this example we have combined the start and end dates with the BirtDateTime.diffDay function to build a string that not only shows the start and end date, but shows the duration of days.

The Formatter function takes the current locale into account when formatting the element. If you want to test different locales, either set the preview locale in the preferences or use the __locale parameter in your report URL.



As a side note, if you are interested in adding your own functions to the expression builder, see this post.

You can also import Java classes to manipulate data points within the BIRT or Chart event handlers. For example to set your own format on an axis label the following script could be used on a chart.

function beforeDrawAxisLabel( axis, label, context )
{
 importPackage(Packages.org.eclipse.birt.chart.model.attribute);
 importPackage(Packages.java.lang);
 importPackage(Packages.java.text);
 if( context.getExternalContext().getScriptable().getParameterValue("MyParameter") == "date"){
  if (axis.getType() == AxisType.TEXT_LITERAL)
  {
   value = label.getCaption().getValue();
   var dtf = new SimpleDateFormat("MM/dd/yy");
   var dt = new Date(value);
   var fn = dtf.format(dt);
   label.getCaption().setValue(fn); 
  }
 }

As another example, the code below could be placed in a report parameter’s getDefaultValueList event handler to return the current year.

importPackage( Packages.java.util );
cal = Calendar.getInstance(); 
tyear = cal.get(Calendar.YEAR); 
tyear;

Changing Eclipse BIRT Indigo Version Number to BIRT 3.7

For the BIRT project’s Indigo release in June of this year, our plan continues to be to match the BIRT version number with the Eclipse Platform version number used for Indigo. Going forward, this will make it easier for everyone to remember which version of BIRT is designed to work with which version of the Platform. And with Indigo being the 8th major release of the BIRT project, there is no downside with updating the release number to match the Platform release.


At the outset of the Indigo project, it was an open question whether Indigo would be Eclipse 3.7 or 4.0 – so as a short term measure, we adopted BIRT 4.0 as the working release number for BIRT Indigo. Now that the Indigo release will be designated Eclipse 3.7, we plan to update BIRT Indigo to use the BIRT 3.7 designation.

Building BIRT Based Dashboards

If you are using Actuate’s iServer technology, one of the new features for version 11 is the ability to build dashboards using nothing but the browser. These dashboards can be constructed using open source BIRT reports, portions of a BIRT report, Flash Objects, Google Gadgets, and other web content. Completed dashboards and gadgets can be shared with others and can be included in other web applications using the Actuate JSAPI.

To start building a dashboard, select the new tab icon in the Actuate Information Console.



This will create a dashboard canvas as shown below.



By selecting the down arrow icon on the newly created tab, you can share the dashboard, rename it, set the layout or define data that will be used in the dashboard. Using the Options field will also allow you to set the refresh time for a dashboard. This can be used to present things like live charts or tables. The layout option allows you to define how the dashboard is presented. You can set a columnar option or freeform which allows you to place the gadgets anywhere on the canvas.



The Dashboard and Gadget Gallery menu items contain previously constructed items that have been shared. The New Gadget menu contains the Gadgets that can be placed on the canvas. There are four categories depending on what type of gadget you wish to use: Data Visualization, Data Selection, Report and Extras.

Data Visualization Gadgets

The Data Visualization category contains a set of gadgets that can be used to analyze data. These include a crosstab, a table, and a set of Flash charts.

To use one of these gadgets, simply drag it to the canvas below the menu. For example, dragging the crosstab gadget onto the canvas will launch the Crosstab Builder.

The gadget retrieves its data from a data object or from an on-demand data design. The data design is essentially a set of BIRT data sources, data sets and data cubes. This allows you to leverage existing open source BIRT Reports as content for your dashboard. After assigning the design values for the crosstab you can move the gadget around the canvas to a suitable location.

Placing one of the chart types will launch the Chart Builder, which allows specifying the chart data.

Most data visualization gadgets support further customization for filtering and formatting. In addition the data can be further analyzed by drilling into the data by clicking on actionable fields or by using the Actuate Data Analyzer.

Data Selection Gadgets

The Data Selection category contains a set of gadgets that present selection able data that can be used to drive other gadgets. These gadgets include items like lists, combo boxes and calendar selectors. To use these, just drag the gadget to the canvas and set the data similar to process used for Data Visualization gadgets.

Once the gadget is placed you can link it to another gadget by selecting the target gadget and clicking on down arrow in the upper right corner of the gadget and select link.

This will launch the link editor where you can set field in the target you wish to link on.

In this particular example the selector updates the report parameter OrderDate for the Report Gadget. Once the linking is complete any time the selector is updated the target gadget will get updated with the new value. Selectors can be linked to multiple gadgets. For example, in the following screenshot, the product line list selector updates an exploded pie chart and a detail report.

Report Gadgets

The Report category contains a set of gadgets that handle presenting BIRT report content. This can include BIRT designs or generated documents, which allows for a cached or on demand design paradigm. In addition the design or document can be further processed to only display a given object in a report or document. Finally a parameter gadget allows the user to use a gadget that displays the parameters for a particular report. This gadget can then be linked to a report gadget, which will be updated when the parameter gadget is changed.

When dragging these gadgets to the canvas a New Report Gadget dialog will be displayed, this allows you to configure the location and parameters for a given report.

Using the refresh timer on the dashboard will also allow you to update how often the report is updated.

The Reportlet gadget allows the user to define a portion of a BIRT report or document to display. When using these one extra field is available in the New Report Gadget dialog. The field is the Bookmark field and corresponds with bookmark property for the report item in the BIRT report design.

In this example the bookmark Hist is located in the MasterSalesDashboard report design. The Hist bookmark in the BIRT report design corresponds to a grid report item that contains historic sales data.

The Parameter gadget functions similarly to the Report and Reportlet gadgets, but only displays the parameter entry dialogs. Once you have a Parameter gadget and a Report gadget on the same canvas you can link them by editing the Report gadget using the down arrow in the upper right corner of the gadget.

In the Edit Report Gadget dialog choose the Parameter Display Setting tab and in the first drop down select Link to Selectors. For each report parameter use the succeeding drop down menus to link them to the parameter gadget.

Once this is complete updating the parameter in the Parameter gadget will update the report in the Report gadget.

Extras Gadgets

The Extras category contains a set of gadgets that allow you to include images, HTML, customized text, video and Google gadgets in the dashboard.

Each gadget will have a custom dialog for configuring the gadget. For example the Import Gadget item dialog allows you to specify the location for a Google Gadget.

Conclusion

This post is an overview of the building BIRT based dashboards using Actuate technology. If you want more details see the help documentation. If you want to try out this functionality download it from BIRT-Exchange.

BIRT Background/Watermark Images

Back in September I wrote a post about the BIRT Image report item. In that post I did not say much about creating background or watermarks for BIRT reports. In this post I will cover a few ways this can be accomplished.

One of the simplest approaches is to just add your watermark/background image to the resource folder that you have configured in the designer. Note that you will also need to add the image to the resource folder of the deployed environment. Once you have the image in the resource folder, select the style icon in the outline view and create a new style. Set the background image for the new style to the image you added to the resource folder. Next select the master page tab in the report editor and then right click on the canvas background and apply the new style.



The watermark can also be made to be dynamic. For example assume you want one watermark for pdf output and one for html. To do this you could add the second watermark to your resource folder and then use the following beforeFactory script to swap the image.




if( reportContext.getHttpServletRequest().getParameter("__format") == "pdf" ){
reportContext.getDesignHandle().findStyle("BackgroundStyle").backgroundImage = "watermarkpdf.png";
}else{
reportContext.getDesignHandle().findStyle("BackgroundStyle").backgroundImage = "watermarkhtml.png";
}




There is one drawback when using this approach. Master page content is always created in the generation phase of the BIRT pipeline, which means if you use the BIRT Viewer and export the report to another format the above code will not work. This is because the frameset mapping in the Viewer generates an rptdocument that can be rendered in additional formats without the need to re-execute the report. If the report is not re-executed the master page background image cannot be changed.

Another option which will facilitate exporting the report is to use a grid item instead of the master page for the watermark. For example assume there is one table in the report design. You can add a grid to the report that has one column and one cell. Set the height for the grid to be the size of your page (eg 11 in). Drag the table into the cell. Select the grid row and apply your background style.



Using the following onRender script of the grid row will allow you to change the style image at render time.

if( reportContext.getHttpServletRequest().getParameter("__format") == "pdf" ){
this.getStyle().backgroundImage = "watermarkpdf.png";
}

The examples used in this post are available on Birt-Exchange.