Friday, April 25, 2008

Data Driven Reports

Last week I posted about how to debug reports using a tester application. As part of that exercise I showed a dynamic table report where the number of columns in the report was driven by the data. This week, I'm going to examine how we access the database to determine the dynamic column values in that example.

BIRT event handlers have three distinct phases:

- BeforeFactory / OnPrepare
- OnCreate
- OnRender

If you want to modify the design of the report (add columns, drop columns, etc.) it has to be done in the beforeFactory/OnPrepare method. At this time you can get access to the DesignElementHandle objects that make up the report design and modify appropriately. The problem is that the report does not have any access to the data at this time. The DataSources and DataSets have not been run at this time.

In theory, one could get a handle to the DataSource / DataSets and manually instantiate and run them, but it's not pretty. You would need to write a lot of code. I have entered a bugzilla entry to rectify this situation, but I am not sure when or if that will be a feature we can add (volunteers? anyone? anyone?).

So how do you get access to data in the beforeFactory/OnPrepare method? Well it turns out that it is relatively easy to use the Report Engine API to access parameters through the ParameterDefinitionTask. Parameters in turn can be tied to a DataSet. So it seems like we have a route to access data from the OnPrepare method.

To demonstrate, I will create a simple report that combines both the REAPI and DEAPI to dynamicaly create a column for each type of order status and in that column will be a label for the column type. As stated before, the source code for this example can be found on my subversion server at The projects that you will need are:
- birt_api_example
- birt_runtime_lib_222
- script.lib

The first thing we are going to do is to build a simple report that gets all of the orders and separates them by the order status. If you create a data source that uses classic models and add a data set named setOrders with the following SQL.
select status from orders

Drag the data set onto the report layout and you will have a table with all of the statuses. Now it is trivial to create a report that summarizes the count for each status as a row. What we would like to do instead is have the statuses be columns for this report. This means that we want to take our one column report at design time, and make it an N columns at runtime. Since we are not concerned with the detail columns, you can delete the detail row from the table for now.

This leaves you with a report with only one column and one cell that holds a label which says Status. I like to start simple.

To make this happen we need to have a different data set that returns all of the unique statuses, name the data set setStatus and make the sql:

select distinct status from orders

As stated before, the easiest way to get to this dataset's values in the OnPrepare method is to wrap the DataSet inside a parameter. So create a new parameter named pStatus. Make it a combo and add setup the dataSet to be the setStatus that we just defined. For the value field use row[0] which is a standard field that will return the row number for each row. Make the Display field equal to row["status"] as shown in the accompanying screen shot.

Once you have the parameter setup, you are ready to start writing your event handler. If you want to test whether you have the parameter setup correctly, run the report and click on the Show Report Parameters button at the top of the preview panel. It should show you seven order statuses.

Now we need to add an event handler to the report. We could add the event handler in a number of locations. We could add it to the label, the cell which hold the label, the column, or the table. It makes sense to add the event handler to the table for a couple or reasons. The most basic reason is that we may want to support more than one dynamic column within this table.

To create a table event handler, create a new java class that extends the TableEventAdapter. We want to create this class in the script.lib project so that we can debug the code as we produce it.

package deapi_event_handler;


public class DynamicTable extends TableEventAdapter {


If you right click in the java class and select Source -> Override/Implement Methods and then select the OnPrepare method in the dialog you will have the method that we need to override. The first thing we want to do is to get the values that have been exposed through our parameter. I have created a helper class ParameterUtil that has methods to get either a list of values using the ParameterDefinitionTask.

public static Collection<IParameterSelectionChoice> getParameterValues(String parameterName, IReportContext reportContext) {
IGetParameterDefinitionTask task = null;
HashMap curParams = new HashMap();
IReportRunnable runnable = reportContext.getReportRunnable();
try {
task = runnable.getReportEngine().createGetParameterDefinitionTask(runnable);

// get the names of all the parameters
Collection paramRefs = task.getParameterDefns(false);

// for each parameter name, get the parameter value
// add the name and value to a hashmap
for (Iterator iterator = paramRefs.iterator(); iterator.hasNext();) {

ParameterDefn pDefn = (ParameterDefn);
String name = pDefn.getName();
Object curP = reportContext.getParameterValue(name);

curParams.put(name, curP);

// set the parameter values for this task from the hashmap.

// get the parameter that is tied to this table.
IParameterDefnBase scalar = task.getParameterDefn(parameterName);
if (scalar instanceof IScalarParameterDefn) {

// bind the parameters to the query text

// get the values for this parameter from its DataSet
Collection<IParameterSelectionChoice> paramChoices = (Collection<IParameterSelectionChoice>) task

return paramChoices;

} catch (Exception e) {
System.out.println("Failure to get parameters");
} finally {
return null;

As you can see, this method takes a string which names the parameter we are going to use and the report context and returns the values/labels exposed by that parameter. So if we want to use that method we have to figure out which parameter holds the values that we want to show as columns.

To do this, I am going to add a NamedExpression to my table. Once I have defined a NamedExpression on my table, I can look up the name of that expression from the script and use that in my ParameterUtil method. To add a NamedExpression to a table, to to the PropertyEditor for the table and select the NamedExpression tab.

In addition to creating the named expression, you will need to specify that this table will use the Event Handler we created in a previous step.

So now I will modify my OnPrepare method to first get the named parameter value and then to get the values defined by that expression.

public void onPrepare(ITable tableScript, IReportContext reportContext) {

String paramName = tableScript.getNamedExpression("dyn_parameter");

Collection<IParameterSelectionChoice> colNames = ParameterUtil.getParameterValues(paramName, reportContext);

for (IParameterSelectionChoice paramChoice : colNames) {
System.out.println(paramChoice.getValue() + ": " + paramChoice.getLabel());


At this point, you should be able to run the report using the tester. If all goes well, you should see the following in your console.

home: C:\BIRT\ws_ec2008\birt_api_example\Reports
1 property file(s) found
output: C:\BIRT\ws_ec2008\birt_api_example\tester_output_20080425_171337\data_driven.
Executing tester_run\
Start run
0: Disputed
1: On Hold
2: In Process
3: Shipped
4: Cancelled
5: Resolved
C:\BIRT\ws_ec2008\birt_api_example\tester_output_20080425_171337\data_driven.: built. Start render.
Run duration = 2531 ms
Render duration = 641ms
Success 1 : tester_run\
Finished Fri Apr 25 17:13:45 CDT 2008

So what have we accomplished? So far, we have a simple report that is able to dynamically get a list of values from a query defined within the report at runtime, in the onPrepare method. Now that we have this working, we can start on the next step which is to use those values to dynamically create new columns in the report.

Next week, I will examine how you can use the Design Engine API to do just that for both labels and data items.

Wednesday, April 23, 2008

BIRT Resizing Charts

Note: I have added the reports that are used in this post to the subversion repository
the project is birt_api_example
and the reports can be found in /Reports/charts
Alternatively you can just go to:
navigate into the charts and cut and paste into your report designs

When creating charts in BIRT the size is set automatically. The chart can be resized with the mouse after the chart is in the report as well. This is fine if you know before hand the amount of data your chart is going to display. Currently there are several bugzilla entries tracking chart resize features, but I thought I would offer a work around until those entries are completed. Presented below are two options for resizing the chart. Using the simple chart API, and using the afterDataSetFilled chart event to modify the size of the chart based on the data retrieved.

One way of resizing the chart is to use the chart simple API within the beforeFactory event. The simple api allows simple properties of the chart to be changed before the report is executed.

Some examples include setting titles, chart dimensions, and output formats:

var chart1 = this.getReportElement( "Chart1" )
var chart2 = this.getReportElement( "Chart2" )
var chart3 = this.getReportElement( "Chart3" )
var chart4 = this.getReportElement( "Chart4" )

var color1 = chart1.getTitle().getCaption().getColor()

chart1.setColorByCategory( true );
chart1.getTitle().getCaption().setValue( "Color by Category" );
chart1.setColorByCategory( true );
color1.setRed( 255 );
color1.setGreen( 0 );
color1.setBlue( 0 );
chart1.getTitle().getCaption().setColor( color1 );

chart2.setDimension( "ThreeDimensional" );
chart3.getCategory().setSorting( "Descending" );

In this example the charts must be named. Select the general properties for the chart and in the name field enter a value.

The size of the chart can be set using the following code in the before factory.

var mychart = this.getReportElement( "mychart1" );
if( reportContext.getOutputFormat() == "pdf" ){


In this example we resize the chart based on output format.

If you want the chart to resize based on the data populated in the chart data sets it requires a little bit of a work around. First, name the chart as with the simple chart API and enter the following code in your beforeFactory event handler.

var mychart = this.getReportElement( "mychart" );

After you have done that, enter code similar to the following in your afterDataSetFilled chart event handler.

function afterDataSetFilled(series, dataSet, icsc)
if( series.getSeriesIdentifier() == "seriesone" ){
if( dataSet.getValues().length > 4 ){


In this example we have named our series “seriesone” on the third tab of the chart wizard.

If the category values contain more than four entries we double the size of the chart. The sizing could be more dynamic, for example we could use the list size to increment a delta per category as well.

This example works with BIRT 2.2.2, but will not work with the chart output type set to SVG, when rendering to HTML.

For Birt 2.3, everything can be done in the beforeGeneration script and it does work with SVG. No other scripts are needed.

function beforeGeneration( chart, icsc )
xAxis = chart.getBaseAxes()[0];
yAxis = chart.getOrthogonalAxes( xAxis, true )[0];
seriesDef = yAxis.getSeriesDefinitions().get(0);
runSeries = seriesDef.getRunTimeSeries()[0];
//Retrieve list of data values
list = runSeries.getDataSet().getValues();
if( list.length > 3){

Friday, April 18, 2008

Return To Eureka

Two weeks ago I had a Eureka moment. In that post I described how I was able to use reflection to get the DesignElementHandle from an ICell object. It was a fairly simple solution since there was only one super class to the ICell.

Yesterday, I started to re-factor my other example classes to use this helper utility and I figured out that the helper utility need a little more work. It turns out that the ITable class is a bit more complicated then the ICell class. For the ITable object there are five levels super classes to get back to the DesignElement class.

So the ITable object actually resolves to:

  • class
The hierarchy for this class is:
  • class
  • class
  • class
  • class
When we use reflection to find the protected field we have to navigate all the way back to DesignElement which can be n levels up the hierarchy. So we need to use a loop to search for the parent.

Once we have the DesignElement we can use reflection to get access to the protected field for the simple api DesignElement. The simple api has the same hiearchy issue, with one additional super-class.
  • class
  • class
  • class
  • class
  • class
  • class
Fortunately, the solution is not that difficult. The addition of one relatively simple loop allows us to find the parent classes we need.

private static final String DESIGN_ELEMENT_NAME = ".DesignElement";

private static Class findDesignElementParent(Class childClass) throws NoSuchFieldException {
do {
childClass = childClass.getSuperclass();
if (childClass == null)
throw new NoSuchFieldException("Did not find DesignElement");
} while (!childClass.toString().endsWith(DESIGN_ELEMENT_NAME));
return childClass;

You may ask scratch your head and ask why am I searching for the class using the name of the class as opposed to a reference to the actual class name. This goes back to the issue that the classes that we are referencing are not actually exposed through the runtime interface.

In any case, once I have made these changes my utility method is still relatively simple.

public static DesignElementHandle getDesignElementFromScript(IDesignElement scriptObj)
throws IllegalAccessException, SecurityException, NoSuchFieldException {

Class scriptParentClass = findDesignElementParent(scriptObj.getClass());
Field fieldFromScript = scriptParentClass.getDeclaredField("designElementImpl");
if (fieldFromScript == null) {
return null;

// Access the DesignElement field from the Script object
Object designElement = fieldFromScript.get(scriptObj);

// The DesignElement object has a field named "handle"
// which is the DesignElementHandle for that DesignElement
// Use reflection again to get this field
Class simpleApiParentClass = findDesignElementParent(designElement.getClass());
Field fieldFromSimpleApi = simpleApiParentClass.getDeclaredField("handle");
if (fieldFromSimpleApi == null) {
return null;

// Now access the DesignElementHandle from the DesignElement
return (DesignElementHandle) fieldFromSimpleApi.get(designElement);

The complete source is available in the script.lib project on the subversion project. I have moved the code into the ScriptUtil class.

There is an underlying question here. You may ask why do I have to go through those gyrations to get a handle to the object? Shouldn't the API just support this method. That's a good question. The issue is that working with the DesignElementHandle object directly is a relatively advanced feature.

The BIRT script objects are meant to be a report developer interface and have been designed to protect the report developer from obscure and hard to debug conditions. When you start working with the ElementHandles there is very little that is protecting the developer from doing something they don't want to do.

For example, in my Eureka example I demonstrated changing the column span for a cell. What I didn't mention is that for the example in question, the change I made was always going to be appropriate. In reality when you change the column span on one cell in the row, you have to account for the total number of cells. This means that you have to:
  • Modify the column span of another cell in the row
  • Drop cells from the row
  • Add cells to the row
These operations are not difficult, but they do require a level of sophistication and thought that many beginning report developers are not ready to take on. So is there a way to expose methods, but somehow warn that these methods are for power users only? Is it better to just not expose the power methods and use a utility class such as the one I have demonstrated?

I have opened a bugzilla item to expose the DesignElementHandle through the API. I would like to hear what other people think of this idea. How do you expose the power interface while still protecting users that don't want or need access to objects that powerful?

Thursday, April 17, 2008

Testing and Debug of Reports

At EclipseCon this year I did a two hours tutorial on BIRT integration. I think it went fairly well. I went a little long, but covered all the best bits. I would like to do a series of posts on this site that go over some of the same material.

The stuff I really want to talk about is a technique that I have developed for debugging / stepping through your Java Event Handler classes. My method requires a fair amount of setup, but in the end it makes it far easier to step through your event handlers. In version 2.3 of the product, this process is simplified. That said many people are on version 2.2 and will continue to be on version 2.2 for a while.

So if you intend to do any Java event handler development in BIRT 2.2. this is the post for you.


The first thing that we need to do is to understand the report test framework that I use on BIRT projects. This is a really simple format that allows me to batch run a large number of pre-defined BIRT reports. I am not going to dive into the details of how all of this works, but you will have all of the source code, and it is relatively simple to figure out.

All of the code for these examples is stored on my subversion server at:

You will find a lot of projects there, they are all available under EPL. The ones that you will need to follow these examples are:

First a couple of words about the projects. If you take a look at these projects you will notice that they are plug-in projects (look under builders). I use the plug-in mechanism to associate the projects together. You do not need to do this, but I have found that it allows people to download them into their workspace and they will just run.

The birt_api_example project contains the majority of the code that you will work with. The REAPI package contains code showing how to run reports. The DEAPI package contains code showing how to create report designs through code. The Reports folder contains report designs that are used by both the REAPI and DEAPI. In addition, Reports contains reports that use the DEAPI.

The birt_runtime_lib_222 project contains the BIRT runtime libraries. It also acts as BIRT_HOME for both the REAPI and DEAPI examples. The runtime lib takes all of the libs and plug-ins from the and exposes them as a plug-in project. I will return to the what exactly the runtime_lib does in a later post. First, I want to get to the fun stuff.

The final project is the script.lib project which contains java event handler code that you want to run or debug. If all goes well, I will tie all of this back to that project at the end of this post.

So lets start with the birt_api_example project. In large BIRT report development projects, you often have multiple reports that need to be tested under a variety of conditions. Typically, we build a relatively small number of rptdesigns that can create a large number of rptdocuments based on the parameters used when the report runs.

This technique is great since it dramatically reduces the amount of code that needs to be maintained. The problem is that one rptdesign may have a large number of test conditions that need to be run. Manually running each condition by entering the required parameters is going to be difficult at best.

What we wanted was a relatively simple solution where we could store report run configurations and then run them in batch. This is the role of the tester package. There is a single java class which can be run as a java application.

When you run the TestAllReports project the program will look in the tester_run folder for any .properties files. For each properties file the tester will try to run according to the configuration information in the file. The format of the properties file is very simple.

file_name=customers.rptdesign (relative path from ./Reports directory)

Currently I have added support for Strings and java.sql.Date parameter data types. It is trivial to modify to support Integers or Booleans if you require these data types.

Once the tester has opened your properties file it will run the report design using Run then Render tasks to build PDF files. You can modify the code in the testReport method to enable output to HTML, Excel, or Word formats (the code is there but commented out). Reports are run into a new folder that will have the format:


Using the tester you can batch test large number of reports. The tester does not have a way to automate the comparison of rendered files, but there are a number of tools that can be used if your processes require this level of automation.

Debugging Java Event Handlers

Now that we have the tester running, it is time to go ahead and look at stepping through event handler code. The 'official' technique for debugging BIRT event handlers is to create a runtime workspace that runs your reports as defined here. This process is relatively straight-forward, but there is one problem, MemoryWoes. I have a two gig laptop and running MaxPermSize I still run into memory issues when running the Runtime workbench.

What I wanted was a method where I could run my reports through the debugger, without having to load the runtime workbench. Well it turns out that I can use my tester program to do just that.

The starting place for this is the script.lib project. This project is where I am going to place all of my event handlers. I then have an ant build file that I can use to compile my event handlers into a jar file. Back in the tester project, when it loads up the EngineConfig, I add that jar file:

    EngineConfig config = new EngineConfig();
config.setProperty(EngineConstants.WEBAPP_CLASSPATH_KEY, getScriptLibFileNames());


* The engine needs to see a list of each jar file concatenated as a string
* using the standard file system separator to divide the files
private static String getScriptLibFileNames() {
File scriptHome = new File(SCRIPT_LIB);

if (!scriptHome.exists()) {

File[] dirFile = scriptHome.listFiles(new JarFilter());
StringBuffer scriptlibClassPath = new StringBuffer(); //$NON-NLS-1$
for (int i = 0; i &lt; dirFile.length; i++) {
if (scriptlibClassPath.length() > 0) {

return scriptlibClassPath.toString();


Now, I know what you are thinking. Why did you have to add the file as a jar file, why not just add the folder to the classpath. The short answer is that I don't know how to add folder structures to the classpath, and I didn't bother to learn. This is not due to laziness, my thought is that when I deploy to my application, I am going to deploy the jar file. By using the jar when I run the tester, I have a better test environment.

Testing It Out

Now that you have an explanation of what we are doing, lets test it out. If you go back to the birt_api_example project and look in the tester_save folder you will find a file. Copy this file to the tester_run folder. Examining the file you will see that I am running the report found in:


Opening this design directly and run it and you will see that a three column table generates a report with eight columns. How does that happen? If you look on the properties for the table, you will see that I am using an event handler named TableMagic, which is in the script.lib project. (I have pictures coming, but blogger is having issues with images right now)

Now put a breakpoint in the class. (try using from the Java perspective). If you have copied the file to the tester_run folder, and you execute the tester in debug mode, it will take you directly to the break point.

How cool is that? You are now debugging your java classes without launching a runtime workspace. The best part is using this type of debugging, I can walk you through the DEAPI calls that I have made to dynamically modify my report based on data from the database.

Now I feel like Howie Mandel on Deal or No Deal. Just when I get to the good stuff, I am going to have to take a break. Next week, I will go through the TableMagic event handler to discuss how you can use the DEAPI to create data driven report designs.

Monday, April 14, 2008

BIRT 2.3 Milestone 6 New and Notable Features.

BIRT 2.3 Milestone 6 has been released and contains improvements to scripting, the addition of the DTP graphical Query Builder prototype, cross tab and chart improvements, and a couple new extension points. Be sure to try out the Query Builder. The DTP team is looking for feedback. Cross tabs can now be modified using script and marker script events are available for charts.

To read more about these features and others available in the 2.3 M6 release, go here.

Friday, April 04, 2008


I wanted to share a little DEAPI related discovery. If I wait to type it up the 'right' way I may not ever get it done, so here it is. I have been doing a lot of work with the DEAPI inside of a report.

If you have not worked with the DEAPI inside of a report, I should remind you that using the DEAPI is only valid within the OnPrepare method of the reportItems or from the BeforeFactory method of the ReportDesign.

When you work with the DEAPI you need to be aware of three basic types of objects in the report engine. Event handlers are passed objects that are part of the script api package. Script objects have limited functionality. For instance, an ICell script object can tell how many cells the column is spanning cell.getColumnsSpan(), but does not allow you to set the column span.

The type of object you want to work with is the DesignElementHandle, which for a cell is CellHandle. DesignElementHandles are defined in the Model API and give you access to all of the functionality in the model. Once you have the DesignElementHandle, you can change the report design in almost any manner.

In between the DesignElementHandles and the script objects are the DesignElements. The DesignElements objects represent the actual instance of the object in the report. DesignElement objects are not exposed by the report engine, and are not particularly useful except for one key role, they provide a link between the script objects and the DesignElementHandles.

If a report were a house: the DesignElementHandles are the blue prints, the DesignElements are the house itself and the script objects are pictures of the house. If you modify the blue print (DesignElementHandle) early in the build process, you can completely change the way the report is built. To stretch the metaphor you are looking at a picture of someone else house (script object) and using that to change the blue prints (DesignElementHandle) for the house that you are building.

Clear as mud? Good...

I have always navigated to the DesignElementHandle by using the name value of the script object.

   DesignElementHandle designHandle = rptContext.getReportRunnable().getDesignHandle();
ModuleHandle moduleHandle = designHandle.getModuleHandle();
TableHandle tableHandle = (TableHandle) moduleHandle.findElement(inTable.getName());
//modify the table handle as I want

Some script objects (cells, rows, columns) do not support the name value, in that case I can use the getParent() method to navigate to a container which supports the name value. Typically I end up back at the table.

From the tableHandle you can then navigate to the appropriate, row, column, group within the report. This works okay, but it can be tricky. In some cases, if you are looking to modify one particular cell it is down right hard. What I wanted to be able to do is get the CellHandle directly from the scriptable object that that is passed to the event handler. You end up having to write a bunch of code to get to navigate back to the object where you started.

It turns out that the ScriptObject has a protected field named designElementImpl which is a handle to the DesignElement object. The DesignElement object has a field "handle" which is the DesignElementHandle. The problem is that neither of the fields are exposed through the API.

Don't you hate it when you know something is there but you just can't figure out how to access it?

I finally got tired of writing all kinds of code to navigate back to an object. It turns out you can use reflection to get access to the fields that are there, but not exposed through the API.

I have created a simple generic method that will get the associated DesignElementHandle from the script object. This is fairly fresh, and has not been fully wrung out for all issues, but it is a decent place to start.

* Modify the columnSpan of a cell item
public void onPrepare(ICell cell, IReportContext reportContext) {

Integer unitSpan = reportContext.getGlobalVariable("cell_span");

try {
// call generic method to get DesignElementHandle
DesignElementHandle cellHdl = getDesignElementFromScript(cell);
if (cellHdl instanceof CellHandle) {
((CellHandle) cellHdl).setColumnSpan(unitSpan);
} catch (Exception e) {
// trap errors

* Generic method to access DesignElementHandle from a scriptable object
* @param scriptObject
* @return
* @throws NoSuchFieldException
* @throws IllegalAccessException
public DesignElementHandle getDesignElementFromScript(IDesignElement scriptObj)
throws NoSuchFieldException, IllegalAccessException {

// The designElementImpl field is a protected field
// that is declared on the super class
Field fieldFromScript = scriptObj.getClass().getSuperclass()
if (fieldFromScript == null){
return null;

// Access the DesignElement field from the Script object
Object designElement = fieldFromScript.get(scriptObj);

// The DesignElement object has a field named "handle"
// which is the DesignElementHandle for that DesignElement
// Use reflection again to get this field
Field fieldFromSimpleApi = designElement.getClass().getSuperclass()
if (fieldFromSimpleApi == null){
return null;

// Now access the DesignElementHandle from the DesignElement
return (DesignElementHandle)fieldFromSimpleApi.get(designElement);

Wednesday, April 02, 2008

BIRT and Ecore!

As many of you are aware BIRT uses the Data Tools Platform (DTP) project’s Open Data Access (ODA) standard to access data. Currently BIRT supports the following data sources:

Flat File
Web Services
Scripted Data Set

In addition, BIRT provides two project wizards for creating new ODA drivers.

The ability to query Ecore models, for use within BIRT reports, has frequently been requested. Jeff Ramsdale and Tim Myer have recently contributed an ODA driver to the DTP project that queries the Ecore model using the Object Constraint Language (OCL). To read more about this driver take a look at the proposal.
Be sure to drop by the DTP newsgroup to comment on or discuss this new ODA.