XPages series #12: XAgents and performance bottlenecks
Karsten Lehmann 17 July 2011 10:18:55
XAgent is a term that describes the equivalent of a classic Notes web agent in XPages technology: an XPage is called via URL and produces any kind of data (e.g. HTML, dynamic images, data in JSON format or ODF documents) by sending Strings or even raw bytes directly to the browser.Chris Toohey and Stephan Wissel have already blogged about this topic a few years ago and discussed some use cases:
IBM Lotus Notes Domino REST Web Services via XPage XAgents
Web Agents XPages style
How to write an XAgent
To write an XAgent, you basically need to do two things: first you add the attribute
rendered="false" to the xp:view tag of an XPage to prevent the XPages engine from rendering any output. Second, you need to write the code that produces the data and add it to the beforeRenderResponse event of an XPage. Here are two simple example XAgents. The first one writes XML content to the writer object of the servlet response (which is used to return character data):
<
?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" rendered="false">
<xp:this.beforeRenderResponse
>><![CDATA[#{javascript:try {
var uName = session.createName(session.getEffectiveUserName());
var exCon = facesContext.getExternalContext();
var response = exCon.getResponse();
var writer = response.getWriter();
response.setContentType("text/xml");
response.setHeader("Cache-Control", "no-cache");
writer.write("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
writer.write("<test>Hello "+uName.getAbbreviated()+"</test>");
facesContext.responseComplete();
}
catch (e) {
_dump(e);
}}]]></xp:this.beforeRenderResponse></xp:view> The second example demonstrates how you can load a file resource from the database design and write it to the output stream object of the servlet response:
<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" rendered="false">
<xp:this.beforeRenderResponse><![CDATA[#{javascript:try {
var exCon = facesContext.getExternalContext();
var response = exCon.getResponse();
var outStream = response.getOutputStream();
//load a file resource from db design and write it to the output stream
var cl=com.ibm.domino.xsp.module.nsf.NotesContext.getCurrent().getModule().getModuleClassLoader();
var buf=new byte[16384];
var len;
var filePath="/test/document.docx";
var inStream=cl.getResourceAsStream(filePath);
if (inStream==null) {
var errMsg="File "+filePath+" not found";
_dump(errMsg);
response.sendError(404, errMsg);
}
else {
//set mime type according to file type
response.setContentType("application/vnd.openxmlformats-officedocument.wordprocessingml.document");
response.setHeader("Cache-Control", "no-cache");
//send http header with filename (needed so that the browser does not use the XPage's name as filename)
response.setHeader("Content-Disposition", 'attachment; filename="document.docx"');
//read bytes from inStream into the buffer
while ((len=inStream.read(buf))>-1) {
//and write the data to the output stream
outStream.write(buf, 0, len);
}
inStream.close();
outStream.close();
}
facesContext.responseComplete();
}
catch (e) {
_dump(e);
}}]]></xp:this.beforeRenderResponse></xp:view> The loaded file resource must be part of the NSF project's build path to make the XPage code find it (that's where the module classloader is looking for files).
Please refer to part two of this XPages series to get more information about how to add a folder to the build path.
If you just want to offer a file for download, developing an XAgent to stream the data would probably be the wrong solution.
In that case, you should add your files to the database design in the Java Perspective of Domino Designer instead and reference them directly in a URL. This is something that is used in the OpenNTF project XPages Mobile Controls to add a newer Dojo version (e.g. 1.6 with advanced mobile device support) to a Domino server than the default one (e.g. version 1.4.3 for Lotus Domino 8.5.2).
But the second code example from above could of course be modified to do something more useful, for example to replace placeholders in the file, create a zip file from multiple files on the fly using Java's ZipOutputStream or render a captcha image for spam protection.
Sounds great?
Well, before you start thinking about building large applications based on this approach, you should read on to get to know about it's limitations.
The limitations
What I'm going to show you is not only a limitation of XAgents, but of XPages technology in general. For XPages producing the user interface of an application, this limitation may not be critical, but it can become a show stopper for using them the XAgent way.
I've created a small test application with three XPages: Start.xsp, Frame1.xsp and Frame2.xsp (download link). The first one (Start.xsp) contains two iframes and a button. By clicking on the button, the other two XPages are loaded as iframe contents:
var now=new Date();
var nowTime=now.getTime();
var iframe1=document.getElementById("iframe1");
var iframe2=document.getElementById("iframe2");
iframe1.src=document.location.pathname+"/Frame1.xsp?startTime="+nowTime;
iframe2.src=document.location.pathname+"/Frame2.xsp?startTime="+nowTime; Frame1.xsp contains beforeRenderResponse event code that delays the rendering for 5000 milliseconds (
java.lang.Thread.sleep(5000)), while Frame2.xsp waits for 2500 milliseconds. You would expect that both XPages are opened and rendered concurrently. Because of the delay, Frame2.xsp should be visible first, then Frame1.xsp.
Unfortunately, the output looks like this instead (click on the image to open the test application):
Frame1.xsp is displayed first, followed by Frame2.xsp. Frame2.xsp takes about 7600 milliseconds to be rendered, which is the sum of both delays, because the XPages engine does not render multiple XPages in an application at the same time for a single user.
It was a real surprise when we noticed this effect in a customer project. Our UI was based on the Ext JS toolkit which is heavily using REST services to read the UI component data. Our plan was to use XAgents for those REST services, but we quickly noticed that all UI elements only received data one at a time, which was a no-go for the application UI performance. Nothing was loading in parallel.
There are technical reasons for such a restriction, as Philippe Riand, XPages Chief Architect at IBM, explained to me in April 2011.
As you may know, the XPages runtime is based on JSF technology, which uses a tree of components on the server side that represents the structure of a web page and its current state. The tree is created/restored when a HTTP request comes in and saved/discarded after it is completed.
Unfortunately, this tree does not support being accessed by two concurrent threads, which would break many things like data sources and component states.
To prevent this to happen, XPages synchronizes on the user session object (HttpSession), so that only one thread can access the tree at a time. But by synchronizing on the user session (and not just on the component tree instance), the XPages dev crew also prevented any other XPage in an application from running (the Domino server seems to assign session objects on a per database basis).
The consequence is that you should think twice about your XPages application architecture, if you have many concurrent HTTP requests or if some of them take a lot of time to be processed. An XAgent may be the easiest solution to deploy, but may not produce the best user experience in all cases.
Workarounds and solution
There is some hope that the situation will improve in Domino 8.5.3.
As Philippe said, the next XPages engine will be smarter in synchronization and synchronize less than previously.
In addition, there will be a new database property called
xsp.session.transient. This flag means that unique session objects will be created per request to the server and discarded right after the request ended. This is a first attempt to provide session less mode. If you use this option, then you can create one database with all the services and no synchronization will happen, as each request will have its own session object.
I have requested a more granular option so that you can activate session less mode for single XPages in an application, but I'm not sure if this will make 8.5.3.
For Domino 8.5.2, I can see three things that you can do if you can't live with this performance issue:
1. Move XPages that run for a long time to separate databases
This does not help a lot, but if you have an XPages application with an XPage that needs some time to process, you could move it to different database. In that case, the rest of the application would still be responsive, although your long running XPage for example produces a 100 MB log output in XML format that you want to download.
2. Write your own servlet in Java
By writing your own servlet as an Eclipse plugin running in Domino 8.5.2's OSGi framework, your code can run without any performance bottleneck caused by synchronization. See this article in the OpenNTF blog for implementation details.
3. There is an app for that :-)
As you may know, we have developed XPages2Eclipse, an extensive toolkit for XPages development, which is currently in beta-testing (June 2011). The articles Building servlets in JavaScript and Building servlets in Java demonstrate how you can write your own servlet implementation in JavaScript or Java language.
This solution reads all your code from the database design, so that you don't have to deploy every servlet individually. Just install the toolkit on a Domino server once, then you can create any number of servlets by writing code in Domino Designer and replicate them to the server.
- Comments [14]