Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • Demo application for Domino JNA Views to play with

    Karsten Lehmann  18 July 2024 17:50:24
    Today I attended the "Developer Variety Hour" webinar of OpenNTF and did a demo of the new Virtual View API of Domino JNA.

    I built a small web application that combines a Sunburst diagram (taken from the D3 website, enhanced by a training session with Anthropics Claude AI chat :-) ) with a categorized view, built on top of a Bootstrap 5 table.

    The application looks like this:

    Image:Demo application for Domino JNA Views to play with

    and it's available for you to play with here: https://www.mindoo.de/test/jnavirtualviews.nsf/sunburst.xsp

    Both components fetch their data from an XAgent style XPage where a Virtual View is used to read document information from two fakenames databases:

    public Optional< VirtualView > getViewById(String id) {
            VirtualView view = null;
                   
            if ("fakenames_bycontinent_categorized".equals(id)) {
                           
                    view = VirtualViewFactory.INSTANCE.createViewOnce(id,
                                    17, // view version number
                                    10, TimeUnit.MINUTES, // keep in memory for 10 minutes
                                    (viewId) -> {

                            return VirtualViewFactory.createView(
                                                   
                                            // use a Java function to compute the "Continent" category:
                                                   
                                            new VirtualViewColumn("Continent", "$1", Category.YES, Hidden.NO, ColumnSort.ASCENDING, Total.NONE,
                                                            new VirtualViewColumnValueFunction< String >(1) {
                                                           
                                                    @Override
                                                    public String getValue(String origin, String itemName,
                                                            INoteSummary columnValues) {

                                                            String companyName = columnValues.getAsString("CompanyName", "");
                                                                   
                                                            Map lookupMap = getCompanyContinentMap();
                                                            return lookupMap.getOrDefault(companyName, "Unknown");
                                                    }                                                        
                                            }),

                                            // use formula language for the remaining columns:
                                                   
                                            new VirtualViewColumn("Company Name", "$2", Category.YES, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "@Left(CompanyName;1) + \"\\\\\" + CompanyName"),

                                            new VirtualViewColumn("Lastname", "Lastname", Category.NO, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "Lastname"),

                                            new VirtualViewColumn("Firstname", "Firstname", Category.NO, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "Firstname"),

                                            new VirtualViewColumn("CompanyName", "CompanyName", Category.NO,
                                                    Hidden.YES, ColumnSort.NONE, Total.NONE,
                                                    "CompanyName")

                                            )
                                            // load data from two fakenames databases
                                            .withDbSearch("fakenames1", "", "fakenames.nsf", "Form=\"Person\"")
                                            .withDbSearch("fakenames2", "", "fakenames2.nsf", "Form=\"Person\"")
                                            .build();
                    });
            }

            return Optional.ofNullable(view);
    }

    Initial index generation for about 40.000 fakenames docs takes 5 seconds on our server. The view is set to expire after 10 minutes and will be automatically be rebuilr.

    The first column value is computed via Java code and looks up the continent for the company name of a fakenames person (let's call it a JOIN operation).

    Since all data access is done in-memory (no legacy Domino view is required), REST API response times are ultra-fast and take 50ms for each request. Part of those 50ms is opening the two fakenames database and checking if anything relevant has been changed/added/deleted. As presented in the webinar, changing DB documents is immediately reflected in the web app.

    Building this demo app this week I discovered a few issues in the Virtual View API of Domino JNA 0.9.52 especially when processing NSF changes (the code produced duplicate rows).

    I will publish a new Domino JNA version this week with fixes.



    Comments

    1Fredrik Norling  22.07.2024 8:20:06  Demo application for Domino JNA Views to play with

    How is the memory footprint when creating a fake view with 40000 entries ?

    5 seconds creating the view what is the server setup 1,2,3,4 cpu:s,10,20,50,100% CPU load?

    2Karsten Lehmann  22.07.2024 15:19:45  Demo application for Domino JNA Views to play with

    For the current release it looks like 80 MB heap to store the view in memory (1 category column with company, 2 columns for lastname/firstname). Just checked it in JProfiler. More columns means more memory. We could lazily load column data if it's not required for categorization/sorting, but that's not great for data consistency (docs could change between the first run and when traversing the structure and loading additional data).

    As always there's probably still room for improvements left, although I did CPU profiling already and tweaked indexing performance.

    Our test server is from Contabo, VPS L SSD:

    Intel Xeon CPU E5-2630 v4 2.20 GHz, 8 Cores, 30 GB RAM, 800 GB SSD drive.

    Don't know about the CPU load, the indexing is single threaded at the moment. Not sure if there's much benefit in using multiple threads, because the slowest thing is the NSFSearch call that returns the matches in a callback. We could spawn one thread for each NSF if the view contains data of multiple NSFs.