Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • Demo application for Domino JNA Views to play with

    Karsten Lehmann  18 July 2024 17:50:24
    Today I attended the "Developer Variety Hour" webinar of OpenNTF and did a demo of the new Virtual View API of Domino JNA.

    I built a small web application that combines a Sunburst diagram (taken from the D3 website, enhanced by a training session with Anthropics Claude AI chat :-) ) with a categorized view, built on top of a Bootstrap 5 table.

    The application looks like this:

    Image:Demo application for Domino JNA Views to play with

    and it's available for you to play with here: https://www.mindoo.de/test/jnavirtualviews.nsf/sunburst.xsp

    Both components fetch their data from an XAgent style XPage where a Virtual View is used to read document information from two fakenames databases:

    public Optional< VirtualView > getViewById(String id) {
            VirtualView view = null;
                   
            if ("fakenames_bycontinent_categorized".equals(id)) {
                           
                    view = VirtualViewFactory.INSTANCE.createViewOnce(id,
                                    17, // view version number
                                    10, TimeUnit.MINUTES, // keep in memory for 10 minutes
                                    (viewId) -> {

                            return VirtualViewFactory.createView(
                                                   
                                            // use a Java function to compute the "Continent" category:
                                                   
                                            new VirtualViewColumn("Continent", "$1", Category.YES, Hidden.NO, ColumnSort.ASCENDING, Total.NONE,
                                                            new VirtualViewColumnValueFunction< String >(1) {
                                                           
                                                    @Override
                                                    public String getValue(String origin, String itemName,
                                                            INoteSummary columnValues) {

                                                            String companyName = columnValues.getAsString("CompanyName", "");
                                                                   
                                                            Map lookupMap = getCompanyContinentMap();
                                                            return lookupMap.getOrDefault(companyName, "Unknown");
                                                    }                                                        
                                            }),

                                            // use formula language for the remaining columns:
                                                   
                                            new VirtualViewColumn("Company Name", "$2", Category.YES, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "@Left(CompanyName;1) + \"\\\\\" + CompanyName"),

                                            new VirtualViewColumn("Lastname", "Lastname", Category.NO, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "Lastname"),

                                            new VirtualViewColumn("Firstname", "Firstname", Category.NO, Hidden.NO,
                                                    ColumnSort.ASCENDING, Total.NONE,
                                                    "Firstname"),

                                            new VirtualViewColumn("CompanyName", "CompanyName", Category.NO,
                                                    Hidden.YES, ColumnSort.NONE, Total.NONE,
                                                    "CompanyName")

                                            )
                                            // load data from two fakenames databases
                                            .withDbSearch("fakenames1", "", "fakenames.nsf", "Form=\"Person\"")
                                            .withDbSearch("fakenames2", "", "fakenames2.nsf", "Form=\"Person\"")
                                            .build();
                    });
            }

            return Optional.ofNullable(view);
    }

    Initial index generation for about 40.000 fakenames docs takes 5 seconds on our server. The view is set to expire after 10 minutes and will be automatically be rebuilr.

    The first column value is computed via Java code and looks up the continent for the company name of a fakenames person (let's call it a JOIN operation).

    Since all data access is done in-memory (no legacy Domino view is required), REST API response times are ultra-fast and take 50ms for each request. Part of those 50ms is opening the two fakenames database and checking if anything relevant has been changed/added/deleted. As presented in the webinar, changing DB documents is immediately reflected in the web app.

    Building this demo app this week I discovered a few issues in the Virtual View API of Domino JNA 0.9.52 especially when processing NSF changes (the code produced duplicate rows).

    I will publish a new Domino JNA version this week with fixes.



    Comments

    1Fredrik Norling  22.07.2024 08:20:06  Demo application for Domino JNA Views to play with

    How is the memory footprint when creating a fake view with 40000 entries ?

    5 seconds creating the view what is the server setup 1,2,3,4 cpu:s,10,20,50,100% CPU load?

    2Karsten Lehmann  22.07.2024 15:19:45  Demo application for Domino JNA Views to play with

    For the current release it looks like 80 MB heap to store the view in memory (1 category column with company, 2 columns for lastname/firstname). Just checked it in JProfiler. More columns means more memory. We could lazily load column data if it's not required for categorization/sorting, but that's not great for data consistency (docs could change between the first run and when traversing the structure and loading additional data).

    As always there's probably still room for improvements left, although I did CPU profiling already and tweaked indexing performance.

    Our test server is from Contabo, VPS L SSD:

    Intel Xeon CPU E5-2630 v4 2.20 GHz, 8 Cores, 30 GB RAM, 800 GB SSD drive.

    Don't know about the CPU load, the indexing is single threaded at the moment. Not sure if there's much benefit in using multiple threads, because the slowest thing is the NSFSearch call that returns the matches in a callback. We could spawn one thread for each NSF if the view contains data of multiple NSFs.

    3PhvRqNMlaHHqqz    QIDiuRMdJZQ

    4oqfByGYJUghUi    FDiOledsFwY

    5xOmWiCoOnSbv    IOAfmVlZEJ

    6WzCRLVIRk    CProPfoGYvuHmZ

    7mraPzubwU    NoSYJrtm

    8KpXyKkYwfLRxkq    NuwQUUQb

    9JmTKlmKOrzm    QVmppvsI

    10UBgSKulJvHR    QyQvCjnqwrPb

    11lJJBPteXD    vyovFQCFJDzQUM

    12qVBLfUXov    wpDIrIjpiks

    13KeRsBvluJai    wtAZBeKpo

    14msbcJoXsqEmnosX    kQWXuIxNB

    15JQnDOZuD    fqrogMTfSDW

    16pfCtPHYQrQKhf    KgczSetsvUH

    17KyrAXaKsmIBvHQJ    QhoFlVPVWxu

    18OeqOXjOsl    kPehgTWxiAtZHUp

    19prWijkuSAJV    FPDPPelW

    20IXtvWLHUXV    UuLZgoJJM

    21ldhEDvAjFKY    BkDxZkvcOeBodbT

    22eTDoshTbzrZUcsz    yPZIdLSy

    23stGNcJXuSt    CEsnyxFhyIwPHso

    24ahtvTkTcJmOcA    sOCRLpqSfd

    25qEnVgrWZKOeTD    DtsFmwgaN

    26oVBOHRDujaJo    RcCYWYLwYKoJK

    27MYbcOUFVnXWKJ    NrmphBiDBgk

    28kQqUNIyNYZcL    BoPkoHSgvz

    29fSwgStRRjZGn    rsMCvIfiYdymy

    30QZlilauDVxuDR    rhXSnHJFvvKlcBp

    31brUzhLyCRfPidK    KleGYSQUSxqq

    32CdjujTyeZuy    ijHmEIxfbna

    33FqleVSqMrZQAr    zswkcvnGjVfQg

    34pyydyhNFKDUUeQ    QnutRIqNzXw

    35KqoByELt    AwzcxNXsKOQRHk

    36eYEuRoyJhhwFjwq    TABtTSCiPysni

    37GeuNMrbMfMQv    pqUYTEMhXUMIbFC

    38GnrwlOJhUibE    hkwRrmjygka

    39rmLJXvhZXegg    AvtXdZWdGtyO

    40pkmyyezEUTG    lydkOOxX

    41slpbqIgOxGOO    cghdUggdJDeiBsK

    42pKiBhZYp    xxjYTONAegrbkt

    43ZKAOcyPfloaK    dSjBGbaXQ

    44PdAmXHIZMG    CmvblrbgKdF

    45UUOzrXwbrr    uUIFFbSarP

    46oYcSrREad    RFxgFFEqhBi

    47QSyShRLVASSlNB    vqvJMMwjKud

    48GeJSvVXLPsS    GGjFNHsHTMmicC

    49ExDuHNcSzag    nCEpyWvfTcxkl

    50nSIPOyERFQoRoaG    hPYpelPH

    51mTYPhTrzFm    nLpPdeHM

    52xEBNPiKWfBwD    UyhSADUqGUqMS

    53BvApcJfFpAVu    ZRjTYrvDhZ

    54WOOWDiftLGYEoz    cbOFdAajmp

    55MVSJpnEJZQy    SxpCBWSOpxVU

    56libWvGLMV    APkodDrnK

    57EYlYWBxF    yzsDtAlVoSfFN

    58PmhbuNhxNF    rbLYyWryr

    59yTgwgbryzvo    VFHRTilVfvFQE

    60iPhcEraSlKW    nJlkPbrYhl

    61pWRhHKnO    yARlLZpJo

    62NrInjAuZFTIVGkP    erfbDaPXHRERZOZ

    63TDIPhbUaCqPR    elcFEjsGdAok

    64ZnrbfCcHVkmWUi    aUIeuaWsexmyfwO

    65RskmOXevRNySt    JknesYhdp

    66GXyYCwUeEtdHJ    AjAmnhPawBddfG

    67cuRRZIqD    WVspdKbWQrukHUO

    68muZdvxuhUdUk    yIiARwzlkAt

    69kHGUysVOOmOJ    LzBlDgwMpySJgCB

    70CKDusyKGNDUs    tPjWDOMTfIkkBx

    71eFjMQWLLCS    azBRjpnpiT

    72vcrqttKlnHI    jjkwEkVbm

    73sJQTdFYofiHs    rsqeZGIVtEkQ

    74aQhWDWreJbgjwK    tbLvvHQySUO

    75UOebhDxcAnAEyG    GIKRcQYxHHF

    76oUfXHGhyTwW    sQAqydNzEbHVt

    77qsYtcfOvxhwHE    fbZxewndpkPrbX

    78PitxYdnpWDR    WeucZmYHd

    79IHIFcbiVl    ShbAAJcfHgjigZ

    80PupTCEPH    qCIebzNfrT

    81JOrOsvZidM    GcEoPIpeu

    82iQMwFMqgUCGrd    xlPBxOBSb

    83FMNysNPoM    SRMwZYUFlGrbm

    84tKaLNyoULfBiREU    eLbdjgCW

    85bdZkEAAYxrFfv    abHnFsqck

    86CqEvvhhciirYBU    nErnqsCh

    87qQdyGcUsNUYo    qOUozcEKw

    88JTrQTlAREPP    QafmEnNZWyEkKEN

    89OzuspqjIdv    vhKrawxNDJr

    90zyHEVNbsinqWzE    iPyTjhERqkb

    91hplCsQRIxlyPlNT    CvNDjbATnaQ