Dalsgaard Data A/S (Intl)

HCL Notes on Mac OS

Thu, 23 Oct 2025 10:26:25 GMT

What to do if your Notes client on Mac OS starts to "misbehave"?

I have been using the Notes client on Mac OS for a few years. Sometimes with challenges - but with the current version 14.5 (and now with FP5) it is actually not that bad...

However, I have a few issues that I have reported and some of them have been reproduces and will have fixes coming in next releases. Most of the issues I see are related to focus - e.g. in the mail file when I want to put a received mail into a folder (and the same goes for Send and File). When the mail is open it will most of the time loose focus from the dialog to put the mail in a folder. You will have to click on the dialog to get the focus again - or press F6 - and then you can start typing the name of a folder to quickly position to that folder and add/move the mail to the folder. If you do the same from a view/folder (e.g. the inbox) then the focus is right - one of the fixes that have now been released.

Sometimes I see another issue if I open a bookmark pointing at a view in one of my own databases. Normally, it should open the outline in the frame for the menu to show the selected view like:

However, for no apparent reason this sometimes stop working and it will show the navigation as collapsed for the exact same view:

And that is just annoying and comes with some other "quirks" (e.g. focus in the sidebar). It took me quite some time to figure out a way to "reset"
this behaviour by basically removing all of or part of the Expeditor folder. If you remove all then you loose a lot of the things you may have set up to have the client look and work like you want. So the trick is to remove as little as possible to avoid resetting your customizations. You can do that by:

Close the Notes client
Open a Terminal
Issue this command:

rm -fr "~/Library/Application Support/HCL Notes Data/ Exp editor/Applications/.metadata/.plugins/com.ibm.rcp.personality.framework"

Start the Notes client again

And your Notes client is back to "normal" behaviour!

I hope this will be fixed in a future release. The above information has been provided to Tim Clarke who is the product manager of the Notes client - so I am sure it will be addressed when possible.

HCL Nomad and TLS via CertMgr task

Mon, 08 Sep 2025 11:45:06 GMT

So I had this idea to test out Nomad Web on a brand-new v.14.5 Domino server as an alternative to the normal Notes client - but it had to fit into the "normal" way I give access to our servers...

Normally when I open up for access to an internal service I use an internal proxy server to orchestrate all the web sites/service. One majo benefit of this is to handle all TLS (using LetsEncrypt) in one central place. And this has worked and still works fine!

I had recently built an image from the community server to include e.g. Nomad on the Domino that I run in a container (on a QNAP NAS - but that is a different story!). So after having upgraded the server to use the new image I just needed to start the Nomad server task ("load nomad" - which I always handle in program documents) for Nomad to be ready on my server. Nomad will listen to requests on port 9443 "out of the box". But it also complained that it was not running on TLS. In the past I have lifted off the TLS cert on the proxy server and then continued to send the request on http (ie. not encrypted). However, I have also become aware that this may not be the best way to handle it (although I do think we have control over all access internally on our network - unless some foreign webcam, printer, or ... is listening to the internal traffic....???). Anyway, this was then the occasion to set up TLS on the inside.

First, I set up a website to be used from the outside called "nomad.domain.com" on the Nginx proxy server. Using the "certbot" code from LetsEncrypt it is just a matter of seconds to have a working TLS certificate on your Nginx server. So far so good - this is all wellknown ground.

Then I set up an internet site document on my Domino 14.5 server called "domino2.domain.com". However, this specific server is normally not accessible from the outside as there is no website on the proxy server to send anything that way. But as I wanted to use the "certmgr" task on Domino to automatically maintain the TLS certs for me I would need LetsEncrypt to be allowed to reach the server on port 80 (as you can see here ). So how to do that? Well, after some friendly chats with some of the AI bots I ended up with this configuration on the nginx server:

server {

listen 80;

server_name domino2.domain.com;

location /.well-known/acme-challenge/ {

proxy_pass http://domino2.domain.com:80;

proxy_set_header Host $host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

}

location / {

return 404; # Normal operation – only .well-known is allowed

}

From the "outside" any subdomain of the " domain.com" will be sent to our public IP. And on the inside all traffic on port 80 is sent to the proxy server that then looks at the request. But internally "domino2.domain.com" will resolve to the Domino server - hence the proxy server can identify the request from the outside (well only from LetsEncrypt) and send it on to "itself".

Next I already had CertMgr running on our internal mail server (to maintain TLS for the domain name used for mail. I was not sure if I had to start certmgr on domino2 or just create a replica of certstore.nsf from the mail server. I looked at a couple of articles ( here and here ) without really becoming sure what to do. But it turns out that it doesn't matter. The certmgr task handles it either way. I ended up loading certmgr on the second server as well - and it nows that the other one is the "boss" do it just acts as a client (and creates certstore.nsf). Next, you set up TLS the "normal" way - and with the Nginx configuration in place (Ok, it took a couple of attempts) then a new certificate is received.

Then we want to make sure that the connection internally works to allow nomad to load on domino2. I started by first making sure I could open just the standard "homepage.nsf" on TLS. First you need to open up for port 443 in the server document under "Ports / Internet Ports / Web" - ensure TLS Port status is "Enabled". Next, you look at your internet site document where you need to have host names set to: "domino2.domain.com" - and you may as well add "nomad.domain.com" now (to ensure proper logging when the final setup is ready). And then you must go the Security tab and change the name of the Key filename to: domino2.domain.com (the exact same name that you used in the TLS Credentials document in the certstore.nsf. This will tell Domino which certificate to use.

With all this in place you should be able to open https://domino2.domain.com (when inside your local network - effectively what the nginx server will nee to be able to forward requests to).

One thing that turned out is that you have to tell Nomad how to find the right certificate from certstore.nsf (or really the TLS cache). It is not yet clever enough to use the same mechanism as the rest of Domino. You need to set a notes.ini variable (please do use a configuration document!). In our case:

NOMAD_WEB_HOST=domino2.domain.com

Restart the nomad task ("restart task nomad") and you should now be able to load nomad using: https://domino2.domain.com:9443

Having reached this point I wanted to be able to give access to nomad on the "normal" TLS port 443 from "the outside". So back to the Nginx proxy for the website "nomad.domain.com". I had to build a configuration based partly on what we normally do and partly on this article and before I finally had it working. This is the config:

map $http_upgrade $connection_upgrade {

default upgrade;

'' close;

}

# Copy this setup to the new server, set the server_name and name the virtual server the same

server {

server_name nomad.dalsgaard-data.dk;

location / {

proxy_pass https://domino2.domain.com:9443$request_uri;

# include /etc/nginx/default.d/*.settings;

proxy_set_header Host nomad.domain.com;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

proxy_set_header X-Forwarded-Proto $scheme;

# Activate SNI, so Domino2 can select the right certificate

proxy_ssl_server_name on;

}

location /nrpc-wss {

proxy_pass https://domino2.domain.com:9443/nrpc-wss;

proxy_http_version 1.1;

proxy_set_header Host $host;

proxy_set_header Upgrade $http_upgrade;

proxy_set_header Connection $connection_upgrade;

}

listen 443 ssl; # managed by Certbot

ssl_certificate /etc/letsencrypt/live/nomad.domain.com/fullchain.pem; # managed by Certbot

ssl_certificate_key /etc/letsencrypt/live/nomad.domain.com/privkey.pem; # managed by Certbot

include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot

ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}

server {

if ($host = nomad.domain.com) {

return 301 https://$host$request_uri;

} # managed by Certbot

listen 80;

server_name nomad.nomad.domain.com;

return 404; # managed by Certbot

}

... and now I have a nomad set up where I route through our proxy for external connections and can use nomad internally with a direct url to Domino - but still using TLS.

Nu kommer HCL Notes/Domino version 14.5!!

Mon, 26 May 2025 12:36:31 GMT

Efter lang ventetid og 3 EAPs (early access products) bliver den nye version snart frigivet

På dette års Engage User Group i Haag i Holland i sidste uge blev en masse nye features vist sammen med nært beslægtede produkter til udvikling, REST service, mobil, etc.

Det blev også annonceret, at Notes/Domino 14.5 vil blive frigivet den 17 juni 2025.

Du kan registrere dig til den officielle frigivelse her .

Der er en hel bunke af nye features, som du vil finde interessante:

Sikkerhed nu helt fremme på højde med industri-standarder (TLS, OIDC, Passkeys, etc)
Kunstig intelligens (AI) gennem Domino IQ, som respekterer adgangsrettigheder til data i Domino databaser
Opdateret Java (version 21)
Performance og skalerbarhed

Now HCL Notes/Domino version 14.5 is coming!!

Mon, 26 May 2025 12:25:39 GMT

After a long wait and 3 EAPs (early access products) it will soon be released

At this year's Engage User Group in The Hague in the Netherlands last week a lot of the new features in Notes and Domino was showcased together with "companion products" for development, REST services, mobile, etc.

It was also announced that Notes/Domino 14.5 will be released on: 17 June 2025.

You can register to attend the launch here .

There is a bunch of new stuff that you want to get to know:

Security caught up with the industry standards (TLS, OIDC, Passkeys, etc)
AI through Domino IQ (that respects your access rights to data in Domino databases)
Updated Java (version 21)
Performance and scalability

Override HTTP headers for an XPage

Wed, 30 Apr 2025 21:05:42 GMT

Normally, we don't need to override HTTP headers - but you may need it for caching

When you create web pages in HCL XPages then they often have quite a lot of interactivity and you want all changes to be updated and shown right away. However, I recently had a case where I wanted to cache one particular page as it would get a high number of users in. a short time. We had seen that over a couple of occasions.

So after having verified my ideas I went into setting the cache controls in the beforeRenderResponse header of the page in question. That looked something like this:

<xp:this.beforeRenderResponse><![CDATA[#{javascript:

var m = Configuration.getCacheExpiryMinutes();

var response = facesContext.getExternalContext().getResponse();

var now = new Date();

var expiresDate = now.getTime() + (m * 60 * 1000);

response.setHeader("Cache-Control", "public");

response.setDateHeader("Expires", expiresDate);

print("m=" + m + ", Expires: " + expiresDate.toString()); // For debugging

}]]></xp:this.beforeRenderResponse>

Then I just needed to test, right?

Well, it turned out that something wasn't going as expected. Using the developer tools of my browser I found that the response headers were not like I expected:

HTTP/1.1 200 OK

Server: Lotus-Domino

Date: Wed, 30 Apr 2025 20:17:14 GMT

Cache-Control: public

Expires: Wed, 30 Apr 2025 20:19:12 GMT

Content-Type: text/html;charset=utf-8

Expires: -1

Cache-Control: no-cache

Content-Encoding: gzip

Content-Length: 1546

Please note that the two headers "Cache-Control" and "Expires" appears TWICE.... So Domino adds its own headers ignoring that I added them as well. This breaks any proxy caching (I use Nginx). I did many things to try and solve this. But it was only with a little help from a colleague, Sjef Bosman, and one particular old article by Sven Hasselbach that I found out what to do. ..

Basically, there is an old IBM cache header class that you need to extend and override one method. It's all in Sven's article. So this is the class I created:

package dk.catchlog.control;

import javax.servlet.http.HttpServletResponse;

import com.ibm.xsp.context.RequestParameters.ResponseCacheHeader;

// Allow overriding cache headers for an XPage. See more: https://hasselba.ch/blog/?p=709

// Add to beforePageLoad event

public class CacheHeader implements ResponseCacheHeader {

public CacheHeader() {

}

@Override

public boolean initResponseCacheHeader(HttpServletResponse arg0) {

// Important: Return true to allow overriding XPages headers

return true;

}

Please note the overriden function returning "true". That is the key to make this work.

So back to our XPage and modify the code in the beforeRenderResponse (or beforeLoadPage) is extended a little:

<xp:this.beforeRenderResponse><![CDATA[#{javascript:

var m = Configuration.getCacheExpiryMinutes();

var response = facesContext.getExternalContext().getResponse();

var now = new Date();

var expiresDate = now.getTime() + (m * 60 * 1000);

response.setHeader("Cache-Control", "public");

response.setDateHeader("Expires", expiresDate);

// Set response cache header object

var reqParam = facesContext.getRequestParameters();

var cacheHeader = new dk.catchlog.control.CacheHeader();

reqParam.setResponseCacheHeader(cacheHeader);

print("m=" + m + ", Expires: " + expiresDate.toString()); // For debugging

}]]></xp:this.beforeRenderResponse>

Back to testing - and now we find these headers:

HTTP/1.1 200 OK

Server: Lotus-Domino

Date: Wed, 30 Apr 2025 20:56:06 GMT

Cache-Control: public

Expires: Wed, 30 Apr 2025 20:58:04 GMT

Content-Type: text/html;charset=utf-8

Content-Encoding: gzip

Content-Length: 1546

And bingo! Now we have the right headers - so the rest is up to setting up caching in the Nginx proxy server and you are good to go!

A litte side note... I had a look at the events beforeRenderResponse and beforePageLoad . This is actually a splendid example of something that you would put into the beforePageLoad as it is called very early in the life cycle before any data initialization and ideally is used for initialization. So I ended up putting my SSJS code in this event. But it will work in both events!

How to allow CORS in Domino

Tue, 22 Oct 2024 21:24:56 GMT

In today's IT landscape it is quite normal that you have to call "services" in client driven website - and that can easily be across domains...

In this situation you may end up running into CORS problems. CORS is short for: "Cross-Origin Resource Sharing" and it solves a security problem by allowing certain sites to access data from a different domain without blocking it. Basically, the server hosting the service defines that some "foreign" domains are allowed to access the services.

A very typical indication of having CORS problems starts with seeing messages like these on the console:

Access to fetch at ' http://server.../path/db.nsf/JSONinital?openagent&StartDate=01012021&EndDate=01022021' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

cors.html:18 GET http://server.../path/db.nsf/JSONinital?openagent&StartDate=01012021&EndDate=01022021 net::ERR_FAILED 200 (OK) (anonymous)@

So what do you do?

Since version 10.0.1 of Domino there has been a DSAPI extension that you can use. Basically, you insert the filter name in the DSAPI filter field on the second tab of an internet site. For Windows you will have to add an "n" so you should enter "ncorsext" in that field. The extension will then read a file in the "data/domino/cors" directory (you may need to create the "cors" folder) named: "cors-rules.json". The file will need to have contents like:

{

"version": "1.0",

"rules": [

{

"resource": {

"path": "/"

"allowOrigins": ["*"],

"allowMethods": [

"POST","GET","HEAD","OPTIONS"

"allowCredentials": true

}

]

}

This file basically allows access to all paths ("/") from all origins ("*").

When you restart the web server ("http" task) then you should see this on the console:

HTTP Server: DSAPI CORS Filter Loaded successfully

... and then all should be good, However, in our case we never got this working on a Domino 12.0.1 server... We created a support case with HCL and tried a bunch of things and captured debug information. Find much more details in eg. this support article. To be honest we also found some misconfigurations that "blurred" the tests and config attempts that we tried...

So what to do then?

Well, we ended up using simple web rules to add the relevant CORS headers to the website instead of the DSAPI filter. In my opinion a much cleaner setup as we do not need to have access to edit a file in the file system.

So this is the web rule that we set up:

Hvordan kan man definere og bruge en SSJS funktion på en XPage

Thu, 05 Sep 2024 14:51:55 GMT

Definer en lokal SSJS funktion uden at bruge et 'library'

Jeg har længe været irriteret over ikke at kunne definere en server side Javascript (SSJS) funktion på en enkelt XPage eller oftere en Custom Control.

Man kan naturligvis altid tilføje en funktion til et ' script library', men måske skal funktionen kun bruges indenfor denne ene kontrol (mange steder) og ved at lægge den i et globalt 'library' så vil vi "forurene" systemet som helhed.

Så i dag slog en idé ned i mig, da jeg sad og redigerede en Custom Control. Jeg prøvede den af - og den virker...!!

Jeg prøvede simpelthen at tilføje en funktion til en datacontext variabel:

<xp:this.dataContexts>

<xp:dataContext var="localSSJS">

<xp:this.value><![CDATA[#{javascript:

// Sneak a method in here that can be called from the entire this CC

var getCellStyle = function(id){

var selected = document.getValue(FN_RiscCellId) == id;

return 'matrixCell'+getLabel('color5x5Matrix'+id, compositeData.formName) + (selected ? ' circle' : '');

}

}]]></xp:this.value>

</xp:dataContext>

</xp:this.dataContexts>

Og senere kunne jeg bruge koden i Custom Control'en som her:

<xp:td id="td11">

<xp:this.styleClass><![CDATA[#{javascript:getCellStyle('2.5')}]]></xp:this.styleClass>

:
:
</xp:td>

Og voilà - vi har en funktion defineret inden for. én XPage eller Custom Control som vi kan bruge mange steder for at undgå at duplikere logik - og stadigvæk uden at tilføje den til et globalt 'script library'!

How to define and use a SSJS function inside an XPage

Thu, 05 Sep 2024 14:44:42 GMT

Define and use an SSJS function without using a library

I have long been annoyed by not being able to define a server side Javascript (SSJS) function inside an XPage or more often a Custom Control.

You can always add the function to a script library, however, the function may only apply to one control and as such will "pollute" the overall system in a script library.

Then today an idea crossed my mind and I tested it out - and it works...!!

Basically, I tried to add the function to a datacontext variable like this:

<xp:this.dataContexts>

<xp:dataContext var="localSSJS">

<xp:this.value><![CDATA[#{javascript:

// Sneak a method in here that can be called from the entire this CC

var getCellStyle = function(id){

var selected = document.getValue(FN_RiscCellId) == id;

return 'matrixCell'+getLabel('color5x5Matrix'+id, compositeData.formName) + (selected ? ' circle' : '');

}

}]]></xp:this.value>

</xp:dataContext>

</xp:this.dataContexts>

And then the code is used later in the Custom Control like this:

<xp:td id="td11">

<xp:this.styleClass><![CDATA[#{javascript:getCellStyle('2.5')}]]></xp:this.styleClass>

:
:
</xp:td>

And voila - we have a function defined inside one XPage or Custom Control that we can use in several places to avoid duplicating logic - and still don't have to put in a globally visible script library!

25 years!

Fri, 17 Mar 2023 09:39:56 GMT

Dalsgaard Data A/S celebrates 25 years!

On 17 March 2023, it was 25 years since the first working day at Dalsgaard Data - at that time not yet registered as a limited company.

In the beginning the nature of the work was what we called "body shopping" - i.e. mediation through a consulting agency. That has developed to having our own clients with slightly more versatile tasks ranging from development to operations/architecture - as well as tasks in the field between these.

Part of the work is still based on Notes/Domino, which at the time had just been bought by IBM, but is now owned by HCL. I started on version 4.6, and while writing this I am currently installing an HCL Domino 12.0.2 in a container (Docker). Quite an impressive journey for a technology that was ahead of its time in many respects at that time.

In addition to Notes/Domino, development for mobile platforms has also become a permanent part of the toolbox, primarily Xamarin (from Microsoft) but also web-based frameworks for mobile. Finally, the Couchbase database has become a larger part of the toolbox. It has many of the characteristics that a Notes database has - but also some wild features for scaling and integration with mobile.

It has been a fantastic journey - and we will continue it!

25 år!

Fri, 17 Mar 2023 09:24:56 GMT

Dalsgaard Data A/S fylder 25 år!

Den 17. marts 2023 var det 25 år siden, at det var første arbejdsdag i Dalsgaard Data - den gang endnu ikke et aktieselskab.

Opgaverne har ændret karakter fra starten, hvor det var dét som vi kaldte "body shopping" - altså formidling gennem et konsultentbureau, over til egne kunder med lidt mere alsidige opgaver både indenfor udvikling og drift/arkitektur - samt opgaver i feltet mellem disse.

En del af arbejdet er stadigvæk baseret på Notes/Domino, som den gang lige var blevet købt af IBM, men som nu ejes af HCL. Jeg startede på version 4.6, og sidder netop nu mens jeg skriver dette og installerer en HCL Domino 12.0.2 i en container (Docker). Ganske imponerende rejse for en teknologi, som den gang på mange punkter var forud for sin tid.

Ud over Notes/Domino er udvikling til mobile platforme også blevet en fast del af værktøjskassen, primært Xamarin (fra Microsoft) men også web-baserede frameworks til mobil. Endeligt er databasen Couchbase blevet en større del af værktøjskassen. Den har mange af de karakteristika, som en Notes-database har - men også nogle vilde features til skalering og integration med mobile.

Det har været en fantastisk rejse - og den fortsætter vi!

Hvor er min faces-config.xml??

john.dalsgaard (John Dalsgaard) — Tue, 04 Oct 2022 08:01:49 GMT

Du har netop installeret en ny Domino Designer og vil udvikle XPages....

Og nu vil du gerne tilføje en bean til din applicaiton - men hvor er "faces-config.xml" filen henne?? Du finder den jo normalt under "Application Configuration" i Applications navigator til venstre.

Det er et stykke tid siden jeg har sat dette op, så jeg havde fuldstændigt glemt hvor jeg kunne finde det.

Men det er nu ikke så vanskeligt. Åben Preferences, vælg Domino Designer o g derefter Applications Navigator - og voilà h er finder du så et flueben, som mangler at blive sat ��

Når du har sat fluebenet ud for "Faces-config" så vil du nemt kunne finde filen igen - og formentligt glemme hvordan du fandt den næste gang du har installeret en ny version af Domino Designer ��

Where is my faces-config.xml??

john.dalsgaard (John Dalsgaard) — Tue, 04 Oct 2022 07:46:19 GMT

You have just installed a new Domino Designer and want to do XPages development....

And then you want to add a bean to your application - but where is the "faces-config.xml" file now?? You are used to find it under "Application Configuration" in the Applications navigator.

It's been a while since I have set this up so I had entirely forgotten how to do it.

But it's not too difficult. Open your Preferences, select Domino Designer and then Applications Navigator - and voilà there you find a check mark missing ��

Once you have checked the "Faces-config" item you will have easy access to the file again - and most likely forget how you found it next time you have installed a brand new version of Domino Designer ��

HCL Notes & Domino 12.0.2 in public beta

john.dalsgaard (John Dalsgaard) — Thu, 28 Jul 2022 16:34:03 GMT

You can get access to the next big update toHCL Notes & Domino which is in beta now!

HCL has just held a webinar showing many of the new facilities coming in the next update of the server as well as the client. And very important - everybody with a license can download the coming version and try it out at home. There is also a possibility to influence the functions of the final version by giving feedback to the developers via a forum dedicated to the beta version.

You can see the presentation and the webinar itself.

HCL Notes & Domino 12.0.2 i public beta

john.dalsgaard (John Dalsgaard) — Thu, 28 Jul 2022 16:26:39 GMT

Du kan få adgang til den næste store opdatering af HCL Notes & Domino, som er i beta nu!

HCL har netop afholdt et webinar med gennemgang af de nye faciliteter, der er i den kommende opdatering af både serveren og klienten. Og vigtigt - alle med en aktiv licens kan gå ind og hente den kommende version og afprøve den selv. Der er også mulighed for at påvirke den endelige version ved at give feedback til udviklerne via et forum specielt til beta-versionen.

Du kan se præsentationen , og selve webinaret

Use smileys (or emojis) with your IBM Domino REST services

john.dalsgaard (John Dalsgaard) — Tue, 03 May 2016 21:15:48 GMT

I have an XPages app that has an interface to a native mobile app (running on Android and iOS). The app can be used as a "full" client to the application - so it both reads and writes data to the Domino based application.

As you probably know we have some extra keyboards on the mobile devices - which allows us to express our emotions. They are called emojis or simply smileys �� . Normally when we see these special characters in IBM Notes (or web based applications running on IBM Domino) they are translated to something that may not always be visible... - or more correctly, shown as a series of hieroglyphs. Now, one of our users entered a description on his cell phone and included a smiley and submitted it to the server - and it just came back "awkward" - so I decided to try and figure out how to cope with these so often-used special characters.

The journey to the result was long - and I learned a lot about unicode - and didn't understand everthing (e.g. "high surrogate" and "low surrogate"). And in the end it turned out that I didn't need to know all this!!

I use an MVC model where I read data into Java objects and after that all data handling is done through the Java objects. A Java String is naturally in unicode (actually it is the "utf-16" character set that uses two bytes per character whereas "utf-8" only uses one byte per character). In Denmark we have three extra national characters (compared to English) and for quite a while I have used "utf-8" to encode any data that needs to be sent to/read from other systems (e.g. in a REST service). This has proved to work well - that is, until now.

What I normally do is something like this:

HttpServletResponse response = (HttpServletResponse) FacesContext.getCurrentInstance().getExternalContext().getResponse();
ResponseWriter out = FacesContext.getCurrentInstance().getResponseWriter();
response.setContentType("application/json");
response.setHeader("Cache-Control", "no-cache");
response.setCharacterEncoding("utf-8");

try {

out.write(gson.toJson(data));

} catch (Exception e) {

e.printStackTrace();

} finally {

// Stop the page from further processing;
FacesContext.getCurrentInstance().responseComplete();

}

The "gson" variable is just a "GsonWrapper" as I have written about earlier . The "data" object is the Java object containing the data that I want to send out on the wire (and GSON converts that to JSON for me). The bean where this code is called is placed in an XAgent (i.e. an XPage that does not render any output):

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" rendered="false" viewState="nostate">
<xp:this.afterRenderResponse><![CDATA[#{javascript:ServiceController.process()}]]></xp:this.afterRenderResponse>
<xp:spanstyle="font-weight:bold">JSON Service</xp:span>
<xp:br></xp:br>
NB this page is a placeholder for returning all different data objects as JSON. Therefore, this page is NOT rendered....
<xp:br></xp:br>
</xp:view>

So far I have had the call to the processing function in the bean in the "afterRenderResponse" event of the XPage. This turns out to be important. And I cannot remember why I ended up calling it there instead of in the "beforeRenderResponse" event...

This concept has served me well - it shows all the special characters and performs well. That is until someone sent a smiley... ��

The problem is that there are not enough bits in "utf-8" to show all of the characters in the world. However, there are som built-in magic that can use two bytes to represent a character ("utf-16" uses two bytes to show all characters - just like the good old Lotus double byte character). The beauty is that we normally don't have to care about it - the programming tools that we use will normally seamlessly handle this. However, we need to work with the "bytes" as opposed to "characters" to allow the seamless handling. So we need to send a byte stream as opposed to a String back as a response to the http request that wanted to read some data from our service. To make this happen we need to use a "DataOutputStream" instead of the "ResponseWriter" that I used above. Our code to send the response will now look like this:

HttpServletResponse response = (HttpServletResponse) FacesContext.getCurrentInstance().getExternalContext().getResponse();
response.setContentType("application/json");
response.setHeader("Cache-Control", "no-cache");
response.setCharacterEncoding("utf-8");

try {

DataOutputStream out = new DataOutputStream(response.getOutputStream());
String json = gson.toJson(result);
byte[] utf8JsonString = json.getBytes("UTF-8");
out.write(utf8JsonString, 0, utf8JsonString.length);
} catch (Exception e) {

e.printStackTrace();

} finally {

// Stop the page from further processing;
FacesContext.getCurrentInstance().responseComplete();

}

But if you had the XAgent the same way as I had then you will see an error on the server console:

HTTP JVM: java.lang.IllegalStateException: Can't get an OutputStream while a Writer is already in use

Now what is that about??? Well, if your XAgent calls the bean to handle the service in "afterRenderResponse" then a "Writer" has already been initiated. So you need to call it in "beforeRenderResponse":

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" rendered="false" viewState="nostate">
<xp:this. beforeRenderResponse ><![CDATA[#{javascript:ServiceController.process()}]]></xp:this. beforeRenderResponse >
<xp:spanstyle="font-weight:bold">JSON Service</xp:span>
<xp:br></xp:br>
NB this page is a placeholder for returning all different data objects as JSON. Therefore, this page is NOT rendered....
<xp:br></xp:br>
</xp:view>

And that seems to be it!!! I have been over considerations about how to save data in NotesItems (as MIMEentities instead of a pure NotesItem) - and converting the emojis as per the article about MySQL (see below) - but none of this seems necessary!! - in my environment anyway.

And now our users can express emotions when describing the fish they caught (its an angler's app) - without breaking anything and without us having to remove the funny little characters!

Happy coding!!

Sources that helped me understand this problem and find the solution:

Elevating permissions for your third party jar

john.dalsgaard (John Dalsgaard) — Tue, 06 Oct 2015 21:11:48 GMT

This is the third article related to wrapping third party jars into OSGi plugins. I have meant to write this article for a long time but.... - well, now was the time for it.

The previous articles in this series are:

Depending on the jars you want to use in your application you may run into problems with permissions. This is part of Java's security model - which is different from what we are used to in the Domino world. We have ways to control it via the java.policy file in the jvm (or better use the java.pol file as I have written about before). However, this requires changing files in the filesystem - and in many cases these changes can/will be overwritten by an upgrade or even installation of fixpacks. That is not good for the stability of our application as we may not be notified of this upgrade in advance - and suddenly the application does not work.... - and the problem most likely ends up on your desk.

So what can we do to get around this and control it without having to set security parameters that are global to the entire JVM?

I had this challenge when Nathan Freemann showed me how to do this in a way that is entirely controlled within my application - which I by far prefer

Basically, you need to create a wrapper to the third party code so that you can wrap the privileged calls to allow "elevation" without having to use the permissions in the java.policy file. First you create the wrapper class - in this case I call it "GsonWrapper". I also added a constructor with the possibility to use another date format - as I sometimes have different requirements based on the client (especially native vs. mobile web).

public class GsonWrapper {

private static final String DFT_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss";

private Gson gson = null;/* Constructors... */

public GsonWrapper() {

this(DFT_DATE_FORMAT);

}

public GsonWrapper(final String dateFormat) {

gson = new GsonBuilder().setDateFormat(dateFormat).create();

}

Now to the most interesting part. The way we can make a privileged call is by using an AccessController:

public String toJson(final Object obj) throws Exception {

AccessController.doPrivileged(new PrivilegedExceptionAction() {

public Object run() throws Exception {

GsonWrapper.this.toJsonImpl(obj);

return null;

}

});

return result;

}

... which requires us to implement a method that we can call from within the doPrivileged block to do the actual privileged call to the underlying jar:

public void toJsonImpl(Object obj) {

result = gson.toJson(obj);

}

The result is stored in an instance variable ("result") that builds the "gap" between the implementation method and the wrapper. Sort of the same way you have to do it if you dynamically build and execute your own LotusScript code using "Execute" and want some data back ��

And that is it!

You are now ready to use the wrapper and that could look like this:

// Requires Gson JAR to be put in the jvm ext library... - and AllPermissions set in java.pol/java.policy !!!!
Gson json = new GsonBuilder().setDateFormat(ISO_DATETIME_FORMAT).create(); // 1

// ... alernatively wrapping it in an OSGi plugin (with AccessController.doPrivileged(....))
GsonWrapper json = new GsonWrapper(ISO_DATETIME_FORMAT); // 2

out.write(json.toJson(data));

In this simple example you need to either use the first declaration of "json" (marked "1") OR the second (marked "2"). The code obviously will not compile as it is now ��

The way I have built this you can also see that only the declaration determines if we use the wrapper. I have kept the same method and arguments as in the underlying jar - which makes the code less dependent on the wrapper.

The "data" variable can be ANY Java object structure (single object, list/array of objects, etc) - and it all transforms nicely to the equivalent JSON structure. Very cool!

The only issue I see with this method is that you will have to create all the methods in the wrapper that you potentially need from the underlying jar - with each of the method signatures (argument lists) that you want to use.

Happy coding!

Entire source code for the wrapper (including toJson and fromJson):

package dk.dalsgaarddata.ext.gson;

import java.lang.reflect.Type;
import java.security.AccessController;
import java.security.PrivilegedExceptionAction;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;

/**
* @author jda
* @version 1.0.0
*
* Date: 2015.09.20
*
* This utility class wraps a call to Gson into a privileged call to allow elevation without having
* to grant "allaccess" to all jars in the jvm/lib/ext
*
* This class needs to be exported to a jar file and placed in the jvm/lib/ext library for it to be able handle
* the priviledged calls - or I can be included in the OSGi plugin :-)
*/
public class GsonWrapper {

private static final String DFT_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss";
private Gson gson = null;
private String result = null;
private Object userClass = null;

/* Constructors... */
public GsonWrapper() {

this(DFT_DATE_FORMAT);

}

public GsonWrapper(final String dateFormat) {

gson = new GsonBuilder().setDateFormat(dateFormat).create();

}

public void toJsonImpl(Object obj) {

result = gson.toJson(obj);

}

public void fromJsonImpl(String obj, Class<?> cls) {

userClass = gson.fromJson(obj, cls);

}

public void fromJsonImpl(String obj, Type type) {

userClass = gson.fromJson(obj, type);

}

@SuppressWarnings( { "unchecked" })
public String toJson(final Object obj) throws Exception {

AccessController.doPrivileged(new PrivilegedExceptionAction() {

public Object run() throws Exception {

GsonWrapper.this.toJsonImpl(obj);

return null;

}

});
return result;

}

@SuppressWarnings( { "unchecked" })
public Object fromJson(final String obj, final Class cls) throws Exception {

AccessController.doPrivileged(new PrivilegedExceptionAction() {

public Object run() throws Exception {

GsonWrapper.this.fromJsonImpl(obj, cls);
return null;

}

});
return userClass;

}

@SuppressWarnings("unchecked")
public Object fromJson(final String obj, final Type type) throws Exception {

AccessController.doPrivileged(new PrivilegedExceptionAction() {

public Object run() throws Exception {

GsonWrapper.this.fromJsonImpl(obj, type);
return null;

}

});
return userClass;

}

XPages ressources in a theme - http vs. https

john.dalsgaard (John Dalsgaard) — Mon, 14 Sep 2015 20:54:46 GMT

I am at the moment moving a site from standard http to https (using Nginx as a proxy in front of Domino - but that is a different story). However, I did run into a small issue. When I looked at the console in Google Chrome this message appeared:

index.xsp:6 Mixed Content: The page at 'https://www.fiskogtast.dk/fangst.nsf/index.xsp' was loaded over HTTPS, but requested an insecure stylesheet '//fonts.googleapis.com/css?family=Actor'. This request has been blocked; the content must be served over HTTPS.

... and when looking at my theme I found this:

<resource>
<content-type>text/css</content-type>
<href>//fonts.googleapis.com/css?family=Actor</href>
</resource>

... and well, yes I had not taken into account that I was going to run SSL when adding that reference a while back. But I have taken on a new habit since then - now I use "protocol relative" urls. That is basically the same that we do when specifying "/db.nsf/..." in our apps as "domain relative" urls. So in this example I could use //fonts.googleapis.com/css?family=Actor . So I changed the href accordingly and refreshed my XPage....

But this did not do what I expected. Now the reference to the external resource was concatenated into an internal reference:

I tried to disable "Use runtime optimized Javascript and CSS resources" in the XSP properties - but didn't seem to make a huge difference...

So I did a little more ressource and found (via this link by intec ) that you can use the rendered attribute on a resource tag in a theme. So I ended up with this (not so elegant) solution -that seems to work fine:

<resourcerendered="${javascript:context.getUrl().getScheme() != 'https'}">
<content-type>text/css</content-type>
<href>//fonts.googleapis.com/css?family=Actor</href>
</resource>

<resourcerendered="${javascript:context.getUrl().getScheme() == 'https'}">
<content-type>text/css</content-type>
<href>https://fonts.googleapis.com/css?family=Actor</href>
</resource>

Happy coding!

Overriding constants in Java

john.dalsgaard (John Dalsgaard) — Thu, 03 Sep 2015 20:48:35 GMT

If you are like me then you really want to minimize duplication of code and reuse as much as possible. A common use case that has caused me trouble is this scenario where I have a common function that uses some context (in the form of constants). For a particular subclass I just need to redefine the constant to make everything behave like I want. Something like this:

public class Base {
protected final static String letter = "-";
protected void print1(){
System.out.println("Letter=" + letter);
}
}

public class A extends Base {
protected static String letter = "A";
}

And called like this:
A a = new A();
a.print1();

I had expected to see "A" as the output from above - but no, it is "-"...

Basically, what happens is that you define a new variable in a different scope - instead of overriding the existing constant/variable... And when you think about it I suppose it is quite natural. But then how can one get around this - and keep the basic idea of what I was trying to obtain? Well, you simple use a getter - and override that. So if you do this instead:

public class Base {
protected final static String letter = "-";
protected String getLetter(){
return letter;
}
protected void print2(){
System.out.println("Letter=" + getLetter());
}
}public class A extends Base {
protected static String letter = "A";
protected String getLetter(){
return letter;
}
}And called like this:
A a = new A();
a.print2();

... then you get the expected "A" from the method defined in the Base class but called in the subclass A.

Perhaps very simple - but it took me a little effort to understand that I could not just override the constant/variable and had to change the approach slightly.

Happy coding!!

Can you update a calendar entry using the DAS REST service?

john.dalsgaard (John Dalsgaard) — Mon, 10 Aug 2015 20:36:10 GMT

This morning I saw a question on StackOverflow that I had a look at. Basically, the person asking had tried to update an existing calendar entry - without success. As I have been playing a little with REST I thought I would give it a quick go. And correct, it couldn't be done the way the person asking the question had tried.

So let's get some basics in place:

If you want to CREATE data you use a POST request.
If you want to change INDIVIDUAL fields then you use a PATCH request
If you want to update ALL fields in an extisting record/data set then you use a PUT request.

Being armed with this knowledge I started out trying with a PATCH approach... Long story short - the calendar service seem not to allow PATCH requests. So that one is out of the question. And having said that I have not found any places in the documentation stating that it would support PATCH methods. So all fine...

I then tried a PUT request and with a little playing around (and with the help of the documentation for Domino Access Services (DAS) 9.0.1 ) I made it work :-)

First I specified the request with ALL fields from the result of either querying the event (or when creating it). Following subsequent tests I found that this is not necessary - but you have be aware of the defaults being added for the fields you do not specify - e.g. if you only specify a start time (as in the StackOwerflow question) you get the same end time... So you may want to specify all fields...

Example of a request:

{
"events": [
{
"href": "\\/demo\\/cal.nsf\\/api\\/calendar\\/events\\/0C3AC054CE152609C1257E9D00204F95-Lotus_Auto_Generated",
"id": "0C3AC054CE152609C1257E9D00204F95-Lotus_Auto_Generated",
"summary": "Rotary",
"location": "Main room",
"start": {
"date": "2015-03-30",
"time": "18:30:00",
"utc": true
},
"end": {
"date": "2015-03-30",
"time": "20:00:00",
"utc": true
},
"class": "public",
"transparency": "opaque",
"sequence": 0,
"last-modified": "20150810T055255Z",
"x-lotus-notesversion": {
"data": "2"
},
"x-lotus-appttype": {
"data": "0"
}
}
]
}

And then the important thing. In the PUT request you have to specify the FULL id (not just the UNID part) like:

http://json.dalsgaard-data.dk/demo/cal.nsf/api/calendar/events/0C3AC054CE152609C1257E9D00204F95-Lotus_Auto_Generated

And then it works!

f you want to know more about what you can do with REST services then check out my presentation from SUTO Lon Slideshare

Public access to e.g. an Angular app in your Notes database

john.dalsgaard (John Dalsgaard) — Tue, 16 Jun 2015 20:13:09 GMT

Ok, if you are like me you may want to play with new technologies within your well-known environment (like the IBM Notes/Domino database), e.g. an Angular.js app.

Now, if you are used to the well-known security model in Notes/Domino you may also have your database locked down so no un-authenticated users can access your database. And now you add an "app" to the WebContent folder (via the package explorer view). However, the first time you try to access your "app" it redirects to a login page. What is happening???

Well, you have been hit by a problem that is easily solved by "ordinary" design elements that you can mark for "public access". However, that is not possible for these types of design elements - without a little code....

So I have created the following LotusScript (!!) agent to do the trick. You may need to adjust the path of your app - but the principle should put you on the right track:

Option Public
Option Declare Sub Initialize
Const APP_DIR = "app/"
Const FN_PUBLICACCESS = "$PublicAccess"
Const FLAG_TRUE = "1"
Dim sess As New NotesSession
Dim db As NotesDatabase
Dim nc As NotesNoteCollection
Set db = sess.currentDatabase
Set nc = db.CreateNoteCollection( False )
Call nc.SelectAllDesignElements( True )
Call nc.BuildCollection
Dim d1 As Long
Dim d2 As Long
Dim title As String
Dim flagsExt As String
Dim noteid As String
Dim nextid As String
Dim i As Long
Dim doc As NotesDocument

noteid = nc.Getfirstnoteid()
For i = 1 To nc.Count
'get the next note ID before removing any notes
nextid = nc.GetNextNoteId(noteid)
Set doc = db.GetDocumentByID(noteid)
title = doc.GetItemValue("$title")(0)
flagsExt = doc.GetItemValue("$flagsExt")(0)
If LCase(flagsExt) = "w" And Left(LCase(title),Len(APP_DIR)) = LCase(APP_DIR) Then
d1 = d1 + 1
If Not doc.Getitemvalue(FN_PUBLICACCESS)(0) = FLAG_TRUE Then
d2 = d2 + 1
Call doc.replaceItemValue(FN_PUBLICACCESS,FLAG_TRUE)
Call doc.save(True, False)
Print title & " - set to allow public access read"
End If
End If
noteid = nextid
Next

Print nc.count & " design elements checked. " & d1 & " elements found in '" & APP_DIR & "' and " & d2 & " updated to give public access"

End Sub

REST Services in IBM Domino/XWork

john.dalsgaard (John Dalsgaard) — Thu, 20 Nov 2014 21:10:28 GMT

Today I gave a presentation about REST Services in IBM Domino/XWork server at the DanNotes conference in Korsør. The appetizer for the presentation was:

Loose coupling of systems is key to future development! Why? Because it will allow us to change the "components" as we go along instead of creating monster big systems that are tied together using all sorts of different technologies. Webservices have been a way to obtain this over the last decade. More recently a special variant has become very popular, namely the JSON based REST service.

Imagine you could extend your data out to the world outside your Domino environment? - in an easy way....! And imagine those data could easily be incorporated into other systems via standardized interfaces... Could that extend the value of your current systems further? Could this be a way to use new technologies to modernize your users' experience of working with your systems?

Come and take away knowledge about how to open your Domino/XWork based systems up to the world outside using JSON based REST services. They are going to be key to future development in Domino/XWork - whether you want to use data in browser solutions (e.g. based on angular.js or ExtJS) or native mobile apps (built in whatever technology is best).

You can find the presentation on Slideshare.com - and the demo database is available for download from Bitbucket.org .

... oh, and by the way - I was elected for the board of DanNotes that is the Danish Lotus Notes user group!

Login / logout when using Ajax against Domino

john.dalsgaard (John Dalsgaard) — Wed, 29 Oct 2014 21:06:49 GMT

I tried to figure out how to correctly login and logout of Domino from an Ajax service that I call from a mobile app. Well, to be more specific I actually had login working fine - but logout just wouldn't do it correctly....

The way I saw this was by issuing the following command on the Domino server console:

tell http show users

But now, let's step a little back and let me explain what I do.

I have created an XPage (service.xsp) that is just the "glue" to bind the requests to a managed bean. It looks like this:

<?xml version ="1.0" encoding ="UTF-8"?>

<xp:view xmlns:xp= "http://www.ibm.com/xsp/core" rendered= "false" viewState= "nostate" >

< xp:this.afterRenderResponse ><![CDATA[#{javascript:ServiceController.process()}]]></ xp:this.afterRenderResponse >

</ xp:view >

I have left out the short description for the programmer to see how to use it - but you should add it... The ServiceController is set up in the faces-config.xml file (as any other managed bean). Now, the "process()" method will find out if the request is a "GET" or "POST" method and handles the request accordingly. For logins I do a POST where I submit the username and password (and some other details about the device). From the bean I start an http connection with the url "/names,nsf?login" and then I add the cookie (DomAuthSessId or LtpaToken) to the response of the call to service.xsp.

This is the source code that does all this (some validation and error handling left out):

String urlParameters = "username=" + userName + "&password=" + password;

String request = Util. getSiteRootUrl () + "/names.nsf?login";

URL url;

try {

url = new URL(request);

HttpURLConnection connection = (HttpURLConnection) url.openConnection();

connection.setDoOutput( true );

connection.setInstanceFollowRedirects( false );

connection.setRequestMethod( "POST" );

connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");

connection.setRequestProperty( "charset" , "utf-8" );

connection.setRequestProperty( "Content-Length" , "" + Integer. toString (urlParameters.getBytes(). length ));

connection.setUseCaches( false );

DataOutputStream wr = new DataOutputStream(connection.getOutputStream());

wr.writeBytes(urlParameters);

wr.flush();

wr.close();

BufferedReader in = new BufferedReader( new InputStreamReader(connection.getInputStream()));

StringBuffer sb = new StringBuffer();

String inputLine;

while ((inputLine = in.readLine()) != null )

sb.append(inputLine);

in.close();

HttpCookie authCookie = null ;

String cookieString = connection.getHeaderField( "Set-Cookie" );

if ( null != cookieString) {

List<HttpCookie> cookies = HttpCookie. parse (cookieString);

for (HttpCookie cookie : cookies) {

if ( AUTH_COOKIE .equalsIgnoreCase(cookie.getName()) || AUTH_COOKIE_MULTI .equalsIgnoreCase(cookie.getName())) {

authCookie = cookie;

}

connection.disconnect();

if ( null == authCookie) {

// Find reason code in reply from server:

String reason = getInputValueFromHtml( "reasonType" , sb.toString());

if ( "" .equals(reason)) {

reason = "2" ;

}

debug( "Not logged in. Reason: " + reason + ", " + getLoginFailReason(reason, userName));

addErrorMessage( null , getLoginFailReason(reason, userName));

throwValidationErrors();

} else {

// User logged in. Add cookie to request and response and redirect...

Cookie c = new Cookie(authCookie.getName(), authCookie.getValue()); // Need to convert from HttpCookie to Cookie

if (StringUtil. isNotEmpty (authCookie.getDomain())) {

c.setDomain(authCookie.getDomain());

}

c.setPath( "/" );

((HttpServletResponse) Util. getExternalContext ().getResponse()).addCookie(c);

}

} catch (IOException e) {

e.printStackTrace();

}

This actually works fine! You can see that I have a "Util" class that will provide various elements (eg. ExternalContext, request, response, etc.) - but you get the idea from the method names.

The beauty of this approach is that the authorization cookie gets attached to the response and will automatically get added to the next request sent to the service.xsp - thus recognizing the user as logged in.

So what I do is first log the user in for services that require authentication. Then I call the services (a handfull) and then log the user out again. First, I simply removed the authentication cookie from the response. But that was not sufficient. Then I tried to do create an http connection in the same way as the login - just with the url "/names.nsf?logout" - I still had sessions left in the http task. Now with the help from several other XPages specialists I was pointed at this response from Tony McGuckin on StackOverflow . I then tried to translate the code to Java and call that for logout. But it still left http sessions hanging around....

I played around with various trials - and found out that if I called a url with "?logout" directly from my mobile app (Ajax) then it actually logged out and the http session was released - but then it did not call my service... and then I could not call my own code while doing the logout (e.g. logging).

Well, that could be solved by using plain old Domino web url methods, i.e. the "redirectto" action. So if I called a url like this:

//srv.company.com/db.nsf?logout&redirectto=/db.nsf/service.xsp?open%26command=logout

So far so good.... Then I tried out doing it the other way around. I now call the service.xsp (with a GET request) like this:

//srv.company.com/db.nsf/service.xsp?open&command=logout

and then the logout command will call this Java code:

try {

String logoutUrl = Util. getSiteRootUrl () + "names.nsf?logout";

logoutUrl = logoutUrl + "&redirectto=" + URLEncoder. encode (Util. getBaseUrl () + "service.xsp?open&command=loggedOut" , "utf-8" );

Util. getExternalContext ().redirect(logoutUrl);

} catch (IOException e) {

e.printStackTrace();

}

... and then control is returned to the service controller (though with a new "loggedOut" command) that will return the right JSON to the mobile app calling the service in the first place.

This works like a charm...!

Conclusion: Do not try to solve the logout by just removing cookies. You may succeed in logging out the user (well, you will) - but you will have left the http session hanging around until it times out.

Tuning your XPages...

john.dalsgaard (John Dalsgaard) — Thu, 16 Oct 2014 19:59:50 GMT

Ok this may promise a little more than I can hold.

However, I just wanted to point the attention to some good advice that I had from Tony McGuckin (co-author of Mastering XPages ) at IBM Connect 2014 in Orlando. This piece of advice surfaced through a review of a large application that I was building - and whether the architecture would "hold water" should the application become very popular. The basic concept is that we cache all data in memory in the form of Java objects. I have told about this previously in presentations/articles on MVC and caching .

Tony gave me three pieces of advice to follow to ensure that the application would be able to scale:

1. viewState

This is a very interesting setting. Basically, for pages where you write data back to the server you should use "full" (which is the server default). This will maintain the component tree so the server knows what you are working on (e.g. knows the state of the entire page when you do a partial refresh). You may have to watch the setting "Maximum pages on disk" (in the XPages properties) as well since it determines how many "steps" of the component tree that the server keeps. The default for 8.5.3 is 4 (four) pages. This has been increased to 16 for version 9.0. Normally, this setting will suffice - however, a certain usage pattern can blow you out of that number of pages. If a user opens a page in a new window to check some information for the form they are filling in - then you quickly pass even 16 steps (~pages).

Ok, back to "viewState". You can set this for the entire application in the XPages properties under "Persistence" where it is named "Page persistence mode" (the actual property is: "xsp.persistence.viewstate"). I doubt you will want to set this setting application wide. This would only apply to a "normal homepage" where you only present information. Most of the XPages solutions I see do have some interaction with entering data in them. However, you can set this setting on each XPage. You will find it under "All properties" for the XPage.

The viewState can take 4 different values. Default is "fulltree". In this situation you are used to. You can use "delta" which will only keep "current doc in memory". This setting as well as "deltaex" are for advanced uses - and only applies when using RAM persistence - exclusively! Have a look at page 1033 in the Mastering XPages book (2nd edition) for more details if you need to consider these. Finally, you could set viewState to "nostate". This, however, is a very useful setting in many cases. If you have a "readonly" page in your application (e.g. the landing page) you should consider not saving any state for it. Another splendid example is an "XAgent" (which is an XPage that is not rendered - typically used as a "service" to provide e.g. JSON data).

2. Short session timeout

The second advice is to have a low session timeout. Now why would one do that? then we will loose all the data in the user's session and have to reload it if needed. You may actually think that it is better to put the session timeout UP to avoid any problems while the user is working (e.g. timeout while they are entering data....). Well, the problem with sessions is that if the user just closes the browser (and don't do a proper logout) all of the session data is kept in memory until the session times out. So with an application with many users you may quickly end up short of memory due to many "dead" sessions waiting to time out to free their memory. This is the same reason why you want to be careful not to put more data in the session scope than needed.

A better approach then is to put the session timeout DOWN. I use 5 minutes. And then combine that with a "Keep session alive" mechanism (there is one in the XPages Library - or you can build your own). Basically, this just initiates an AJAX call in the background to "touch" the server to let it know that you are alive. As long as it does this just before the session time out it will keep you going for long.... On the other hand, users that close their browsers without logging out will have the sessions recycles within 5 minutes freeing up ressources for new users.

You find the session timeout in the XPages properties on the "General" tab. You set a number of minutes. The name of the property is: "xsp.session.timeout".

3. Long application timeout

There is only ONE application scope for an application (as opposed to the session scope). Therefore, it makes sense to put the application scope timeout UP. I use 300 minutes. This way you can keep all general stuff ready at hand all day. I cache a lot of data in the application scope in a large application at present. This means that it is blistering fast - even for the first user (since I cache the information prior to the first user hitting the application). And once the data is cached it stays there.... I don't really want to "waste" the effort of reading all the data into the application scope too soon. Of course, you will need to implement a proper mechanism to maintain the cache current if you do any updates - but that is a totally different topic.

You find the application timeout in the XPages properties on the "General" tab. You set a number of minutes. The name of the property is: "xsp.application.timeout".

I hope that you can use the above to tune your applications - or at least I may have brought some aspects to your attention that you can then investigate for your own purposes.

Happy coding!

Caching in XPages - not as straightforward as you would believe

john.dalsgaard (John Dalsgaard) — Tue, 09 Sep 2014 19:54:39 GMT

Well, you may know that I am a fan of MVC (Model-View-Controller) pattern (see more here ).

One tempting feature of using an MVC pattern is that it is pretty straightforward to implement a caching mechanism. This can easily be done in the facade layer of the model. And just imagine what happens if all of the data is ready in memory - and you only need to go to the disk/database when some of the data is updated. I can assure you that this means SPEED....

But as with so many things - more wants more. Wouldn't it be nice to preload all data on start of the HTTP server? Then you would avoid the first user hitting the delay of loading data from the database - it will already be ready in memory.

This article is about my experiences with trying to obtain this. And I must warn you - I have not yet found the "silver bullet". But I have discovered a number of mechanisms that may help others understand the underlying features of XPages.

My custom cache - take 1

Ok, how does one create a cache? Basically, you need a way to reference the same memory - no matter from where you do it. I started out using a managed bean in the application scope. As long as the application scope "lives" the data is cached in memory.

I do this by having a "DataBean" - set up as a managed bean in faces-config.xml:

<managed-bean>

<managed-bean-name>Data</managed-bean-name>

<managed-bean-class>dk.dalsgaarddata.demo.bean.DataBean</managed-bean-class>

<managed-bean-scope>application</managed-bean-scope>

</managed-bean>

And here is a snippet of the code. Since it is a managed bean it implements Serializable and you can see that I use lazy initialization for setting the facade bean:

public class DataBean implements Serializable {

private static final long serialVersionUID = 1L;

private PersonCRUDFacade personFacade = null;

public PersonCRUDFacade getPersonFacade() {

if (null == personFacade) {

personFacade = new PersonCRUDFacade();

}

return personFacade;

}}

In the facade bean I have implemented the cache as a HashMap - with the key of the data as key of the entry in the HashMap. Here is a snippet from the facade bean:

public class PersonCRUDFacade implements Serializable {

private static final long serialVersionUID = 1L;

private Map<String, Person> persons = null ;

private PersonDAO getDao() {

return new DominoPersonDAO(); // Not serialized - handles access to the Domino database

}

private Map<String, Person> getCachedPersons() {

if ( null == persons ) {

System.out.println("read persons into cache...");

persons = getDao().getAllPersons(); // With key also as key to the hash map

}

return persons ;

}

public List<Person> getAllPersons() {

return sortByName(getCachedPersons());

}

public Person findPerson(String key) {

return getCachedPersons().get(key);

}

The above are just small snippets. I have left out some of the details (like reading in the cache as Anonymous - but using sessionAsSigner). But hopefully you get the idea.

So the next thing I did was to create an XPage (preload.xsp) that basically reads all the data that I want to cache (by referencing the various facade beans through Expression Language - and simply just returning the size of the map). And then I added the following to notes.ini (through a configuration document, of course):

XPagesPreload=1

XPagesPreloadDB=demo/mvc.nsf/preload.xsp

Restart the server, and looking at the console we can see that data is loaded into the cache. Great! But when we open the application - we can see that the data is loaded into cache again.... What is happening???

My custom cache - take 2

My first thought was that for some reason the application scope was not the right place to put the cache - since it somehow was cleared after preload and before I opened the application. My next thought therefore was to create a true singleton object (using an enum which seems to be best practice) and cache the facade beans in there. And a small snippet to give you the idea:

public enum CacheContainer {

INSTANCE;

private PersonCRUDFacade personFacade = new PersonCRUDFacade();

private CacheContainer() {

System.out.println( "Instantiating enum CacheContainer..." );

}

public PersonCRUDFacade getPersonFacade() {

if ( null == personFacade) {

System.out.println( "Create personFacade..." );

personFacade = new PersonCRUDFacade();

}

return personFacade;

}

In our DataBean we can use the singleton object this way:

public PersonCRUDFacade getPersonFacade() {

return CacheContainer.INSTANCE.getPersonFacade();

}

Back to testing with preload - ... and the same result...! The cache is re-read after preload when the application is accessed afterwards.

My custom cache - take 3

Ok, so how can we then create a "true" caching mechanism? Well, the ServerBean (part of the OpenNTF Domino API) comes to our rescue. This is the true server side caching mechanism. It implements ConcurrentHashMap and as such is ideal for the purpose. So off I went implementing the ServerBean approach. This is a snippet from the DataBean class:

public PersonCRUDFacade getPersonFacade() {

PersonCRUDFacade personFacade = (PersonCRUDFacade) ServerBean.getCurrentInstance().get(PersonCRUDFacade. class .getName());

if ( null == personFacade) {

personFacade = new PersonCRUDFacade();

ServerBean.getCurrentInstance().put(PersonCRUDFacade. class .getName(), personFacade);

}

return personFacade;

}

So back to testing the preload etc. And now we get an error - which seems a little strange when you look at it:

javax.faces.el.EvaluationException: Error getting property 'persons' from bean of type dk.dalsgaarddata.demo.bean.DataBean: java.lang.ClassCastException: dk.dalsgaarddata.demo.dao.facade.PersonCRUDFacade incompatible with dk.dalsgaarddata.demo.dao.facade.PersonCRUDFacade

But what does that mean??? That seems to say that the same class is not compatible with itself...???? What is going on????

The explanation

Following the above error I tried to put a String with a timestamp into the cache - and retrieve that at the same time as I tried to retrieve the PersonCRUDFacade instance. And guess what - the String object was cached quite all right. This led to the explanation of what was going on.

First, the singleton approach would probably have worked quite as good as the ServerBean approach. The problem lies in classloaders....

This has been pointed out by Nathan Freeman and Philip Riand - both very competent and way above my level of knowledge on XPages.

So when you preload an XPage it is done in one classloader. When you open an XPage in application it is done in its own classloader.... i.e. a DIFFERENT classloader - and they cannot share the same objects.... And what happens if the application scope times out? Well, the classloader unloads all classes and start from scratch when a new user hits the application - and in our case that means they have to re-read data into the cache.... (well with the ServerBean implementation we would get the ClassX is incompatible with ClassX error)

Sigh....

What can you do??

I am currently working with this approach:

Do not try to preload a specific XPage - it won't help you
Call the application via a URL that will preload the data, e.g. as mention via a special preload XPage. I do this from a DDM probe - which checks that the application (and server) is alive - I even check every 15 minutes which is shorter than my application timeout, so my data is cached while the server is running....

This is not ideal - but it works. The only other solution would be to have the class (facade bean) outside of the NSF. You could put it in a JAR and wrap it up as an OSGi plugin (see the steps here ) - but that makes deployment a totally different game - and you may not want to do that.

I hope to have saved you some of the hassle that I have been through by sharing the above information.

Happy coding!

Triggering daily scheduled agents in Domino - from tomorrow

john.dalsgaard (John Dalsgaard) — Mon, 14 Jul 2014 19:45:15 GMT

Have you ever come across this little scenario:

You have a scheduled agent set to run at 5:00 in the morning
For some reason you need to change something in this agent (or enable it when updated from a template). You do this just before noon...
What happens? Suddenly, your agent starts running!!! .... and this most likely was not what you expected!

This happens to agents that are scheduled to run once a day. If you modify them in any way later that day they will behave as if they had not run (which they will also tell you, if you right-click on the agent in Domino Designer and select "View log") - and they will run shortly after! I have seen this confuse many admins (and developers) over time.

I have seen many interesting ways of trying to cope with this from within the code - unfortunately, none of them have been fool-proof, say if the scheduled agent was to run in the evening, i.e. after it was modified).

I have found a simple solution some time ago - and recently saw someone writing about this in another context - so I thought I would share since it is so simple:

Set your agent to run "More than once a day"
Set it to allow to run between e.g. 5:00 and 6:00
Now set it to run every e.g. 5 hours
Voilá - if you enable the agent after 6:00 then it will not run until tomorrow

Names in IBM Domino - a warning!!

john.dalsgaard (John Dalsgaard) — Wed, 04 Jun 2014 19:41:15 GMT

Ok, if you are like me you have worked with names in IBM Notes/Domino many, many times. And you may even have built your own directory solutions to put users into your application instead of in the central directory (like I have described in this article ). If you have done the latter then you may have faced (or will in future) the dirty details of naming conventions in Domino...

I have this system where users can register themselves. I calculate a "full" name based on their name and a key that I create to ensure that the fullnames are unique. This is all well and works fine for login etc. etc. However, today I learned that you want to be careful with the naming. I only ever use the abbreviated name in this application. So I create an abbreviated name, say: "John Dalsgaard/12345/Org" and store this as the "FullName" in my user document (equivalent of the Person document in the Domino Directory). This all works well - apart from this situation:

If you have author access to the database and use an authors field to enforce the security model you have built into your logic - and you use a ROLE in the acl to give certain users access to delete documents created by other users - then you will find that these users will NOT be able to delete those documents although they have the role and the role is mentioned in the authors field!!

Ok, I use Java. So when I want to check for roles i use something like this:

getSession().getCurrentDatabase().queryAccessRoles("John Dalsgaard/12345/Org")

It returns the roles I would expect. So far so good. However, if I called:

NotesContext.getCurrent().isDocEditable(doc)

then that returns "false" - which reflects that the server is throwing errors at me! How can that be?

Next test is to try something simple:

getSession().evaluate("@UserRoles")

... no roles returned??? What on earth is going on? So I tried calling:

getSession().evaluate("@UserName").elementAt(0)

to ensure that I was who I thought I was!! It returned:

"John Dalsgaard/12345/Org"

and this is where some clever people who helped me (Paul Withers and Nathan Freemann) figured the problem was. It turns out that the FullName field of your user MUST be in the canonical format, i.e. the above SHOULD have been:

"CN=John Dalsgaard/OU=12345/O=Org"

When I changed my user document to have the full name in that format (and the same was reflected in the $Users view) then suddenly the roles were returned correctly from @UserRoles - and the delete worked! The odd thing that made it more difficult to track down was that the edit with a user that had his/her abbreviated name registered in the authors field actually WORKED. But please note that you should also ALWAYS use the canonical name in authors/readers fields - just to be safe!

You can read more about this on Paul Wither's blog in this article - please pay special attention to the quote about formatting names.

A lesson on scoped managed beans in XPages...

john.dalsgaard (John Dalsgaard) — Tue, 01 Apr 2014 19:24:02 GMT

I learned a lesson on working with scoped managed beans yesterday. Since I may not be the only one to not know how to handle this I thought I would share. If you know already then just bear with me...

Ok, let us first set the scene:

I use a managed bean that has application scope
Within this bean I cache another Java object (not a managed bean on its own)
I always access the latter through the managed bean - at least that was what I thought
I found that I kept instantiating the "inner" Java object...?

Definition of the managed bean

This bean is set up in the faces-config.xml file as:

<managed-bean>

<managed-bean-name>Configuration</managed-bean-name>

<managed-bean-class>dk.dtu.aqua.catchlog.bean.ConfigurationBean</managed-bean-class>

<managed-bean-scope>application</managed-bean-scope>

</managed-bean>

The managed bean and the "cached" Java object

So the ConfigurationBean class will be loaded on first reference and then live as long as the application is active. There are two pieces of code within that class that are important for what I want to demonstrate. First the definition of "meta" which is the Java object that I want to cache and use in various places in my code:

private final MetaDataBean meta = new MetaDataBean();

This object is instantiated when the ConfigurationBean is instantiated. Then I have a "getCurrentInstance()" method to get a hold on the managed bean itself from other places in my code:

public static ConfigurationBean getCurrentInstance() {

// This is a neat way to get a handle on the instance of this bean in the application scope from other Java code...

FacesContext context = Util.getFacesContext();

ConfigurationBean bean = (ConfigurationBean) context.getApplication().getVariableResolver().resolveVariable(context, "Configuration");

return bean;

}

How I use the Java object in the managed bean

In short what it does is to use a variable resolver to find the object in the application scope known as "Configuration" (which we told was the name of the instance of the managed bean in faces-config.xml).

So we are all set now. I can use the "meta" class in my code like:

Person person = ConfigurationBean.getCurrentInstance().getMeta().getPersonFacade().findPerson(key);

This code simply gets to the ConfigurationBean in the application scope and uses the meta object already instantiated in the ConfigurationBean to get a facade object to find a person in the database based on a key. So I can load the meta data once and reference it again as long as my ConfigurationBean lives. This is an excellent place to implement a caching method....

The problem...

Ok, if you are like me then you may think that you don't want to write the whole line above to get to the meta data. So what I did was create a base class that implemented the following code:

private MetaDataBean meta = null;

protected MetaDataBean getMeta() {

if (null == meta) {

meta = ConfigurationBean.getCurrentInstance().getMeta();

}

return meta;

}

... and then I could just use this code in my class (which extended the base class):

Person person = getMeta().getPersonFacade().findPerson(key);

Same code, right? No.... - what happens in this last situation is that I now create an instance of "meta" EVERY time I call the "getMeta()" method. And that was NOT my intention - on the contrary. This is due to serialization - all of the objects above are serializable - and they should be when they are (or are referenced from) managed beans...

I found out because I have a "debug" statement in the constructor of my classes that prints a line when the class is instantiated. Very handy in this case.... - otherwise I may not have found out there was a problem yet.

To find out where it was instantiated the "extra times" I added these statements to the constructor of my MetaDataBean:

Throwable t = new Throwable();

t.printStackTrace();

This produces a stack trace that will show you what code caused the extra instantiations of the class. That should put you on the right track to find the culprit.

So what to do? Well there are two solutions to this:

Solution 1: Don't "chache" the cached object

Just always grap the object from the ConfigurationBean and you are fine. In this case the "getMeta()" in our base class would look like this:

protected MetaDataBean getMeta() {

return ConfigurationBean.getCurrentInstance().getMeta();

}

Solution 2: Make the "local" reference to the cached object "transient"

Add the keyword "transient" to the definition of the local "meta" declaration. Then you tell serialization to not try to serialize the object and just ignore it. In this case our base class would look like this:

private transient MetaDataBean meta = null;

protected MetaDataBean getMeta() {

if (null == meta) {

meta = ConfigurationBean.getCurrentInstance().getMeta();

}

return meta;

}

You could see another use case for the "transient" keyword in this video by Tim Tripcony on value binding. And finally thanks to Nathan Freeman for the help to nail down this issue.

Deploy an Eclipse update site to IBM Domino and IBM Domino Designer

john.dalsgaard (John Dalsgaard) — Wed, 15 Jan 2014 09:58:47 GMT

This article is the second using third party JAR files as plug-ins in your Domino environment. See the first article about wrapping an existing JAR file into a plug-in and the third article about security and permissions .

To recap: Now you have your own Eclipse update site (as we built in the previous article) - or you could have downloaded another Eclipse update site. The steps to deploy them are the same!

I will continue using the example from the first article where we want to provide Apache Commons Lang as a plug-in for our applications.

Steps we need to perform

Build the local update site with the plug-ins as features in Eclipse (done in the first article )
Import the local update site into the Domino update site database
Deploy the update site database to a Domino server and Domino Designer

This article will describe the second two parts of job, importing the local update site and deploying it to a Domino Server and Domino Designer.

Import the local update site into a Domino update site database

We need to create database to "hold" the Eclipse update site (also called "local update site") in the Domino infrastructure. This is a database that can be replicated between servers as any normal Domino database.

1. Press [Ctrl] + N (or select File / Application / New...).

2. Then check the "Show advanced templates" and find the "Eclipse Update Site (9)"

3. Give the update site a name and a title and press [Ok]

I have developed the habit of putting my update sites in a single directory. I find that gives a good overview - but you don't have to.

Now, the update site is ready for us to use for distributing our local update site with the plug-in. If you haven't already done so, open the update site database

4. Select to import a local update site:

5. Navigate down to the site.xml file in the Eclipse update site project and import it:

If you put the local update site project in the “default” location then you will find it under the “workspace” folder in your Notes data directory.

And now you have imported the feature:

6. Finally, you should sign all the content in the update site database by selecting “Actions / Sign all content” and press “Yes” to the following confirmation dialogue:

Now we are ready to use our new plug-in in the Domino environment

Deploy the update site database to a Domino server and Domino Designer

This task consists of two steps:

Configure Domino to use an update site
Configure Notes/Domino Designer to use an update site

First, let's configure Domino - and after that, we will configure Domino Designer to use the plug-in

Configure Domino to use an update site

This is simply controlled by setting a variable in notes.ini called: OSGI_HTTP_DYNAMIC_BUNDLES

You can enter more than one update site separated by commas. However, there need to be an update site with the name(s) specified in this variable

when you start the server or it will throw an error.

Go to your Domino Directory on the server and open configurations. If you do not already have a configuration document for the server in question you

need to create one.

1. Open the configuration and select "NOTES.INI Settings"

2. Select Set/Modify Parameters and enter the name of the variable: OSGI_HTTP_DYNAMIC_BUNDLES (it is NOT present in the list – but typing it in works fine)

and press “Add/Update” for the variable to be changed and save the configuration settings document.

3. Now restart the server for the changes to take effect (you need to restart the server when you have changed the list of update sites).

[18210:00002-1956095776] 10-01-2014 11:54:14 JVM: Java Virtual Machine initialized.
[18210:00002-1956095776] 10-01-2014 11:54:14 HTTP Server: Java Virtual Machine loaded
[18210:00002-1956095776] 10-01-2014 11:54:14.75 CSRF Init: iNotes_WA_Security_ReturnUrlCheck> c_CSRFReturnUrlCheck: 1
[18210:00002-1956095776] iNotes Init: Credential Store Configuration not enabled, less secure mode
[18210:00002-1956095776] 10-01-2014 11:54:16 HTTP JVM: CLFAD0330I: NSF Based plugins are being installed in the OSGi runtime. For more information please consult the log
[18210:00002-1956095776] 10-01-2014 11:54:26 XSP Command Manager initialized
[18210:00002-1956095776] 10-01-2014 11:54:26 HTTP Server: Started

Notice the “… NSF Based plugins are being installed in the OSGi runtime…” message. To see what has been installed you can type the command “tell http osgi ss” and it will show you the entire list of all plug-ins loaded. Typically, you will want to filter this a bit. This is simple – you just append the start of the full name of the plugin… Say, if I want to show just the plugins beginning with “dk” I would write:

te http osgi ss dk

[22997:00002-3437475616] 10-01-2014 13:55:22 Framework is launched.
[22997:00002-3437475616] 10-01-2014 13:55:22 id State Bundle
[22997:00002-3437475616] 10-01-2014 13:55:22 13 <<LAZY>> dk.dalsgaarddata.apache.commons.lang3_1.0.0.201401101344
[22997:00002-3437475616] 10-01-2014 13:55:22 22 <<LAZY>> dk.dtu.aqua.ext_1.0.1.201401021840

That’s it! Now these plug-ins are ready to be used on the server. Please do remember to remove them from the jvm/lib/ext library if you (like me) already had them deployed that way….

If you later change/update the plug-ins then you will have to restart http on the server for them to become effective. On the server console type: “restart task http” (if you just use “tell http restart” you will NOT have the OSGi bundles re-loaded)

Configure Notes/Domino Designer to use an update site

The best way to achieve the update is by using a “provisioning widget”. But first you want to copy the url to the update site to the clipboard.

1. Open the update site database and select “Actions / Show Urls…”:

4. Paste the url into the field and press the “Load” button.

5. Select the feature(s) and press “Next”. Then provide a meaningful name and a description and press “Finish” (or leave what was set in the feature project)

6. Then confirm that you want to install the plug-in:

7. Finally, Notes needs to be restarted for the plug-in to become effective

And you are ready to use your plug-in in Designer.

Once you have restarted Notes you can check what plug-ins have been installed. Select File / Application / Application Management – and expand the Notes\workspace\applications\eclipse section (typically the second) and there your feature should be shown.

You can now start using the functionality in a database. Open a database in Domino Designer and select the Xsp Properties (available at the bottom of the outline under “Application Configuration” if you have opened the XPages or Domino Designer perspective). Select the “Page Generation” tab:

and tick the new plug-in to enable it and save the properties.

Now create a Java class and use the functions of Apache Commons Lang3:

package dk.dalsgaarddata.demo;
import org.apache.commons.lang3.StringUtils;;

public class Test {

public void capitalize(String x) {

System.out.println(StringUtils.capitalize("great"));

}

Deploying plug-ins through the use of widgets makes it incredibly easy to distribute the plug-ins to your developers (or Notes users) by using the standard features to deploy widgets - either through sending an email with the widget or by using policies.

Happy coding!

References

The XPages Extensibility API Developers Guide: http://www-10.lotus.com/ldd/ddwiki.nsf/dx/Master_Table_of_Contents_for_XPages_Extensibility_APIs_Developer_Guide

Background on plug-ins, features and update sites on Domino: http://yaport.pro/samples/sandbox.nsf/ByDateNJ/404B644224E898DA8525729F0060277B/$FILE/ID211.pdf?OpenElement

How to build an update site: http://wiki.eclipse.org/Create_and_Export_MySQL_JDBC_driver_bundle

How to deploy updates on Domino: http://www-10.lotus.com/ldd/ddwiki.nsf/xpAPIViewer.xsp?lookupName=XPages+Extensibility+API#action=openDocument&res_title=XPages_Extension_Library_Deployment&content=apicontent

How to install plug-ins via widgets into Notes/Domino Designer: http://www.openntf.org/blogs/openntf.nsf/d6plinks/CGUN-9DMR6W (from 12:00 and the next 6-7 minutes)

Wrap an existing JAR file into a plug-in

john.dalsgaard (John Dalsgaard) — Fri, 10 Jan 2014 09:52:37 GMT

Have you ever wanted a smooth way of deploying existing Java code in the form of JAR files to your own code? You can manually copy these files into the Java runtime environment on your servers and clients - however, this approach quickly gets messy. Why not use the existing functionality available to do this? In the Java world you would use update sites to perform the task and load the plug-ins automatically on start (server and client). No fiddling with the physical directories on the disk of computers where you deploy the code.

Follow these steps to deliver a JAR file using an Eclipse update site on the IBM Domino/XWork server and in IBM Notes/Domino Designer. For the purpose of this article I want to provide Apache Commons Lang as a plug-in. I have downloaded the latest binary version, unpacked it, and now I want to distribute the jar file with the binaries along with the javadocs to be used in my applications.

Steps we need to perform

Build the local update site with the plug-ins as features in Eclipse
Import the local update site into the Domino update site database
Deploy the update site database to a Domino server and Domino Designer

This article will describe the first part of the job, building the local update site with the JAR file. In the second article I will show how to deploy the local update site to IBM Domino/XWork server and IBM Notes/Domino Designer. In the third article I will discuss some security issues and how to work around them. This article is already way too long - now I have warned you!

Build the local update site in Eclipse

This task consists of three steps:

Create a new plug-in project from existing JAR files:
Create a new feature project to "contain" the plug-in project
Create a new update site project to "contain" the feature project

This can seem a little overwhelming but that is the way you have to do it. Feel free to delete the feature and update site projectes if you think they "clutter" your workspace - they are easily re-created if you need to update your plug-in. This guide on Creating an XPages Library by Maire Kehoe is your most important source for how to do this.

To make sure we have access to all the plug-ins and the Notes environment when we develop this code, we are going to use Domino Designer. We are just going to use the "Java" perspective which you open by selecting "Window / Open Perspective / Java".

Create the plug-in project(s)

First, we need to add the jar file(s) to a plug-in project. Then we will need to prepare the plug-in project to be installed as a library so we can "enable" it (in the Xsp Properties) for the applications we want to use it in.

1. Create a new plugin project from existing JAR files (right-click / select New / Other... and type "plug"):

2. Next - include external JARs

3. Name the project using a fully qualified name. Specify the plug-in version (I selected the version of the JAR - append ".qualifier" to get a timestamp and target as the OSGi framework (Equinox). I also deselected "Unzip the JAR archives into the project" and finally you may select a working set to organize the projects:

A note on naming your projects/plug-ins:

If you want to ensure that the name of the plug-in does not conflict with other plug-ins. Therefore, I will recommend that you use your on domain/company name as I have done in this example. I did first create the plugin calling it "org.apache.commons.lang" - it works, but you are better off using an identifier that you control

Click "Finish" and now the system will ask you whether you want to switch to the Plug-in Perspective:

You do not have to do this (I don't) so you can just press "No".

Now you need to configure a couple of options on the project and create a small bit of Java code to make the plug-in visible to your XPage/OSGi frameworks. Most of this will be handled in the editor for the MANIFEST.MF file in the META-INF folder of your project. So when I later refer to the "Overview" this is just the editor holding the MANIFEST.MF file.

4. First you set the Execution Environment to JavaSE-1.6. This includes the Extension Library which has some classes we need to finish the job:

5. Right-click on the the project name in the outline and select "Build Path / New Source folder..." and give it the name "src":

6. Press "Finish" and then right-click on "src" and select "New / Other..." and find "Package" and give it the name you have given the project:

7. Now back to the "Overview" where you create an Activator by clicking on the link under "General Information" (make sure the package name is the one you just created - you can press [Browse...] to easily select it). Name the class "Activator":

8. Now there is a compilation problem with the Activator. Click in the left side on icon marking the first error:

... and select the first option to add "org.osgi.framework" to the imported packages.

9. Click on the next error and select the first option to "Add unimplemented methods":

10. Finally add a line to give you the plug-in id in a single place (just below the "public class Activator..." definition):

public static final String PLUGIN_ID = Activator.class.getPackage().getName();

Save the Activator class.

11. Now back to the "Overview" where you select to "Activate this plug-in when one of its classes is loaded" - this is lazy loading and - and "This plug-in is a singleton":

12. Then you switch to the "Dependencies" tab and select "Add" under "Required Plug-ins". Start typing "com.ibm.xsp" and select "com.ibm.xsp.core"

Save your project, i.e. the MANIFEST.MF file (where you see "Overview") for the build path to be updated - otherwise the next step will not find the super class when you type the start of the name...

13. Right-click on the package under "src" and select "New / Class". Name the class "XspLibrary". Select "Superclass" and start typing "com.ibm.xsp.abstractx" and select the class "AbstractXspLibrary":

14. Click Ok to select the super class and complete the definition of the class:

15. Click "Finish" to open the class for editing. I suggest you add a few lines to the XspLibrary class:

package dk.dalsgaarddata.apache.commons.lang3;
import com.ibm.xsp.library.AbstractXspLibrary;
public class XspLibrary extends AbstractXspLibrary {

public static final String LIBRARY_ID = XspLibrary.class.getName();

public XspLibrary() {

System.out.println("Loading Apache Commons Lang3 (" + getPluginId() + ")");

}

@Override

public String getPluginId() {

return Activator.PLUGIN_ID;

}

public String getLibraryId() {

return LIBRARY_ID;

}

The "System.out.println(...)" line in the constructor will print to the server console once the library is loaded by the OSGi framework. This can be handy - and at least nice to know. The string returned by the "getLibraryId()" method is the text that will be shown in the Xsp properties in your NSF database when you need to enable the plug-in later on.

16. Now we need to contribute this plug-in as an extension. In the "Overview" you select the tab "Dependencies" and add another required plug-in called "com.ibm.commons". You wan to make sure that this plug-in is moved up over "com.ibm.xsp.core" (by using the "Up"/"Down" buttons):

17. Now back to the "Overview" where you select the "Extensions" link on the right side. Confirm that you want to display the extensions and extension points:

18. Then add an extension "com.ibm.commons.Extension":

19. and mark the "(service)" under the com.ibm.commons.Extension when you are back on the Extensions tab. Then enter in the fields on the right (under "Extension Element Details"):

type: com.ibm.xsp.Library

class: (full name of your XspLibrary class)

Note that this information is saved in a file named "plugin.xml" (though you do not have to bother with that since it is also edited through the "Overview").

There is a very important last step. Since we added a source directory and some code that we need the outside world to see, we have to add a "." to our classpath - otherwise the XspLibrary will not be visible outside the plug-in project - and then it will not show up in the Xsp properties of an NSF (effectively making it useless since it can never be enabled....).

Remember to add your package as an exported package - otherwise you will not be able to use it from your code (which was the whole idea of this exercise). You do not need to do this if you specified a package name that already existed in the jar(s) you added (but you shouldn't use this approach). In our example I selected my own package name and therefore need to add the "dk.dalsgaarddata.apache.commons.lang3" package.

20. Go to the "Runtime" tab and add a "New" to the classpath, enter a "." (dot - without the quotes) and move it to the front using the "Up" button - and add your package name:

21. The final thing is to update the class path for the plug-in (that will get rid of a warning that you may have seen and make sure the plug-in is ready). On the "Overview" tab select the link "Update the classpath settings" at the bottom. Then you save the project:

Finally, our plug-in project should be ready to be added to a feature etc.

Now we are ready to create a feature project

1. Right-click on a project and select "New / Other.... / Feature Project":

2. Next - give it a fully qualified name (I use the same as my plug-in project with ".feature" appended), feature name, version (remember to add ".qualifier" to the version name), etc.

3. Select Next - and select the plug-in project(s) for the feature

You may be asked to open an associated perspective. Press "No" for now. You will not need it:

4. You then need to go to the "Plug-ins" tab of the feature.xml file. Here you have to specify that the plug-in should be unpacked after installation. This is important as Domino Designer cannot see the classes in the jar if you don't do this. The jar(s) will not be unpacked on the server - so no extra overhead there.

5. Now the feature is in. I recommend that you enter some information in the "Information" tab - though this is not strictly necessary. However, this is what others see when installing your plug-in:

6. The final step is to create an update site project. You can do this directly from the Overview of the feature

7. Give the update site a meaningful name. I use the same approach as for the feature - just adding ".update" at the end:

8. Now add a feature. You may want to add a category as well where you can put the feature under. This makes really good sense if your plug-in will be added to an update site databse with more features. Here I add a category (the "name" is what users will see - do not use spaces):

9. Select the category and press "Add Feature..."

10. Now the sites.xml of the update site is almost ready....

11. ... but very importantly, you need to "Build all" to generate the update site...!! This will create the "plugins" and "features" directories and populate them with the relevant information:

If you later need to re-build the update (say if you made some changes to your plug-in project) then you should delete these two folders with their content to avoid having several versions of the same plug-in/feature in the update site.

Working sets

If you do not see the feature and update projects in your working set you just have to select the "window menu" (the three horisontal bars) and "Deselect Working Set"

Then you can find your project, right-click on it, select "Assign Working Sets..." and add it to the working set where you created your plug-in project.

Now, you are ready to use your update site in Domino and Domino Designer.

Happy coding!

Update - 2014.09.26

It has come to my attention that my own environment cheated a little upon me. To make sure that you can use the plugin in other Domino Designers than the one having the plug-in, feature and update site projects installed, you need to unpack the plug-in after installation. I have added this to the creation of the feature project as a new item 3.

Thanks to Cameron Gregor for pointing out (see this article on StackOverflow ), Serdar Başeğmez for testing and Declan Sciolla-Lynch for explaining the details.

/John

MVC and XPages

john.dalsgaard (John Dalsgaard) — Thu, 28 Nov 2013 20:00:29 GMT

Here is my presentation from the DanNotes conference. See how to use the Model-View-Controller (MVC) design pattern with IBM XPages.

PDF: DanNotes MVC in XPages

On S lideShare

Custom login view in IBM Domino

john.dalsgaard (John Dalsgaard) — Wed, 31 Jul 2013 18:55:16 GMT

Ok, this is not new information - actually, it is so old that I had almost forgotten and had to fiddle around with some details to make it work. Therefore, I thought I might as well describe the process for everyone else to avoid wasting time on it.

The situation is that you want to customize what information you want your user to use to identify themselves. It could be an employee id or a username that the user select themselves. In a standard Domino directory you have the option to select (under the security tab in the server document):

More name variations with lower security
Fewer name variations with higher security

If you look at the design of the Server form you will see that the field HTTP_AuthView is controlling this setting. Depending on the value you select this field will hold either "($Users)" or "($LDAPCN)" (respectively). These values point at two views that control what information the user can use to identify themselves.

Now, you have a couple of options to control the login identification:

Use the formulas in these views to find out what information you should put into what fields in the person documents to obtain the desired identification. Many do this - and you don't have to change any design
Add another option to the HTTP_AuthView field that will point logins to another view. Remember this view must be available in all directories if you are using Directory Assistance
Create your own view in another database and include it via Directory assistance

The last two options require the same considerations from you. I have an application that will allow users to register themselves and use the information for login. I do not want to use our primary directory or a secondary directory for this purpose. The application will know the user object (yes, a lot of Java beans here.... - but that is another story) and, therefore, there is no need to duplicate this information into a directory etc.

So I created a view in my application, named it something suitable like "UserLogin" but gave it two aliases (separated by the pipe character): "($Users) | ($LDAPCN)". This way the server will find exactly this view independently of how the server document was configured. Of course you also have to set up Directory Assistance for this extra "directory" (which is my application) - and remember to set it to Yes for "Trusted for credentials"

Now back to the view:

The first column should contain the information that you want users to use to identify themselves. You can use multivalues - just mark the column to "show multiple values as separate entries". This column must be sorted.
Then you need a column that has the programmatic name "FullName" - you can put whatever information you want into the field as long as you ensure it has this name - but you MUST ensure that the name is in the "canonical" format (see more in this article ).
Finally, you need the HTTPPassword column

And that is it!

A quick note on the HTTPPassword field.

Calculate it when saving your user document using the formula: @Password(Password) - if your password field is called "Password" (else you will have to use the name of your own field).
If you want to enable the more secure internet passwords (normally configured in the Directory Profile in an ordinary Domino Directory) then you should add a field called "$SecurePassword" and set it to the value "1" (text) in your user document. This means that you will have to evaluate the @Password(..) command against the user document if you are creating/updating the information through code (which obviously you do if you use Java beans).

f you have followed along you now have your own directory service where you have full control over the way the users identify themselves - and it is fully integrated with your application.

Java security in IBM Domino

john.dalsgaard (John Dalsgaard) — Thu, 23 May 2013 18:51:43 GMT

It has come to my knowledge that what I considered "old" (i.e. well-known) knowledge may not be that know to many people.

For some Java code to be allowed to run on a Domino server you may need to add some security settings to the java.policy file (e.g. allow access to external network ressources). This file can be found in the jvm/lib/security library. However, whenever the Domino server (and the Notes client for that matter) is upgraded (even small upgrades) this file will be overwritten!!

So you have to add your adjustments again - and as a developer you may not even know that the server was upgraded (say from 9.0 to 9.0.1). Well, this is where you will be interested in another file in the same directory: java.security. This file contains a couple of interesting lines:

# The default is to have a single system-wide policy file,
# and a policy file in the user's home directory.
policy.url.1=file:${java.home}/lib/security/java.policy
policy.url.2=file:${java.home}/lib/security/java.pol
policy.url.3=file:///${user.home}/.java.policy

Please note the second policy.url line. That means that you can add a file to the directory called " java.pol ". It has the same syntax as java.policy. But the main difference is that it does not get removed on upgrades....

Updated: 25-06-2014

Mark Leusink had an issue with the above and went the extra step to verify that when you do an upgrade to Domino 9.0.1 FP1 the entire jvm/lib/security is replaced (due to an upgrade of the JVM). This will also be a problem with 9.0.2 (not released yet). So please back up your java.pol file before upgrading to either!!

You can see Mark's findings on his blog .

According to others who have run into this same issue it seems only to be related to Domino servers running on Windows.

Using XPages? - want to disable Dojo?

john.dalsgaard (John Dalsgaard) — Thu, 23 May 2013 18:47:42 GMT

This is a small very technical tip. It has been documented elsewhere. However, today I came across someone who did not know and, therefore, I decided to write it here.

If you are developing XPages using IBM Domino Designer from time to time you may want to stop the loading of the "fat" Dojo libraries. You may have decided to use jQuery Mobile for your mobile solution (as I do for a mobile app ). On the other hand you may not want to disable Dojo for the entire application (in the xsp.properties file). So what do you do?

You can add the following SSJS code to the beforePageLoad event of your XPage:

facesContext.getRequestParameters().setJsLibrary(0);

... and you are done!