Blog

Dalsgaard Data A/S fylder 25 år!

Du har netop installeret en ny Domino Designer og vil udvikle XPages....

Du kan få adgang til den næste store opdatering af HCL Notes & Domino, som er i beta nu!

I have an XPages app that has an interface to a native mobile app (running on Android and iOS). The app can be used as a "full" client to the application - so it both reads and writes data to the Domino based application. As you probably know we have some extra keyboards on the mobile devices - which allows us to express our emotions. They are called emojis or simply smileys 😉 . Normally when we see these special characters in IBM Notes (or web based applications running on IBM Domino) they are translated to something that may not always be visible... - or more correctly, shown as a series of hieroglyphs. Now, one of our users entered a description on his cell phone and included a smiley and submitted it to the server - and it just came back "awkward" - so I decided to try and figure out how to cope with these so often-used special characters. The journey to the result was long - and I learned a lot about unicode - and didn't understand everthing (e.g. "high surrogate" and "low surrogate"). And in the end it turned out that I didn't need to know all this!! I use an MVC model where I read data into Java objects and after that all data handling is done through the Java objects. A Java String is naturally in unicode (actually it is the "utf-16" character set that uses two bytes per character whereas "utf-8" only uses one byte per character). In Denmark we have three extra national characters (compared to English) and for quite a while I have used "utf-8" to encode any data that needs to be sent to/read from other systems (e.g. in a REST service). This has proved to work well - that is, until now. What I normally do is something like this: HttpServletResponse response = (HttpServletResponse) FacesContext.getCurrentInstance().getExternalContext().getResponse(); ResponseWriter out = FacesContext.getCurrentInstance().getResponseWriter(); response.setContentType("application/json"); response.setHeader("Cache-Control", "no-cache"); response.setCharacterEncoding("utf-8"); try { out.write(gson.toJson(data)); } catch (Exception e) { e.printStackTrace(); } finally { // Stop the page from further processing; FacesContext.getCurrentInstance().responseComplete(); } The "gson" variable is just a "GsonWrapper" as I have written about earlier . The "data" object is the Java object containing the data that I want to send out on the wire (and GSON converts that to JSON for me). The bean where this code is called is placed in an XAgent (i.e. an XPage that does not render any output): JSON Service NB this page is a placeholder for returning all different data objects as JSON. Therefore, this page is NOT rendered.... So far I have had the call to the processing function in the bean in the "afterRenderResponse" event of the XPage. This turns out to be important. And I cannot remember why I ended up calling it there instead of in the "beforeRenderResponse" event... This concept has served me well - it shows all the special characters and performs well. That is until someone sent a smiley... 😀 The problem is that there are not enough bits in "utf-8" to show all of the characters in the world. However, there are som built-in magic that can use two bytes to represent a character ("utf-16" uses two bytes to show all characters - just like the good old Lotus double byte character). The beauty is that we normally don't have to care about it - the programming tools that we use will normally seamlessly handle this. However, we need to work with the "bytes" as opposed to "characters" to allow the seamless handling. So we need to send a byte stream as opposed to a String back as a response to the http request that wanted to read some data from our service. To make this happen we need to use a "DataOutputStream" instead of the "ResponseWriter" that I used above. Our code to send the response will now look like this: HttpServletResponse response = (HttpServletResponse) FacesContext.getCurrentInstance().getExternalContext().getResponse(); response.setContentType("application/json"); response.setHeader("Cache-Control", "no-cache"); response.setCharacterEncoding("utf-8"); try { DataOutputStream out = new DataOutputStream(response.getOutputStream()); String json = gson.toJson(result); byte[] utf8JsonString = json.getBytes("UTF-8"); out.write(utf8JsonString, 0, utf8JsonString.length); } catch (Exception e) { e.printStackTrace(); } finally { // Stop the page from further processing; FacesContext.getCurrentInstance().responseComplete(); } But if you had the XAgent the same way as I had then you will see an error on the server console: HTTP JVM: java.lang.IllegalStateException: Can't get an OutputStream while a Writer is already in use Now what is that about??? Well, if your XAgent calls the bean to handle the service in "afterRenderResponse" then a "Writer" has already been initiated. So you need to call it in "beforeRenderResponse": JSON Service NB this page is a placeholder for returning all different data objects as JSON. Therefore, this page is NOT rendered.... And that seems to be it!!! I have been over considerations about how to save data in NotesItems (as MIMEentities instead of a pure NotesItem) - and converting the emojis as per the article about MySQL (see below) - but none of this seems necessary!! - in my environment anyway. And now our users can express emotions when describing the fish they caught (its an angler's app) - without breaking anything and without us having to remove the funny little characters! Happy coding!! Sources that helped me understand this problem and find the solution: Hacking utf16 to work around MySQL's utf8 limitations How to stream file from xPages? GSON not sending in UTF-8 Java exception: “Can't get a Writer while an OutputStream is already in use” when running xAgent XPages: The Outputstream and binary data

This is the third article related to wrapping third party jars into OSGi plugins. I have meant to write this article for a long time but.... - well, now was the time for it. The previous articles in this series are: Wrap an existing JAR file into a plug-in Deploy an Eclipse update site to IBM Domino and IBM Domino Designer Depending on the jars you want to use in your application you may run into problems with permissions. This is part of Java's security model - which is different from what we are used to in the Domino world. We have ways to control it via the java.policy file in the jvm (or better use the java.pol file as I have written about before). However, this requires changing files in the filesystem - and in many cases these changes can/will be overwritten by an upgrade or even installation of fixpacks. That is not good for the stability of our application as we may not be notified of this upgrade in advance - and suddenly the application does not work.... - and the problem most likely ends up on your desk. So what can we do to get around this and control it without having to set security parameters that are global to the entire JVM? I had this challenge when Nathan Freemann showed me how to do this in a way that is entirely controlled within my application - which I by far prefer . Basically, you need to create a wrapper to the third party code so that you can wrap the privileged calls to allow "elevation" without having to use the permissions in the java.policy file. First you create the wrapper class - in this case I call it "GsonWrapper". I also added a constructor with the possibility to use another date format - as I sometimes have different requirements based on the client (especially native vs. mobile web). public class GsonWrapper { private static final String DFT_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss"; private Gson gson = null;/* Constructors... */ public GsonWrapper() { this(DFT_DATE_FORMAT); } public GsonWrapper(final String dateFormat) { gson = new GsonBuilder().setDateFormat(dateFormat).create(); } } Now to the most interesting part. The way we can make a privileged call is by using an AccessController: public String toJson(final Object obj) throws Exception { AccessController.doPrivileged(new PrivilegedExceptionAction() { public Object run() throws Exception { GsonWrapper.this.toJsonImpl(obj); return null; } }); return result; } ... which requires us to implement a method that we can call from within the doPrivileged block to do the actual privileged call to the underlying jar: public void toJsonImpl(Object obj) { result = gson.toJson(obj); } The result is stored in an instance variable ("result") that builds the "gap" between the implementation method and the wrapper. Sort of the same way you have to do it if you dynamically build and execute your own LotusScript code using "Execute" and want some data back 😊 And that is it! You are now ready to use the wrapper and that could look like this: // Requires Gson JAR to be put in the jvm ext library... - and AllPermissions set in java.pol/java.policy !!!! Gson json = new GsonBuilder().setDateFormat(ISO_DATETIME_FORMAT).create(); // 1 // ... alernatively wrapping it in an OSGi plugin (with AccessController.doPrivileged(....)) GsonWrapper json = new GsonWrapper(ISO_DATETIME_FORMAT); // 2 out.write(json.toJson(data)); In this simple example you need to either use the first declaration of "json" (marked "1") OR the second (marked "2"). The code obviously will not compile as it is now 😉 The way I have built this you can also see that only the declaration determines if we use the wrapper. I have kept the same method and arguments as in the underlying jar - which makes the code less dependent on the wrapper. The "data" variable can be ANY Java object structure (single object, list/array of objects, etc) - and it all transforms nicely to the equivalent JSON structure. Very cool! The only issue I see with this method is that you will have to create all the methods in the wrapper that you potentially need from the underlying jar - with each of the method signatures (argument lists) that you want to use. Happy coding! Entire source code for the wrapper (including toJson and fromJson): package dk.dalsgaarddata.ext.gson; import java.lang.reflect.Type; import java.security.AccessController; import java.security.PrivilegedExceptionAction; import com.google.gson.Gson; import com.google.gson.GsonBuilder; /** * @author jda * @version 1.0.0 * * Date: 2015.09.20 * * This utility class wraps a call to Gson into a privileged call to allow elevation without having * to grant "allaccess" to all jars in the jvm/lib/ext * * This class needs to be exported to a jar file and placed in the jvm/lib/ext library for it to be able handle * the priviledged calls - or I can be included in the OSGi plugin :-) */ public class GsonWrapper { private static final String DFT_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss"; private Gson gson = null; private String result = null; private Object userClass = null; /* Constructors... */ public GsonWrapper() { this(DFT_DATE_FORMAT); } public GsonWrapper(final String dateFormat) { gson = new GsonBuilder().setDateFormat(dateFormat).create(); } public void toJsonImpl(Object obj) { result = gson.toJson(obj); } public void fromJsonImpl(String obj, Class cls) { userClass = gson.fromJson(obj, cls); } public void fromJsonImpl(String obj, Type type) { userClass = gson.fromJson(obj, type); } @SuppressWarnings( { "unchecked" }) public String toJson(final Object obj) throws Exception { AccessController.doPrivileged(new PrivilegedExceptionAction() { public Object run() throws Exception { GsonWrapper.this.toJsonImpl(obj); return null; } }); return result; } @SuppressWarnings( { "unchecked" }) public Object fromJson(final String obj, final Class cls) throws Exception { AccessController.doPrivileged(new PrivilegedExceptionAction() { public Object run() throws Exception { GsonWrapper.this.fromJsonImpl(obj, cls); return null; } }); return userClass; } @SuppressWarnings("unchecked") public Object fromJson(final String obj, final Type type) throws Exception { AccessController.doPrivileged(new PrivilegedExceptionAction() { public Object run() throws Exception { GsonWrapper.this.fromJsonImpl(obj, type); return null; } }); return userClass; } }

I am at the moment moving a site from standard http to https (using Nginx as a proxy in front of Domino - but that is a different story). However, I did run into a small issue. When I looked at the console in Google Chrome this message appeared: index.xsp:6 Mixed Content: The page at 'https://www.fiskogtast.dk/fangst.nsf/index.xsp' was loaded over HTTPS, but requested an insecure stylesheet '//fonts.googleapis.com/css?family=Actor'. This request has been blocked; the content must be served over HTTPS. ... and when looking at my theme I found this: text/css //fonts.googleapis.com/css?family=Actor ... and well, yes I had not taken into account that I was going to run SSL when adding that reference a while back. But I have taken on a new habit since then - now I use "protocol relative" urls. That is basically the same that we do when specifying "/db.nsf/..." in our apps as "domain relative" urls. So in this example I could use //fonts.googleapis.com/css?family=Actor . So I changed the href accordingly and refreshed my XPage.... But this did not do what I expected. Now the reference to the external resource was concatenated into an internal reference:I tried to disable "Use runtime optimized Javascript and CSS resources" in the XSP properties - but didn't seem to make a huge difference... So I did a little more ressource and found (via this link by intec ) that you can use the rendered attribute on a resource tag in a theme. So I ended up with this (not so elegant) solution -that seems to work fine: text/css //fonts.googleapis.com/css?family=Actor text/css https://fonts.googleapis.com/css?family=Actor Happy coding!

If you are like me then you really want to minimize duplication of code and reuse as much as possible. A common use case that has caused me trouble is this scenario where I have a common function that uses some context (in the form of constants). For a particular subclass I just need to redefine the constant to make everything behave like I want. Something like this: public class Base { protected final static String letter = "-"; protected void print1(){ System.out.println("Letter=" + letter); } } public class A extends Base { protected static String letter = "A"; } And called like this: A a = new A(); a.print1(); I had expected to see "A" as the output from above - but no, it is "-"... Basically, what happens is that you define a new variable in a different scope - instead of overriding the existing constant/variable... And when you think about it I suppose it is quite natural. But then how can one get around this - and keep the basic idea of what I was trying to obtain? Well, you simple use a getter - and override that. So if you do this instead: public class Base { protected final static String letter = "-"; protected String getLetter(){ return letter; } protected void print2(){ System.out.println("Letter=" + getLetter()); } }public class A extends Base { protected static String letter = "A"; protected String getLetter(){ return letter; } }And called like this: A a = new A(); a.print2(); ... then you get the expected "A" from the method defined in the Base class but called in the subclass A.  Perhaps very simple - but it took me a little effort to understand that I could not just override the constant/variable and had to change the approach slightly. Happy coding!!

This morning I saw a question on StackOverflow that I had a look at. Basically, the person asking had tried to update an existing calendar entry - without success. As I have been playing a little with REST I thought I would give it a quick go. And correct, it couldn't be done the way the person asking the question had tried. So let's get some basics in place: If you want to CREATE data you use a POST request. If you want to change INDIVIDUAL fields then you use a PATCH request If you want to update ALL fields in an extisting record/data set then you use a PUT request. Being armed with this knowledge I started out trying with a PATCH approach... Long story short - the calendar service seem not to allow PATCH requests. So that one is out of the question. And having said that I have not found any places in the documentation stating that it would support PATCH methods. So all fine... I then tried a PUT request and with a little playing around (and with the help of the documentation for Domino Access Services (DAS) 9.0.1 ) I made it work :-) First I specified the request with ALL fields from the result of either querying the event (or when creating it). Following subsequent tests I found that this is not necessary - but you have be aware of the defaults being added for the fields you do not specify - e.g. if you only specify a start time (as in the StackOwerflow question) you get the same end time... So you may want to specify all fields... Example of a request: { "events": [ { "href": "\\/demo\\/cal.nsf\\/api\\/calendar\\/events\\/0C3AC054CE152609C1257E9D00204F95-Lotus_Auto_Generated", "id": "0C3AC054CE152609C1257E9D00204F95-Lotus_Auto_Generated", "summary": "Rotary", "location": "Main room", "start": { "date": "2015-03-30", "time": "18:30:00", "utc": true }, "end": { "date": "2015-03-30", "time": "20:00:00", "utc": true }, "class": "public", "transparency": "opaque", "sequence": 0, "last-modified": "20150810T055255Z", "x-lotus-notesversion": { "data": "2" }, "x-lotus-appttype": { "data": "0" } } ] } And then the important thing. In the PUT request you have to specify the FULL id (not just the UNID part) like: http://json.dalsgaard-data.dk/demo/cal.nsf/api/calendar/events/0C3AC054CE152609C1257E9D00204F95-Lotus_Auto_Generated And then it works! I f you want to know more about what you can do with REST services then check out my presentation from SUTO Lon Slideshare

Ok, if you are like me you may want to play with new technologies within your well-known environment (like the IBM Notes/Domino database), e.g. an Angular.js app.  Now, if you are used to the well-known security model in Notes/Domino you may also have your database locked down so no un-authenticated users can access your database. And now you add an "app" to the WebContent folder (via the package explorer view). However, the first time you try to access your "app" it redirects to a login page. What is happening??? Well, you have been hit by a problem that is easily solved by "ordinary" design elements that you can mark for "public access". However, that is not possible for these types of design elements - without a little code.... So I have created the following LotusScript (!!) agent to do the trick. You may need to adjust the path of your app - but the principle should put you on the right track: Option Public Option Declare Sub Initialize Const APP_DIR = "app/" Const FN_PUBLICACCESS = "$PublicAccess" Const FLAG_TRUE = "1" Dim sess As New NotesSession Dim db As NotesDatabase Dim nc As NotesNoteCollection Set db = sess.currentDatabase Set nc = db.CreateNoteCollection( False ) Call nc.SelectAllDesignElements( True ) Call nc.BuildCollection Dim d1 As Long Dim d2 As Long Dim title As String Dim flagsExt As String Dim noteid As String Dim nextid As String Dim i As Long Dim doc As NotesDocument noteid = nc.Getfirstnoteid() For i = 1 To nc.Count 'get the next note ID before removing any notes nextid = nc.GetNextNoteId(noteid) Set doc = db.GetDocumentByID(noteid) title = doc.GetItemValue("$title")(0) flagsExt = doc.GetItemValue("$flagsExt")(0) If LCase(flagsExt) = "w" And Left(LCase(title),Len(APP_DIR)) = LCase(APP_DIR) Then d1 = d1 + 1 If Not doc.Getitemvalue(FN_PUBLICACCESS)(0) = FLAG_TRUE Then d2 = d2 + 1 Call doc.replaceItemValue(FN_PUBLICACCESS,FLAG_TRUE) Call doc.save(True, False) Print title & " - set to allow public access read" End If End If noteid = nextid Next Print nc.count & " design elements checked. " & d1 & " elements found in '" & APP_DIR & "' and " & d2 & " updated to give public access" End Sub