File uploads to Domino servlets (with an Angular demo)

(Updated: add required java security policy changes)

For one of my customers I’ve started working on an application that uses AngularJS for the frontend that talks to a REST API served by IBM Domino. For the REST API we decided use servlets: something that’s covered in great detail on Ed McCormick’s excellent blog. If you haven’t used servlets in a Domino database yet, go read some of his posts on this topic. Make sure you also take a look at the demo application he created containing examples of using servlets for a REST API.

One of the functions in the app (and probably in almost every app you’re going to work on too) is uploading files. Since I couldn’t find any info on how to do that with a servlet running on Domino, I decided to figure it out myself.

In case you’re only here for the code: download the demo app here. The GitHub repo is here.

Screenshot 2015-06-24 10.45.23

The demo shows how you upload files to the servlet and store it in documents in the NSF. I added a simple Angular front end that uses the angular-file-upload plugin to handle the uploads (MIT licensed). Out of the box that gives you multiple file select, image previews, progress bars and drag-and-drop. The code for the upload servlet can be found in the UploadServlet class in the database. Included in the demo database is also a very simple form showing that you can also use a

<input multiple="multiple" type="file" />

to upload multiple files in a single request. Something that (AFAIK) can’t be done on Domino if you’re going the XPages route.

My first attempt at the servlet was to use the same code I wrote a while age to process file uploads to an XAgent. That didn’t really work, because XAgents go through the XPages runtime and that does some pre-processing for you. In a servlet context you don’t get that: you have to work with the unmodified HttpServletRequest object. So I looked to see how the rest of the Java world was handling file uploads with servlets and discovered the Apache FileUpload project. That package abstracts the complexity of dealing with multipart/form-data. In a servlet context it gives you easy access to the uploaded files. So I added the required JARs to my database, read the docs and copied some sample code to get a handle on the uploaded file:

// Create a factory for disk-based file items
DiskFileItemFactory factory = new DiskFileItemFactory();
// Configure a repository (to ensure a secure temp location is used)
ServletContext servletContext = this.getServletConfig().getServletContext();
File repository = (File) servletContext.getAttribute("javax.servlet.context.tempdir");
factory.setRepository(repository);

// Create a new file upload handler
ServletFileUpload upload = new ServletFileUpload(factory);

// Parse the request
List items = upload.parseRequest(request);

The items List in the code above contains all the items in the incoming request: note that every item can be either a form field or a file. You can differentiate between the two by using the isFormField() method.

Since I didn’t copy the required imports in the Java class, I ended up with some error messages about classes that couldn’t be resolved. Clicking on the errors revealed something interesting: I could solve the error by importing the Apache packages from org.apache.commons.fileupload (from the JARs that I just added to the database), but the list also showed a second option: use the com.ibm.xsp.http.fileupload package. Apparently Domino uses the same code under the hood to process uploads. That’s good stuff and means you don’t have to import any JARs to get the upload code to work 🙂

 

Screen Shot 2015-06-22 at 14.06.48

 

The rest of the code in the UploadServlet class in the demo database speaks for itself: I get the uploaded file(s) using the Apache FileUpload class and store all files received (in a single POST request) in the same document. Since the angular-file-upload plugin sends a request to the server for every file, they all end up in separate documents. Of course you can change this behavior by adding some logic to store them all in the same document, but I’ll leave that up to you.

One thing to note: the code in the demo database requires access to the getClassLoader method in Java, so you have to allow that in your Java security policy. Best practice on a Domino server is to create a file called java.pol in the <domino install>/jvm/lib/security/ folder and add:

grant {
permission java.lang.RuntimePermission "getClassLoader";
};

It was already in my settings, because a lot of other libraries need this too.

Enjoy!

 

 

29 thoughts to “File uploads to Domino servlets (with an Angular demo)”

  1. Hi Mark,
    I am looking into servlets + REST api’s at the moment.
    After installation I notice a couple of errors in Designer e.g.

    The import com.ibm.xsp.http.fileupload.disk cannot be resolved UploadServlet.java

    How can I solve this?

    1. I’m assuming you already tried to build/ clear the application.

      What client version are you using? I’ve tested it on 9.0.1FP3 with the ExtLib installed (v10)

  2. I am up to FP4 (server) now and I get a error 500 message. this is what I read in the log:

    Exception Thrown
    java.lang.Error: Unresolved compilation problems:
    The import com.ibm.xsp.http.fileupload.disk cannot be resolved
    The import com.ibm.xsp.http.fileupload.servlet cannot be resolved
    ServletFileUpload cannot be resolved
    DiskFileItemFactory cannot be resolved to a type
    DiskFileItemFactory cannot be resolved to a type
    ServletFileUpload cannot be resolved to a type
    ServletFileUpload cannot be resolved to a type
    at eu.linqed.UploadServlet.(UploadServlet.java:22)
    at java.lang.J9VMInternals.newInstanceImpl(Native Method)
    at java.lang.Class.newInstance(Class.java:1558)
    at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:409)
    at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:389)
    at eu.linqed.factory.ServletFactory.getWidgetServlet(ServletFactory.java:48)
    at eu.linqed.factory.ServletFactory.getServletMatch(ServletFactory.java:37)
    at com.ibm.designer.runtime.domino.adapter.ComponentModule$ServletInvoker.doService(ComponentModule.java:787)
    at com.ibm.designer.runtime.domino.adapter.ComponentModule.doService(ComponentModule.java:565)
    at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doService(NSFComponentModule.java:1319)
    at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:662)
    at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:482)
    at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:357)
    at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:313)
    at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)

    Should I upgrade the client (Designer) as well?

    1. Hi Patrick,

      Might be good to update the Designer to check if it compiles. I’ll see if I can test it on a vanilla client install in a VM.

  3. Thanks very much for sharing this demo and knowledge! I tried it on my dev server and it didn’t work. But, I needed to re-do my dev server anyway, so I build a clean 9.0.1FP4 with the latest XPages extlib as well (9.0.1.v00_13_20150611-0803) and it worked! I liked the angular demo and how it created one document per file uploaded – happens to be something I need to do for a new project.

    But, what I don’t understand is the angular link – I see the component in the demos XPage but I don’t understand how that angular.html is generated or from where. How does one get his hands on that to modify the layout, etc?

    You mentioned above that it is “MIT Licensed” – does that mean it’s hidden within the .NSF demo and not modifiable?

    Thanks!

    1. Hi,

      MIT Licensed basically means that you can use it anyway/ anywhere you want, including use in commercial projects of your own. You’ll only have to include a license text. See also here.

      The angular.html file is served from the WebContent folder in the NSF: that’s a good place to store static HTML files or 3rd party libraries. Open the database in the Package Explorer to see the other stuff I added there (like Angular and the Angular File Upload package).

      1. Brilliant! Thanks for the quick reply. I always thought package explorer was just for viewing “stuff” – guess I better brush up on this piece of Domino Designer. I was looking for the HTML file in the FILES area under RESOURCES, but this is brilliant.

        Thanks for the tip!

  4. Has anyone else figured out the EXCEPTION THROWN error 500 issue yet? I had this on my old Dev server, so I rebuilt a new Dev server and it seemed to clear up. But, twice now I have been working with the demo and the error has come back. Once it shows its ugly head, that is the end of trying to use that database. I can make a new copy of the demo again and start afresh and it will work. But, for example, today I only changed the name of a label on the demo xpage and the upload would no longer work after that, giving the dreaded:

    HTTP JVM: CLFAD0211E: Exception thrown. For more detailed information, please consult error-log-0.xml located in C:/IBM/Domino/data/domino/workspace/logs
    HTTP JVM: CLFAD0246E: Exception occurred servicing request for: /fileuploads.nsf/xsp/upload – HTTP Code: 500. For more detailed information, please consult error-log-0.xml located in C:/IBM/Domino/data/domino/works

    Any ideas? Rebooting the server didn’t help. 🙁

      1. Hmm…the error above says to consult the error-log-0.xml file, which I did. Upon seeing basically the same error in that file that was thrown on the console, I thought it not very helpful. However, after you asked this morning I looked again, but this time I opened up the trace-log-0.xml. Now in the trace-log-0.xml is where it seems to keep the good stuff:

        CLFAD0211E: Exception thrown
        java.lang.Error: Unresolved compilation problems:
        The import com.ibm.xsp.http.fileupload.disk cannot be resolved
        The import com.ibm.xsp.http.fileupload.servlet cannot be resolved
        ServletFileUpload cannot be resolved
        DiskFileItemFactory cannot be resolved to a type
        DiskFileItemFactory cannot be resolved to a type
        ServletFileUpload cannot be resolved to a type
        ServletFileUpload cannot be resolved to a type

        at eu.linqed.UploadServlet.(UploadServlet.java:22)
        at java.lang.J9VMInternals.newInstanceImpl(Native Method)
        at java.lang.Class.newInstance(Class.java:1688)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:409)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:389)
        at eu.linqed.factory.ServletFactory.getWidgetServlet(ServletFactory.java:48)
        at eu.linqed.factory.ServletFactory.getServletMatch(ServletFactory.java:37)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule$ServletInvoker.doService(ComponentModule.java:787)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule.doService(ComponentModule.java:565)
        at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doService(NSFComponentModule.java:1319)
        at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:662)
        at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:482)
        at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:357)
        at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:313)
        at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)

        CLFAD0246E: Exception occurred servicing request for: /fileuploads.nsf/xsp/upload – HTTP Code: 500
        java.lang.Error: Unresolved compilation problems:
        The import com.ibm.xsp.http.fileupload.disk cannot be resolved
        The import com.ibm.xsp.http.fileupload.servlet cannot be resolved
        ServletFileUpload cannot be resolved
        DiskFileItemFactory cannot be resolved to a type
        DiskFileItemFactory cannot be resolved to a type
        ServletFileUpload cannot be resolved to a type
        ServletFileUpload cannot be resolved to a type

        at eu.linqed.UploadServlet.(UploadServlet.java:22)
        at java.lang.J9VMInternals.newInstanceImpl(Native Method)
        at java.lang.Class.newInstance(Class.java:1688)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:409)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:389)
        at eu.linqed.factory.ServletFactory.getWidgetServlet(ServletFactory.java:48)
        at eu.linqed.factory.ServletFactory.getServletMatch(ServletFactory.java:37)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule$ServletInvoker.doService(ComponentModule.java:787)
        at com.ibm.designer.runtime.domino.adapter.ComponentModule.doService(ComponentModule.java:565)
        at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doService(NSFComponentModule.java:1319)
        at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:662)
        at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:482)
        at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:357)
        at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:313)
        at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)

  5. I placed a check mark in the the xsp.properties option “Refresh entire application when design changes,” but that didn’t help. Then I read online that someone had to patch their Designer client to get it fix a similar error. I patched my 9.0.1 client to 9.0.1 FP4 and made an innocuous change to the XPage and re-saved. I saw it rebuilding the application and then when I tested it, it worked again!!

    Yay! Thanks!

    Now, I have one more question . . . how can I get this to populate the document fields with specified data. For example, I’m using the angular example to upload as I like the interface controls and the fact that it uploads only one file per document – this lets me play with Domino’s built in document controls – but I also need to populate a READERS field on the document. So if John Doe uploads 5 files, all those files are only readable by John Doe (and an Admin Role I’ll assign). But, I don’t know where to look for added data to specific fields on upload. It cannot be after the fact either for security reasons – the fields must be populated upon save.

    Anyway, I just don’t know which part of the application to look at for setting those values. I thought default values on the form would populate but they do not.

    1. The angular-form-data plugin has a ‘formData’ object that you can use to send any user entered info along with every file upload.

      To store this data along with the file in the document (or to set a readers field): take a look at the UploadServlet class: that’s where the data (the uploaded file and any additional data sent along) is stored in the Notes document.

  6. Hi Mark,

    I downloaded this excellent database to try and experiment some myself, everything seems ok but I do think I’m missing something and maybe it is on my newly locally installed Domino server. 🙂

    When I try to actually upload by clicking on upload button, I get an 500 error in the browser console stating;
    http://localhost/servlet-upload.nsf/xsp/upload 500 (Internal Server Error)

    Can you help me, what is it I’m missing?

    1. The first thing I would do is to check the exact output of the Internal Server Error using Chrome Dev tools (or similar).

      Next, have a look at the erorr-log.xml file on your Domino server to see the exact error. Maybe it can’t find the servlet class it needs.

      1. Hi again Mark,

        Crome Dev says:
        Failed to load resource: the server responded with a status of 500 (Internal Server Error)

        This is the message I get from the error-log-xml file.
        Exception occurred servicing request for: /servlet-upload.nsf/xsp/upload – HTTP Code: 500

        It seems to me there is a problem accessing the upload servlet.

        I did add the Java.pol file.
        I have checked the server document that Domino Servlet Engine is enabled.

        Is it something I miss when it comes to make sure the server supports servlet, or any access related stuff.
        Error 500 should mean the resource is missing though.

        Best Regards

      2. I also saw this !err.DesignerFacesController.Errorloadingfactoryclass0! in the error-log-xml file.
        This must mean some thing. 🙂

  7. Fixed it, if anybody else reads this and have the same problem.

    The server did not pick up the getClassLoader and it started to work after I rewrote the content of the java.pol file from scratch instead of copy and paste. 🙂

    Thanx for your help Mark.

  8. excellent piece.
    I try to add an additional field to the html. eg

    Message:

    However, I can’t figure out how to include the new field in the formData.
    Your help will be greatly appreciated.
    sorry for the dump question as I am new to AngularJS.

    thanks

    1. Hi Ken,

      You need to do 2 things:

      1. Make sure that the extra field is send along with the request to the server. See for an example the angular.js file in the WebContent folder of the demo database: in the onBeforeUploadItem event, an extra form field (‘name’) is added to the request.

      2. In the class that handles the request on the Domino server, you need to do something with that data. In the example database, class eu.linqed.UploadServlet (https://github.com/markleusink/DominoServletUpload/blob/master/nsf/Code/Java/eu/linqed/UploadServlet.java#L73-L85) you’ll notice in the highlighted section that it stores all non-file fields directly in the target document with the same name as the name that was added to the formData object.

  9. Hi,

    I’m a newbie at XPages but have been using Lotusscript in Domino for almost 20 years. I tried to implement your demo on our test server, carefully added the policy you describe, and signed all of the code and pages that I could. However, when I execute an upload the following is recorded in the error-log-0.xml file:

    values>java.lang.NoClassDefFoundError: com.ibm.xsp.controller.DesignerFacesController (initialization failure) at java.lang.J9VMInternals.initialize(J9VMInternals.java:139) at com.ibm.xsp.controller.DesignerFacesControllerFactory.createFacesController(DesignerFacesControllerFactory.java:38) at com.ibm.xsp.webapp.FacesServlet.createFacesController(FacesServlet.java:730) at com.ibm.xsp.webapp.DesignerFacesServlet.getContextFacesController(DesignerFacesServlet.java:124) at com.ibm.xsp.webapp.FacesServlet.getFacesController(FacesServlet.java:695) at com.ibm.xsp.webapp.FacesServlet.init(FacesServlet.java:86) at com.ibm.xsp.webapp.FacesServletEx.init(FacesServletEx.java:73) at com.ibm.xsp.webapp.DesignerFacesServlet.init(DesignerFacesServlet.java:64) at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:410) at com.ibm.designer.runtime.domino.adapter.ComponentModule.createServlet(ComponentModule.java:389) at eu.linqed.factory.ServletFactory.getWidgetServlet(ServletFactory.java:48) at eu.linqed.factory.ServletFactory.getServletMatch(ServletFactory.java:37) at com.ibm.designer.runtime.domino.adapter.ComponentModule$ServletInvoker.doService(ComponentModule.java:787) at com.ibm.designer.runtime.domino.adapter.ComponentModule.doService(ComponentModule.java:565) at com.ibm.domino.xsp.module.nsf.NSFComponentModule.doService(NSFComponentModule.java:1319) at com.ibm.domino.xsp.module.nsf.NSFService.doServiceInternal(NSFService.java:662) at com.ibm.domino.xsp.module.nsf.NSFService.doService(NSFService.java:482) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.doService(LCDEnvironment.java:357) at com.ibm.designer.runtime.domino.adapter.LCDEnvironment.service(LCDEnvironment.java:313) at com.ibm.domino.xsp.bridge.http.engine.XspCmdManager.service(XspCmdManager.java:272)

    Any idea what can be done to fix this?

    The Domino server is 9.0.1 FP5

    Chris

  10. Hi, first thank you for that powerfull tool! it works perfectly and i used it to make an album creation on a local website.

    The thing is if many users use it simultaneously, some files are not uploaded and some does on each side

    what could i do ? i don’t know how to check if sockets are already used or something like that, let me know how you’ll do that if it’s possible, thank you again

    1. Thanks!

      First things I would check if to see if the users get any error when uploading the files and/ or if there are any errors visible on the Domino side. Domino should be able to handle the simultaneous uploads, as well as the Angular app, so I don’t really understand why this is happening.

      Good luck,
      Mark

Comments are closed.