OMERO.ms-queue (Smuggler) as an alternative scenario
There is no mention to OMERO.ms-queue (OMERO-smuggler) as an alternative import possibility. OMERO-smuggler was developed by an ex core developer of OME (Andrea Falconi) at MRI and financed by FBI with the idea af overcoming the limitations noted in the document. It has been extensively tested in production in Montpellier and it is now maintained (at least the insight part of it) by OME.
Details can be found in the OME repository and in the docs page
The system makes use of persistent queues to manage the imports outside the user's OS session as a deamon (in linux) or a service (windows). The main advantages:
- No additional training. The only thing the user sees is that the imports happen faster and he/she can just logout.
- E-mail notification. In case of success and in case of failure (in this case the list of failed files is provided and an e-mail to the configured admin e-mail address
- User can import into the desired group and dataset.
- The installation instance provides http monitoring of the import process
- It uses http requests. The smuggler deamon can be installed anywhere in the network provided that it has access to the shares (this has not been tested in production as the deamon was running on each acquisition PC)
- The "import agent" is OMERO's CLI. Nothing special server-side.
- Configurable retries. Import retries will take place at specified intervals until it is considered as permanent failure.
- It implements configurable run priorities at the OS level. That is, when properly configured, imports should not interfere with any other process in the PC.
Another advantage is that it provides some interesting evolution possibilities:
- Web import. As the import requests are http messages, it would be relatively trivial to implement web import. Meant by this, the non-triviality woudl be more linked to other considerations independent of the import process.
- The 'import agent' could be modified to be anything else that the CLI.
Limitations
- Not allowed annotating images on import. Indeed this requires the images to be already in OMERO, wo the import must be finalized. It would be possible to trigger anotehr queue message to tag after import success, but we considered this unnecessary.
- We use OMERO long-lasting sessions for this. If the OMERO server reboots, those sessions are gone and imports fail