Advice on client and server technology for a project

[horseatingweeds]

Hi, I have a simple desktop program I put together with Java which displays an image with a grid over it, allowing the user to indicate which grid contains items of interest. I feed this input to a machine learning program to create a training set which is used to classify additional images. The purpose is to identify cancer in images, but could be used for identifying features in any image. Each image the user examines, indicates, and submits increases the training set and ideally increases the accuracy of the next image.

I'd like to develop this into a web based interface. I began working on this using Java with spring framework and a Javascript library called OpenseaDragon in cahoots with a Java library called PyramidIO for deep zoom for what are rather large tissue images.

However, I'm interested in a better way of doing this, and I've been away from Java for a while, being distracted by another project. I'd also rather work with Python or C/C++ on the server side, just because those are the languages used for the technology I'm most interested in, so I'd rather invest the programming time to increase my competence for other projects.

One idea, I was thinking of using Angular JavaScript framework to drive the interface on the client side, and build a web service to feed it from the server, maybe using a Python framework like Jango as a web service layer around my C++ application, which will do all the heavy stuff.

Ideally the interface would work something like this: The user/pathologist would bring up a file browser to select images, one or more. They're going to be pretty big, tens of MBs, so they will queue up in the background and allow the user to keep working. There will be a set of controls and a list of images. The image list will indicate images ready for examination.

When the user selects an image for examination, the system will display the image, indicating the classification of each portion of the image (probably with a grid) along with the probability score, and allow pan, zoom, and select areas (or click grid boxes) in order for the user to correct the classification and submit the image for processing. The image list will show which images are currently processing on the server.

The trickiest part is probably the zoom. The selection method I've used is a grid, but the ability to make selections might be better.

Any opinions on this? Thanks!

[tangor]

Java, Javascript is one language. While rich, it is not complete and there are some shortcomings (generally negligible for the web, but might be problematic for commercial/HEALTH purposes).

What back end (database/experience/authority) is involved? And how does crowd sourcing comments for defining cancer from images?

I can see a zillion reasons why DON'T ... and also recognize the project example is a GOOD THING.

Thoughts on C/C++ in the right direction!

[graeme_p]

Java, Javascript is one language

Really not. Unless I misunderstand you quite badly.

One idea, I was thinking of using Angular JavaScript framework to drive the interface on the client side, and build a web service to feed it from the server, maybe using a Python framework

I am a huge fan of Python and Django, but I also think you should avoid fixing what is not broken.

You could use Jython and Django but you might find yourself on an unsupported combination if Jython 3 appears before Django 1.11 is EOL next year - which is probable. May be other frameworks or other ways of integrating Python and Java might appeal [wiki.python.org...]

For stuff with an elaborate user interface a cross platform app for the front end might be an alternative.

[NickMNS]

Java, Javascript is one language

Java and Javascript are two completely separate languages.

I have created several sites/web-apps using Python and Javascript. But I do not use Django and would not recommend it. It is a publishing framework that is very opinionated and not suited for what you are trying to achieve. I would use Flask which is more bare-bones than Django but has plenty of community support and add-on packages that will provide you with most if not anything you could possibly need. I would further recommend Quart.py which is an Async wrapper for Flask. Being able to carry out Asynchronous tasks will greatly improve performance (or reduce computing costs) for the type of work that you are describing, (async helps with I/O bound tasks but not for CPU bound tasks).

The advantage of using Python server side is that you can then use Python to handle some of the Data-Science and Machine Learning tasks without needing to change languages.

As for the Javascript frameworks, I generally don't use any, I stick with plain JS, and lean heavily on the Fetch/API. Fetch uses promises that are asynchronous and integrates nicely with service workers from creating progressive webapps. I would stay away from jQuery/Ajax, although it also has promises, it is not the same as js Promises, and this can complicate things if you are working with service workers.

Link to Quart: [gitlab.com...]

Note I'm also a big fan of MongoDB, the JSON like format of data integrates nicely with Python's DICT data structure and it uses GridFS to deal with large image/media files which will likely come in very handy.

[horseatingweeds]

Thanks for the comments!

Let me give further details. This project is based on a research project I did as part of my masters program where I used a classification program we are developing at the university. I started out with a proof-of-concept application using .Net and C#, only because I had the environment setup on my computer from a previous class. I used OpenCV to handle the large images, tiling, drawing, etc. I simply called the classification as an external executable -- which was handy, because how the tool works, you can fire off several instances of the executable working on the same set of tiles from an image, and the multiple instances stay out of each others way, all updating a file.

It looked promising classifying the cancer images, so next I built a better interface with Java, this time friendly to a pedestrian computer user, allowing a pathologist to work with it. I drew a grip on the imaged and allowed the user to flip the grips from benign, cancer, or neither. Each image processed added to the learning set. Each new image was classified more accurately leaving the pathologist fewer and fewer corrections to make.

This stage also looked promising, but it was way too time consuming for the user/pathologist. running multiple threads, cooking the laptop, it still took several minutes to process each image through the classification algorithms. So I'd like to make a slick web interface that lets the user upload huge images and get to work immediately -- offloading the heavy processing to a server.

Ideally the user will work on an image at their own pace (zooming, selecting) submitting each one, regardless of where processing is at on the others. The UI would also update frequently, letting the user know the state of the processing images, algorithm settings, resource use, average accuracy, and other handy components. The user would only have to run the GUI/page on their system, the server chugging away on the efficient C++ components.

I also have a few other ideas for tools and experiments that a web interface would be handy for -- making it easy for remote users to make use of resource intensive algorithms and big data on a cheap beefy server. So once I get a framework setup, I would hopefully add new tools to it.

My web development experience is with Drupal 6 and 7, Zend Framework 1, JavaScript and jQuery -- but this was quite a while ago. The past few years I've been doing enterprise integration and working on this Masters program that nearly killed me. Now I'm interested in moving into what I studied -- intelligent system -- autonomous system, AI, machine learning... So I'd prefer to keep my tool set related to that area, which mainly includes C, C++, and like Nick points out, Python. Everybody loves Python.

NickMNS, the Python setup you describe sounds interesting. I'm looking into it. Thanks.

Any other interesting ideas are welcomed.

[graeme_p]

I am amused that @nickMNS and I usually either agree completely or disagree vehemently.

In this case I agree about Python and its amazing ecosystem (I think we all do) but I disagree with Nick (again!) disagree Django and Mongo DB.

Django is not as opinionated as people think. You have to use the URL dispatcher (true of any framework) and the ORM (because everything including most third party modules are built on it). I think its a good idea to use the authentication (and it has pluggable backends so it is not restrictive). Personally, I am usually quite happy using the whole of Django almost all the time. It also has the biggest ecosystem of any Python framework.

As for Mongo DB, I far prefer the guarantees that an RDBS gives you. You get fewer bugs, they get caught earlier and do less damage. Edge cases are easier to cover and race conditions are rarer.

@horseatingweeds, that sounds like a fascinating project.