GPipe to build more complex neural networks

[tangor]

Google’s AI department has introduced GPipe, a new library for distributed machine learning, to the neural network building framework Lingvo. It makes use of pipeline parallelism to scale up deep neural network training and get more precise systems.

[devclass.com...]
More toys for big noise... Anyone here actually using this kind of stuff?

[engine]

Interesting.
[ai.googleblog.com...]

I can see why this is within the Google/Alphabet family.

I suspect it's more likely to be research and development in powerful applications, such as vehicle navigation.

[iamlost]

@tangor: Anyone here actually using this kind of stuff?

Moi :)

I've been playing, as in hobbyist, with LSTM Networks (Long Short Term Memory Recurrent Neural Networks) the last while including Attention/Augmented, all via TensorFlo. Yes, I often don't know what I'm doing.

Voyaging Outside the Google Box [webmasterworld.com], October 2017.

While I've had great success utilising augmented TensorFlo I've never ?graduated? to Lingvo (TensorFlo sequence modelling framework) and it's libraries such as GPipe. Most of the efforts and releases in ML tend towards imagery and collaboration, while I'm focussed on text/language and onsite behaviours as a sole mucker about.
Note: Time is such an implacable constraint!

[NickMNS]

I had it on my list of things to do also. But I have been focused on Py-Torch instead of Tensor flow. But I would be lying if I said that I have actually accomplished anything with this. As Iamlost pointed out:

Note: Time is such an implacable constraint!

[engine]

iamlost and NickMNS If you get time, please tell us more about your experiments.

[iamlost]

@NickMNS: I never have learned Python. Plus TensorFlo was released first and I've built quite a remarkable people assistance/resource base over the past few years. And now too much invested, intertwined in production, to be worth switching. Plus I prefer lower level programming...

@engine: my first use case was bot detection. Given that I sell ad space directly a critical selling point is that visitor numbers are (1) human and (2) accurate.

With the rise of headless browsers, especially since the advent of headless Chrome, identification became increasingly problematic. Perhaps half of the 10-25% of traffic that is currently headless bots can be detected statistically; the other half requires, in my experience, using ML.

Second use case was to improve personalisation/prediction and contextual delivery.

Personalization: the automatic tailoring of sites and messages to the individuals viewing them so that we can feel that somewhere there’s a piece of software that loves us for who we are.
---David Weinberger (coauthor The Cluetrain Manifesto).

ML aids in better identifying who a visitor is, their intent, and how best to help aka prediction.
This has the contextual advantage of identifying irrelevant information for removal and additional relevant information for addition rather than simply sending the default 'page' when an in-site link is clicked.
Note: I consider it a best practice to include customisation aka user defined choice options as well as personalisation aka anticipated/predicted results. Customisation is then both an output/escape control and an input adjustment/filter.