Welcome to WebmasterWorld Guest from 54.242.115.55

Forum Moderators: phranque

GPipe to build more complex neural networks

     
11:28 pm on Mar 6, 2019 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9233
votes: 780


Google’s AI department has introduced GPipe, a new library for distributed machine learning, to the neural network building framework Lingvo. It makes use of pipeline parallelism to scale up deep neural network training and get more precise systems.

[devclass.com...]
More toys for big noise... Anyone here actually using this kind of stuff?
10:06 am on Mar 7, 2019 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:25900
votes: 871


Interesting.
[ai.googleblog.com...]

I can see why this is within the Google/Alphabet family.

I suspect it's more likely to be research and development in powerful applications, such as vehicle navigation.
9:40 pm on Mar 7, 2019 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 25, 2003
posts:1214
votes: 341


@tangor: Anyone here actually using this kind of stuff?

Moi :)


I've been playing, as in hobbyist, with LSTM Networks (Long Short Term Memory Recurrent Neural Networks) the last while including Attention/Augmented, all via TensorFlo. Yes, I often don't know what I'm doing.

Voyaging Outside the Google Box [webmasterworld.com], October 2017.

While I've had great success utilising augmented TensorFlo I've never ?graduated? to Lingvo (TensorFlo sequence modelling framework) and it's libraries such as GPipe. Most of the efforts and releases in ML tend towards imagery and collaboration, while I'm focussed on text/language and onsite behaviours as a sole mucker about.
Note: Time is such an implacable constraint!
10:10 pm on Mar 7, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2409
votes: 640


I had it on my list of things to do also. But I have been focused on Py-Torch instead of Tensor flow. But I would be lying if I said that I have actually accomplished anything with this. As Iamlost pointed out:

Note: Time is such an implacable constraint!
9:08 am on Mar 8, 2019 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:May 9, 2000
posts:25900
votes: 871


iamlost and NickMNS If you get time, please tell us more about your experiments.
5:55 pm on Mar 8, 2019 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 25, 2003
posts:1214
votes: 341


@NickMNS: I never have learned Python. Plus TensorFlo was released first and I've built quite a remarkable people assistance/resource base over the past few years. And now too much invested, intertwined in production, to be worth switching. Plus I prefer lower level programming...

@engine: my first use case was bot detection. Given that I sell ad space directly a critical selling point is that visitor numbers are (1) human and (2) accurate.

With the rise of headless browsers, especially since the advent of headless Chrome, identification became increasingly problematic. Perhaps half of the 10-25% of traffic that is currently headless bots can be detected statistically; the other half requires, in my experience, using ML.

Second use case was to improve personalisation/prediction and contextual delivery.

Personalization: the automatic tailoring of sites and messages to the individuals viewing them so that we can feel that somewhere there’s a piece of software that loves us for who we are.
---David Weinberger (coauthor The Cluetrain Manifesto).

ML aids in better identifying who a visitor is, their intent, and how best to help aka prediction.
This has the contextual advantage of identifying irrelevant information for removal and additional relevant information for addition rather than simply sending the default 'page' when an in-site link is clicked.
Note: I consider it a best practice to include customisation aka user defined choice options as well as personalisation aka anticipated/predicted results. Customisation is then both an output/escape control and an input adjustment/filter.