Google Cloud NEXT '17 - News and Updates

Business Value of Machine Learning and Big Data (Google Cloud Next ’17)

NEXT '17
Rate this post
(Video Transcript)
[MUSIC PLAYING] RIKU INOUE: Thanks for coming. So welcome to the customer successes with machine learning session. My name is Riku Inoue. I'm a project manager of the cloud machine learning. So today, I'd like to share some real stories from cloud customers, Google Cloud customers, who used machine learning to solve their core business use cases. So last year at Google Cloud Next conference, our chairman, Eric Schmidt, made a very bold statement about machine learning. So he basically said machine learning is the next paradigm shift of programming and it's going to be one of the strongest driving force of businesses worldwide. So I was at the conference in 2016, and I was very excited. And still, when you looked at where the machine learning applications are coming from, a lot of concentration was from tech companies. However, fast-forward one year, and we are seeing an explosion of machine learning applications across lots of industries. It's not just tech. It's from everywhere, from manufacturing, financial services, commerce.

And today, I truly feel that the transformation Eric mentioned is actually happening. So I have been talking with a lot of companies from lots of different industries. And amazingly, none of them ever told me that, oh, our business does nothing to do with machine learning. Everyone in any industry had some interesting and unique use cases they want to tackle with machine learning. Or some of them are already building the actual applications to solve their business problems. So this is just a short snippet of the lots of different needs. We're going to touch on some of these use cases, but it's already too long to go one by one in this talk, right? And to unlock these use cases, people need great tool sets that anyone can use and get the results fast. And TensorFlow has been such a tool to enable people to actually build machine learning solutions without being a researcher of machine learning. So TensorFlow is, as you know, an open source library that Google released in late 2015.

Since then, we have been seeing a tremendous amount of support and momentum from the ML community as well as people who just started to use machine learning. So in a sense, TensorFlow is really democratizing machine learning for a lot of us. And machine learning engine is a fully managed service where you can run TensorFlow training and prediction at scale on Google's cloud infrastructure. And today, we just announced the general availability of machine learning engine. And using ML Engine, what you can do is write your powerful machine learning algorithm using TensorFlow, run the training with terabytes of data– it's not just gigabytes, terabytes of real big data– and deploy the model with just one command, and serve the prediction for millions of users, and do that without worrying about building your machine learning infrastructure or maintaining that, which takes away the time. So we'd like you to basically focus on building great machine learning applications instead of managing your infrastructure.

So using these tools, using machine learning to transform your business is not a topic of future, but it's the topic of today. And in this session, I'd like to share seven stories from Google Cloud customers who have taken advantage of machine learning to apply that to their core business problems. And some of them had this problem for a very, very long time, like decades. So hopefully, you can get some ideas and inspirations about how you can use machine learning to solve your business problems or improve your apps, improve your customer experience. So our first story is from AXA. They are one of the largest global insurance firms. And they used machine learning to solve a problem for one of their core business, which is car insurance. So here's a little bit of an interesting number for car insurance. So out of all insured drivers, about 7% to 10% of drivers get into an accident. And most of these accidents are minor. It requires just a small fix-up. But 1% of all these accidents is large.

And that requires over $10,000 of insurance payment. And they call it large-loss car accident cases. And it is very important for an insurance company to understand the risk of somebody getting into such large car accidents. So what they did was to take the information that typically they use to evaluate these risks– there are 70 features they used. There are things like age range of the driver, the region, the information about their insurance policy– and feed them into a neural network that has three hidden layers and ReLU activation. And they got the amazing result of 78% accuracy, which was a huge jump from their previous method that was using random forest, which is another machine learning algorithm. And with this kind of accuracy, this basically enables better pricing for customers. Also, it opens up the possibility of better services, like real-time insurance pricing, which also benefits the customer experience. So let's take a look at the data in action in the neural network.

So they created this very cool visualization to actually showcase the neural network training in action. So on the right-hand side, there are lots of features fed into the neural network. And the lines are the connection inside the neural network. The boxes are the neurons. And the training started. And the color orange and blue means positive and negative values. And you can see the thickness representing the weight of the connection. And training goes on. And on the right-hand side, you can see the graph of test accuracy. So that is the accuracy score for the test data set that is held out from the training data set. It is very important in machine learning to segregate these two data sets. And you can see the training is converging. And after some amount of time, it gets to quite high number of 78. So that was a very cool visualization. So that was an example from insurance industry. Our next story is from Airbus Defence and Space. So many of you might know the name Airbus. And Airbus Defense and Space is a world-leading company in the space industry.

So they had this problem to detect clouds in satellite imagery for 20 years. So they are the experts in this problem, and they have been devising the solution to solve this problem for 20 years. And here's one example image– so snow-capped mountains, and there are clouds. And it's really hard to see, but here is the answer. So if you were able to tell all the pieces of clouds, then congratulations. You have very, very good eyesight. So it's very, hard even for humans. And the process naturally involves a lot of manual work to accurately tag these clouds. And as the amount of data that come through their pipeline increases day by day and at the level of 10,000 images per day, their manual process no longer can scale to sustain that workload. So what they did was to create machine learning-based, automated approach to tackle this problem. And they got an amazing result of reducing the error rate from 11% with their production system to 3% with the new approach. And what's even more amazing is the speed of their development.

So they got the first promising result with their proof of concept within the first month of their trial. So when you think about the time frame, they had this problem for 20 years. And one month of intense project, and they got a 70% reduction of errors. So you can imagine how joyful they were to find this result. And with this new approach, basically, they can use order of hundreds of hours per week of human time, the precious human time, to even more challenging task that require human processing. And here is a very brief summary of their approach. So they combined multiple convolutional neural networks that goes through different– the images go through different kind of image processing to capture different aspects of the image. And they connect these networks with fully connected neural network. So this is quite a huge network to train. So it's very computationally intensive to train. So they used ML Engine's GPU training capability to accelerate the training. And they got the amazing acceleration of 40 times speed.

So that is pretty good. And also, they used the distributed training. So in total, they used to have 50 hours of training time on their desktop machine. So that's quite a long time. You need to wait for a few days. And they reduced the time to 30 minutes on cloud ML Engine. And with that kind of time, basically you can start the job and go grab some coffee and do a small chat, come back, and then the result is already there. And you can iterate on that. So it enables a very fast iteration to improve the model quality in a very short amount of time. Also, they used the feature called HyperTune on ML engine that basically automates the hyperparameter tuning. And hyperparameter tuning, some of you may know that it is a very manual process that takes quite a lot of time. And HyperTune basically do that on behalf of you so that you can be more productive on improving your model. And the third story we have is from Global Fishing Watch. They have a very ambitious goal of preventing overfishing across the globe.

And they do that by providing transparency over the fishing activities happening across the ocean. And that's a very big-scale problem. And overfishing is actually a very severe problem that is relevant to a lot of us. Fish is a very important food source. And also, the resource is under stress. The fishing stock across the ocean is decreasing rapidly. And part of the reason why this problem is so hard to catch is understanding the activities on the ocean takes a lot of manual monitoring, and the existing approach doesn't scale. So they created machine learning-based approach and enabled watching the entire surface of the ocean. That is more than 140 square miles. Now, that is a really big number. It's really hard to grasp. So to put that into the context, it's 37 times the entire United States, which is quite a large area to monitor. And they created the model to analyze the movement patterns of vessels to classify different kinds of boats and ships, and then also understand exactly when and where the fishing activities are happening.

And here's a little illustration of different kinds of movement patterns. And as you can see, there's very distinct movement patterns. And also, there are non-fishing vessels, like cargo ships. So it's important to distinguish these as well. And their approach was convolutional neural network, and they had quite a lot of features. So 100,000 features coming into the network, because there are a lot of time points. And each time point has 11 features. And again, this takes quite a lot of computation. And again, they used the cloud ML Engine GPU to accelerate. And they got 10x speed up and cost performance. And the important piece here is the cost performance, which means basically for the same amount of dollar, they got 10x results. So when GPUs work for your model, it is not only very fast. Also, it's a very much cheaper solution to use. And as a result, they created this visualization of whole fishing activities. Each dot represents fishing happening across the ocean. And as you can see, the activity is happening across a very large area.

And also, there's very interesting patterns happening in different time windows. So that is a very cool use case of getting insights at scale. So now we're going to talk about SparkCognition, who created a ML-based malware detection service for Android. So I'd like to invite Bryan Lares from SparkCognition to share that story with us. BRYAN LARES: Hey. First of all, so my name is Bryan Lares. I'm the director of product management for cybersecurity solutions at SparkCognition. First, I want to start out with how much of an honor it is to be invited on stage with all these great use cases. I mean, tremendous innovation going on on the ML Engine platform. And we're just excited to be a part of it at SparkCognition. I want to make this interactive. So let me see a raise of hands. I'm going to look around and try to count. So there was a big story in the news in the last 24 hours where Wikileaks announced that they dropped or identified about 5,000 different hacking tools apparently created by the CIA to hack all kinds of different devices from PCs to mobile devices to even TVs.

Who read the "New York Times? or "Wall Street Journal" or– yeah, that's big, big news. The one that really struck me the most is there was a report that Samsung TVs had been hacked and used as listening devices as part of one of these hacking tools. So just mind-blowing. Let me ask a follow-up question. Who here thinks that we, as a society, need to use the power of machine learning to try to protect our devices from these kind of hacking techniques? Yeah, me too. So at SparkCognition, we've created a machine learning malware detection engine. And we've developed one for Android, or we're in the process of developing one for Android based on the Google ML platform. I believe we're the first vendor to actually use the ML Engine to create a malware detection engine for Android. Perfect. OK, so the first question you'd probably ask yourself is, why apply machine learning and artificial intelligence to cybersecurity? What's changed? Well, there's really two megatrends that are impacting cybersecurity right now that just scream for the need for machine learning.

First of all, the growth of malware. So there's a massive amount of malware created every day by hackers around the world, and it's reached a scale that's overwhelming the ability of the analyst community to identify and write signatures for those threats. Second of all, the growth of overall devices. There will be 50 billion connected devices on public and private networks by the year 2020, exponentially increasing the attack surface that needs to be defended by infosec teams. So you put those two trends together, and it just screams for the need for what we call at SparkCognition "machine scale." Our amazing marketing team has put together a short video talking about how we partnered with Riku and his team to develop DeepArmor on the Google ML platform. [VIDEO PLAYBACK] [MUSIC PLAYING] – My name's Bryan Lares. I'm the director of product management for cybersecurity solutions here at SparkCognition. DeepArmor is a machine learning-based malware detection engine that we've developed for both the Windows and the Android platform.

– DeepArmor solves the problem of detecting malware using machine learning techniques by effectively building a set of machine learning classifiers and then ensembling them in our own unique way. – It's like having a human researcher go through every single file and be able to make a determination on whether that file's malicious or benign based on the inner workings of the code. – Simply put, we're building a pipeline that starts with the labeled data, the clean and the malicious applications, and we're driving that through from future extraction all the way into building a classifier, and then also testing and then looking at efficacy. – We found three primary benefits to using the Google machine learning platform. The first is subsecond response time. We're able to analyze new threats with subsecond response time from anywhere around the world. Number two, it's allowed us to actually reduce our overall training time, which allows us to adapt to new threats more quickly.

And finally, scalability. The Google machine learning platform has allowed us to quickly scale to millions of users without having to make a large upfront investment in our [INAUDIBLE]. – The Google machine learning platform helps to solve this problem by simply giving us a distributed environment by which to train and build new models. So specifically for DeepArmor, it allows us to rely on Google's infrastructure to perform predictions that scale with minimal latency. I joined this company primarily because of the problems that we're solving and the technology that we're using to solve these problems. – SparkCognition overall is an artificial intelligence company at its heart. I was attracted to SparkCognition for solving I think the most important problem of our generation, which is augmenting human intelligence. [MUSIC PLAYING] [END PLAYBACK] BRYAN LARES: OK. All right. [INAUDIBLE] very, very much. So that's the story of SparkCognition or a little bit about SparkCognition and how we developed our machine learning malware detection engine.

We've actually brought a prototype with us that I can kind of walk you through a user story. And then I'll tell you about some of the benefits. All right. So here's the simple use case, right? And this is– your infosec teams are going to walk you through similar scenarios every single time. So by the way, the gentleman you saw in the video, Matt Brock, is here. He's our lead engineer for the for our machine learning malware detection engine for Android. So he'll be available after the discussion. So imagine that you get an email from a Matt Brock that says that he's from TurboTax, and he's noticed that there's been an error in submitting your return to the IRS. And if you just click this one download link, you'll be able to upgrade your version of TurboTax and be able to resubmit your IRS form. So me being a good citizen and really excited about my return, I will of course be one to hit that link and download the file. So lucky for me, I've actually started using the power of machine learning to protect my mobile device.

So again, this is the prototype of our Android solution. We'll probably be doing market launch at the end of Q2. But the goal is eventually, all we monitor all the files and all the file activity on that device. And any unknown files, we're able to analyze the characteristics of those files and make a prediction on whether it's malicious or benign. So lo and behold, DeepArmor identified this file as malicious with a 98% confidence rating. So our prediction algorithm returns a threat confidence score on how confident it is that file's malicious or benign. And obviously, we can go through threat remediation activities. So this is an accurate case. That file was malicious. In fact, it's the Rootnik piece of malware, which steals information from the device and downloads additional applications to that device. It also uses very advanced debugging and antihook techniques to prevent it from being reverse-engineered. So this was actually live-hitting the Google ML platform from the room here.

Can we switch back to the presentation? Awesome. Thank you so much. So here's the architecture. Here's how we've leveraged the Google ML platform. First, obviously, on the Google ML platform side, for model training, we've collected– the key for malware detection is collecting a huge, broad sample set of malicious or benign files. In the case of our Android model, we've collected over 100,000 malicious and benign files of a very broad sample set. Then we actually have– we've created three different feature extractors to be able to analyze [INAUDIBLE] three different feature vectors to analyze different characteristics within the files. Each file can literally have over 1,000 or thousands of different characteristics that we've put through the machine learning model. We then leveraged the power of TensorFlow to develop our deep neural network classifier. And finally, we deploy that on the Google ML engine to do online prediction. So then I think the second stage of that is actually what happens from your device to actually detect the malware.

So it obviously all starts with a very small endpoint agent on your mobile device. The DeepArmor agent does file monitoring and feature extraction. It then interacts with our cloud orchestration layer within our management console and to actually do threat submission. We then do ingestion through data proc and data vectorization. And then it goes through the Google ML platform with the ML Prediction API for classification, finally going back to our DeepArmor cloud orchestration layer and to our management console, and then finally to the DeepArmor agent for execution, blocking and alerting. This whole process can take less than a second to actually go from identifying a new file on a system to sending it out to our cloud layer and the Google ML platform and coming back with a prediction in less than a second, which is obviously critical for the user experience. So finally, there's three major benefits to applying machine learning to malware detection. First of all– and this is by far the most important– we're seeing detection rates four times more effective versus zero-day threats and polymorphic threats compared to traditional cybersecurity solutions.

So traditional solutions sometimes that are based on signatures see less than 20% efficacy for new zero-day threats and new hacking techniques. So machine learning's making a significant difference in that. Second is speed. As I mentioned, our model combined with the cloud ML Engine is optimized to provide subsecond threat analysis. And finally, scalability. Our load profile device agent combined with the scalability of the Google ML Engine makes it possible to apply this approach to millions of mobile and IOT devices. As an example from the efficacy standpoint, as we were developing this model, with our first feature set, which was strings, we were able to reach efficacy rates of greater than 90% on malware and clean files. When we added in our API feature sets and our permissions feature sets, we're starting to see results of over 99% efficacy for both malicious and benign files. So a significant step forward versus the traditional cybersecurity solutions. So our goal at SparkCognition is to use the power of artificial intelligence to provide more effective and more cost-efficient security to all types of devices.

And we think that this cloud-based model, partnering with Google, can be scaled to all kinds of different solutions from mobile to IoT to servers to clients. So thank you. [APPLAUSE] RIKU INOUE: Thanks, Bryan. It was an amazing demo. So going back to the slides, the next story is from financial services. So SMFG is one of the largest financial service companies in Japan– Sumitomo Mitsui Financial Group. And they partnered with JSOL, which is a GCP partner in Japan who has a lot of experience in data analytics and also financial industry. And they created ML-based credit card fraud detection system. So credit card fraud is probably something familiar. Since like last few years, I got already like two credit card fraud issues happening with my personal card. I don't know how people get that information, but this is a big problem that's relevant to many of us. And it's a huge amount of impact of $5.5 billion every year in the financial industry. And some types of frauds are very hard to detect, and they have a rule-based system in production.

They have been maintaining this system for quite a long time. When a threat or a fraud requires lots of complex rules and combinations of them, typically, it's very hard to detect and as low as 5% to 10% of the detection rate. And with that kind of rate, pretty much you need humans to intervene and have the check after the system returns the results. And they created an automated approach to basically monitor that approval results so that instead of doing the manual intervention, basically you can automatically capture these frauds. And they got amazing 80% to 90% accuracy, even for the most difficult ones that had like 5% to 10% of the accuracy. And they used deep neural network for doing this. And they estimate that it has millions of dollars of impact for their business. And the story doesn't stop here. And they are working on basically making the system adaptive by using machine learning and basically analyze the trends of the frauds and new fraud patterns happening and adapt the system.

Also, SMFG has quite a lot of ambition in terms of integrating machine learning into their core business. And one of the future efforts they are doing is to basically predict the future financial changes of companies. So understanding company financials is a very manual process, again. And it has a very long lead time as well. So every quarter, companies publicize the financial results a statement. And analysts analyze that. And then after that, the banks know, oh, some change is happening. So what they are trying to do is to basically make the process more real-time by analyzing real-time deposit and withdrawal data and monitor that with the AI they are building to basically predict the performance change in the future at much earlier frame rather than catching that after something happened and they see the financial statements. So they have quite a lot of very interesting use cases. And the machine learning solutions are very promising business activities for them. So the next story is quite different.

It's from food industry. Kewpie is the number one brand for food quality and safety in Japan. And as you may know, Japan has incredibly high standard for products. And it's quite a big deal to have consumers selecting them as the number one brand. And they make really good mayonnaise. And if you love mayonnaise, I recommend to try it. So they partnered with BrainPad, which is another GCP partner in Japan. They have a lot of experience in integrating machine learning solutions into customers' business processes. And their problem was to detect defective potato cubes in the production line. Ultimately, this becomes tasty stew. And in order to make very high-quality food, catching defective raw materials is very important. And they have the highest standard in food quality. And in many cases, the raw materials they reject are actually just fine industry standards. So you can see the kind of obsession to the quality they have. And this is an example picture of these potato cubes.

And I invite you to actually detect two defects in here. It's pretty hard, right? And here's the answer. So as you can see, it's very hard and it's very subtle. And also, there are so many different ways that defects can happen that it's really hard to automate. And they tried using machines that cost like more than $100,000, and they couldn't do the task. So they're relying on good inspection workers, and it is a very, very tiring task for them. You need to keep very keen focus on the production line for hours to do this task. So they created something they call a worker-friendly inspection AI that achieved a similar level of accuracy as human inspectors. And what it does is basically, it monitors the video feed from the production line and makes a sound when it detects the defects. And the worker knows, oh, there is a defect, and they can pick up the material. And this makes the work much easier for the worker. And so here is the friendliness of this AI to workers.

And we have a very cool video that they sent us to show you the system in action. So this is the prototype machine. [VIDEO PLAYBACK] RIKU INOUE: And you can see potato cubes flowing down. And you will hear the sound. [TONES PLAYING] And the different tone means different locations on the belt– center or left or right. And probably, you couldn't really detect the defects here, so I'm going to play this again. [VIDEO PLAYBACK] RIKU INOUE: I tried again and again, and I still can't do it. So you can see how hard the task is and how good the system is. And another note is that this batch is the test set, so it has a lot more defects artificially included in the batch. So with this success, they practically saved more than $100,000 per line. And also, they got these results in only six months. And they are quite amazed at this very fast turnaround because typically, this kind of project in their industry takes really long time– much more than six months. And their approach is basically to combine traditional image processing technique and deep neural network approach to detect anomaly in size and color.

So they did a creative approach to combine the kind of conventional and new. And I invite you to come to see the [INAUDIBLE] demo on the third floor at the Big Data and Machine Learning sandbox. It's a similar concept of basically using robots to do the task you want. And you can voice control the robot. It's very fun. So please drop by. And our last example is from commerce. So Aucnet IBS, they are the largest, real-time car auction service in Japan. And they list 5 million cars every year. And 30,000 car dealers depend on their system to run their business. So it's quite a large scale operation. And each listed car requires 30 photos for interiors and exteriors. And what they have been doing is basically sorting these images manually, because the images are very important for these listings, and they need to make this right. And this takes like five minutes per car. And add that up for 5 million cars, you do the math. It's a lot of time consuming. And you need to know what model, what make, what year, and which parts, and what angles are missing.

And doing that sounds like a perfect thing to do with machine learning. So they created an image classifier that does the classification of model, make and year as well as which parts they are. And they got to 95% accuracy, which is quite high and completely enough for production use. So I'm going to show a little demo of their system. Let me log to my computer. So please switch to the demo machine. So here is their demo system. And I have a bunch of photos of a car. So unless you're very familiar with lots of cars, you wouldn't know exactly what year, what make. So I'll drop this here. So it's uploading. And in real-time, they are doing the recognition. You can see the confidence score coming in for each different image. And it's now 55%. Lots of time is spent actually on uploading the photo itself. So it's getting there. So we got the result. So it looks like this is 2012 Toyota Land Cruiser Prado. And they have the price range. And also, they have different angles got right.

And you can see the confidence score here as well. It's very confident about the angles and the parts, tires, and under the car, even engine. So they can effectively sort the images and detect what's missing. And it saves a lot of time when they go through 5 million car listings every year. So please go back to the presentation. On the approach they took to build this classifier is something very, very useful. It's called transfer learning. And the concept of transfer learning is quite simple. So you take pre-trained model. Let's say, in this case, Inception v3. And it's already trained on millions of images, and it's open sourced. So you can get your hands on these models and then chop off last few layers that does the classification and add new layers and retrain using your own image. And the benefit of this is the base model already have learned how to extract features and understand the basic components of images. And you save a lot of time doing these training for millions of images.

And with the new image set, basically you need a lot smaller number of training data set. In their case, they used 200 training images per class. So for one car, you only need 200 images, which is a lot less when you compare from scratch training. So it saves your time. It saves your amount of data. So it's a very versatile method to use. And they used ML Engine's distributed training to do the speedup. And they got 84 times faster speed using 100 workers, 100 nodes, for training. So they got from 28 hours on a single machine to 20 minutes. Again, this enables a lot faster iteration on your model. And since transfer learning is quite a useful method, if you're interested in building your own image classifier, we have a blog post that basically step-by-step explains how to do transfer learning on ML engine. So please go to this link or simply Google "Cloud ML transfer learning." So we have been seeing a lot of examples. And at this point, I hope that you're convinced that machine learning is valuable to a lot of industries.

And I invite you to really consider today starting to look into how you can use machine learning. And GCP can help your [INAUDIBLE]. We provide two broad categories of products. One is pre-trained models– so visual API, speech API, translate, natural language. And we announced the video intelligence API today. Using these without any knowledge about machine learning, you can take advantage of the power of machine learning. And when you want to build your own custom solution using your own data, you can use ML Engine. And ML Engine supports your whole life cycle of machine learning, from data to understanding the data, pre-processing the data, training, and prediction. And as the next step, I recommend you to join some of our ML sessions. There is a lot. This is a subset of these. These are especially relevant when you are trying to build your own models and learn how to do this. [MUSIC PLAYING]


Read the video

TensorFlow is rapidly democratizing machine intelligence. Combined with the Google Cloud Machine Learning platform, TensorFlow now allows any developer to leverage deep learning technology without spending top-level expertise or top-level costs. In this video, Riku Inoue and Bryan Lares share how a car auction service and a global insurance company were able to adopt TensorFlow and Cloud Machine Learning to solve real-world business problems and improve customer service and product excellence.

Missed the conference? Watch all the talks here:
Watch more talks about Big Data & Machine Learning here:

Leave a Comment

Your email address will not be published. Required fields are marked *

1Code.Blog - Your #1 Code Blog