>> I am a Test Cloud engineer on Xamarin here at Microsoft.
>> Okay, great.
>> So I pretty much help organizations get ramped up with automation testing.
>> Okay, great, so I’m familiar with Xamarin Test Cloud. But it sounds like what we’ve done is we’ve added some other types of tests that we support there?
>> Yeah, yeah, exactly. So for a few years, we’ve only supported two frameworks, Calabash and Xamarin.UITest.
>> And just recently we rolled out support for Appium and Espresso. So for today I’m gonna be showing how you can. And take an existing Espresso test and make it work within Xamarin Test Cloud.
>> That sounds awesome. So I can actually leverage the investment that I’ve already made in Espresso, and now leverage that inside of Xamarin Test Cloud.
>> Yeah, exactly. So that’s been one of the biggest cruxes about it. Because if you’re only limited to a few frameworks, it pigeonholes our testing solution. So when we open this up to other frameworks that you can take and upload, it makes it a lot easier for customers to adopt Xamarin Test Cloud.
>> Okay, so before we get too far, we’re assuming people know what Xamarin Test Cloud is. So why won’t we just take a second right now and tell those who might not know what Xamarin Test Cloud is, what it actually is.
>> Sure, so Xamarin Test Cloud is a mobile device lab containing thousands of devices. So where it’s best to use that is when you wanna deploy maybe nightly builds of your testing solution, or weekly builds for regression testing. And you don’t have every single device that your users have to launch your mobile application.
>> Got it.
>> So you can deploy your test with your APK or IPA to Test Cloud and see how it looks and see all the results.
>> Right, when you say a device lab, these are thousands of physical devices.
>> Thousands of physical devices, yeah.
>> Right, these are not emulators or simulators. This is your ability to take your application, actually install it on an iPhone 7, which you might not own yourself, and be able to execute tests against that.
>> Right, because what a lot of companies do is procure just a ton of devices. And those devices become obsolete. With TestCloud, you don’t have to worry about that. We have a procurement team that is gonna take all the devices that are coming out on the market, and they are gonna be there for you to test.
>> Right, so I don’t have to worry about having this suitcase full of devices. And not too mention, even if I had all the devices, I would then manually have to go test on all of those devices. And what we’re doing is allowing you to automate that.
>> For those who know who I am, I am a huge Dev Ops guy and I’m all about automating everything we can and using team services to do that. So the Xamarin Test Cloud test can actually be executed as part of my automated build, correct?
>> So that was actually one of the big reasons why I decided to join the team. I saw a huge need for automation testing because prior to this you’d have a manual tester, a developer with a mobile device connected to his machine, or maybe a simulator, and they would have to go through hundreds of user scenarios. This isn’t really repurposable in any way.
>> So with automation, with VSTS, with Test Cloud, you can automate on almost every device out there and save a ton of time.
>> Awesome, so show me how it works.
>> Yeah, sure thing. So here I have an instance of VSTS, and I’ve created a couple of build agents that we can look through. So the first two I wanna focus on is XTC Weekly and XTC Nightly. And what XTC stands for is actually Xamarin Test Cloud.
>> Perfect, okay, great.
>> So we’re gonna be deploying this based on different timing within our dev ops development cycle.
>> Okay, so these are builds that are gonna run either every night, or you are going to run every week. And based on when I run them, I might change the number of tests that I’m gonna run.
>> Right, exactly. So I mean, it’s rare when you see a development cycle of a week long. Typically it’s like two weeks or a month.
>> Sometimes, yeah.
>> But if you wanna test every single scenario that you have in your arsenal, along with every single device that we have in our cloud, you can set that up. So that’s what this first test is going through. The second test is if you have a smoke test, so if you see devices that are coming up red, you have errors on, you can target those specific devices.
>> If you have certain tests they’re not passing as well, you can target those tests. So you can get your results back a lot quicker.
>> Gotcha, so you don’t have to waste time running tests you know are gonna pass. You can focus on the tests that are causing you issues.
>> All right, perfect.
>> So let’s take a look at this.
>> If I click into this build, what I’m gonna do is just quickly queue it so we can just see what’s going on.
>> Okay, so queuing a build is basically just telling the system, please go run one of these real quick.
>> Right, exactly. So we have to wait for our hosted agent to spin up, which shouldn’t take too long. I actually have just a few tests as well that we’re gonna deploy and the mobile application that I have is pretty small, too.
>> Okay, great. So yeah, the hosted agent, for those who don’t know, are agents that we have in the cloud for you automatically just waiting to do your build. So you don’t have to install these. They’re literally sitting in Azure just waiting for you to do your builds for you. Now you can also stand up what we call private agents which is where you actually stand up the agent on your own, and that way you don’t have to wait for us to go find an agent for you. We just jump on your agent and go ahead and run some builds. Do you have an example of a previous build that’s already run, so we can see what happens at the end of it?
>> I do, so let me go ahead and just navigate to that. So here is an example of one that was able to deploy and every single step successfully passed. So what’s happening here once it initializes, the agent is gonna grab our source code. And again, this source code is of an Android native application written in Android Studio. So I actually have this. If I switch over, this application that we’ll be testing is very simple. It’s just a demo app. And you could see it running on the simulator on the right-hand side. So every one of these cards is a note. So you can take notes and it’s more or less like Evernote, but a very slimmed down version of it.
>> But if I wanted to create a new note, I would just have to click on this plus sign. Say Espresso Demo and, all right. So once we have our note, we can save it to our list. And that’s basically all the user can do.
>> Right, so what this application or what this test is designed to do is to go through that same user flow.
>> Now with Espresso Test, you can take existing Espresso Tests that you already have built out, tweak it a little bit to run in Test Cloud and that’s actually what I did with this. So Android developers out there may be familiar with this gradle, the build.gradlefile. All you have to do to incorporate it, at least add the corporate dependency, is this line of code.
>> So whenever a test is being called and this test task is being called, it will automatically build out the right apk file that you need. And then once you have that enabled, you can go into your test, Itself, Add these two lines. It’s basically importing.
>> Namespaces for you.
>> Yeah, exactly.
>> So this is pretty cool, cuz I realize that you’re on a Mac. And this is the point where I like to tell people that when we’re talking about Visual Studio Team Services, it’s not just for your PC. It’s not just for your .NET developers anymore. This is for any language targeting any platform. Cuz we’re talking about a native Android app written in Java on a Mac being built with Visual Studio Team Services and run in Xamarin Test Cloud.
>> Yeah, exactly. I think that’s the beauty about DevOps because you can take a lot of different tools from a lot of different areas and put them all together.
>> And build that pipeline that you need.
>> Exactly, okay, cool. So we make a few tweaks to our tests by adding a dependency, adding a namespace. And now when I run this build through Team Services CI system, I’m now gonna get the APK I need to then give the Xamarin Test Cloud so that it can execute my test for me.
>> That’s correct.
>> All right, perfect. And then it’s gonna go through that exact same flow that you did and verify that that note was actually added. And then I can get those results back in my build.
>> All right, great.
>> So just an additional note here.
>> One other thing that you will have to add is this report helper.
>> So what’s wonderful about this is that it makes the code really easy to read. So we can see that we’re on the main screen. We add a new note. And every action that we’re performing, it’s going to stop and take a screenshot of this instead of Test Cloud. So this is the indicator in Test Cloud or Test Cloud wouldn’t take a screen shot.
>> Okay, can you show me how that looks in Test Cloud? How can we see that?
>> Right, so if I just hop back over, and we can go into one of our tests. In fact, you can see this validating. Means that our build that I executed got to that step and it’s already launched. But if we go into a previously launched test and view the results, so once I enter in a test that it goes to our dashboard which gives us a very high level look at some reporting tools or reporting analytics.
>> I see, so this is the name of the test you’re in and these are those markers that you put in there with that line of code you were telling us about.
>> Right, so this is more true to test driven development. So every single step that a user would take is indicated here. And then that corresponds directly with a screenshot. So right now it’s just getting launched up. You can see in some of the screen shots, the app is actually-
>> But once I continue, you will be able to see us entering the text, or the title and the text and then saving it.
>> Right, what’s really nice about this is these are real devices. You can even zoom directly into one of them and get much more detailed analytics of what’s going on. So much so that you can see memory usage, CPU usage. You can download the device log.
>> And all of this is incredibly helpful for developers when they wanna troubleshoot an issue.
>> Yeah, and allow me to be able to diagnose what’s going on without me having to go and run it myself or even try to go find one of these particular devices to go run the test on.
>> Exactly, so-
>> This is incredible stuff. So now what we’re doing is we’re allowing you to use Xamarin Test Cloud with, give me the four test frameworks again.
>> So the four supported test frameworks is Appium, Espresso, those two are new.
>> And Calabash and UITest.
>> Perfect, so I wanna thank you so much for coming on the show and showing us how to use Espresso with Xamarin Test Cloud. Thank you so much for watching, guys, and we’ll see you next time, thanks.
In this episode, Donovan Brown is joined by Adam Barlow, who shows how to use Espresso tests with Xamarin Test Cloud and Visual Studio Team Services. He shows how to leverage your existing Espresso tests with in your continuous integration build.
For more information on the Xamarin Test Cloud, see the Test Cloud website.