Automated Testing with Xamarin

How Clarity ensures quality with every Xamarin-developed mobile app
Test on 1,000 Devices

Automating Mobile Testing

Testing apps across multiple devices has long been a thorn in developers' sides. Simulators and emulators help to some degree, but bugs still easily slip through the cracks. Xamarin's frameworks offers a fast, effective, intuitive solution. Automated app testing solutions by Xamarin allow developers to perform automatic, cloud-based testing across more than 1,000 actual, physical devices. This is made possible via the Xamarin Test Cloud, which is one of the most innovative and convenient automated app testing solutions out there. Clarity supports the frameworks that are needed to make the most of this innovative solution.

Xamarin mobile development platform logo

These days, testing an app across multiple platforms isn't enough. There are dozens of different mobile devices out there spanning a dizzying array of screen resolutions, form factors and other characteristics. These devices run on different operating systems--and multiple versions of those operating systems--so testing an app to ensure that it will work properly for the majority of users is usually difficult. Given the many variables that are involved, it is just about impossible to validate an app for every possible configuration. Large corporations can afford to purchase dozens of devices on which to perform more accurate tests, but smaller agencies and businesses don't have that luxury. Fortunately, Xamarin levels the playing field.

Testing on the Latest Devices

Xamarin Test Cloud

The Xamarin Test Cloud provides a framework for performing automated, cloud-based app testing across more than 2,000 devices. The devices are not emulators or simulators; they're actual, physical devices, and they are stored at a Xamarin facility that is located in Aarbus, Denmark. Every month, more than 100 new devices are added to the Test Cloud, ensuring that the latest models are up and running for developers' testing needs. Xamarin's framework also reportedly takes requests, so if a device isn't on the list, the company may add it if asked.

The Test Cloud is designed to work with Calabash, a proprietary test system that lets testers and developers to develop automated tests for iOS and Android that assess the app from the user interface level on down. Under this system, tests are written using Ruby Cucumber, and they are like interactive stories. Once a test is ready, it can be executed across the more than 1,000 devices in the Test Cloud.

When working with Clarity solutions, it is generally preferable to create acceptance tests using Xamarin C#, which can't be developed using Calabash. Luckily, Xamarin offers an alternative option, Xamarin.UITest, for those who prefer using C#. Xamarin.UITest is a test creation framework that can be used to create user interface tests via the NUnit Test Library. Just as with developing tests through Calabash, the completed tests can then be uploaded to the Xamarin Test Cloud and run across actual, physical mobile devices in a facility in Europe.

The interface for the Test Cloud is incredibly intuitive, which makes it all the easier to zero in on bugs and other issues. Click to learn more about Xamarin load testing.

 
clarity mobile app designed for medical client in Utah
Upload & Schedule

Uploading to the Test Cloud

After writing a test with Xamarin.UITest, a developer can upload it to the Test Cloud and schedule it for execution. From there, the app is loaded automatically, the tests are performed and results are provided once all of the tests are complete. Conveniently, this setup also allows developers to set up tests to automatically execute any time new code is integrated into an app, removing one more step and giving developers one less thing to worry about.

More Timely Feedback

Continuous integration & Automated Testing

Xamarin provides support for continuous integration, which validates an app's functionality and quality through every step of the development process. Through a combination of continuous integration and automated testing, developers can receive instant feedback whenever new code is integrated into an app. Therefore, developers can know right away if new code will disrupt an app's functionality or break aspects of its user interface. Adding new features to apps is a crucial aspect of keeping users happy, and the process is a lot simpler and less aggravating when a developer can instantly test to see if a given update is going to cause issues.

Physical Device Testing

Running Xamarin.UITests on Local Devices

It is also possible to run Xamarin.UITests on local devices. Some developers prefer using specific devices that they have on hand and like using them every time that they test an app. This option is supported with Xamarin. Xamarin Studio and Visual Studio can both be used to run tests on devices that are connected to computers. It is easy and intuitive to perform tests in this manner, but most developers still use the Test Cloud as well.

Clarity designed medical portal for Coflex
Summary of Automated Testing

Steps for Testing an App with Xamarin

Because it is so intuitive, Xamarin is one of the simplest solutions for testing apps across multiple devices. Most developers pick it up right away. First, the app is automated using a testing framework or Test Recorder. Next, the test suit is uploaded to the Test Cloud, where it is executed. Once the tests are complete, detailed reports are generated and can be analyzed with painstaking detail. Reports even include screenshots to help developers understand how their code is affecting the app on a visual level.

Cross-device Testing

Android versus iOS

To perform automated testing on devices with Xamarin, a developer should create a typical .NET 4.5 class library, and then add NuGet packages for Xamarin.UITest and NUnit. If they are working with Android, they are all set--no additional steps are needed. If they are working with iOS, however, the required hooks must be opened. Fortunately, this is easy enough to accomplish. In the Xamarin iOS app, install the Xamarin.TestCloud.Agent NuGet package, and then add the appropriate code to the AppDelegate class within the app. When done properly, the exact same test can be shared across Android and iOS, saving even more work.

Running tests on the Xamarin Test Cloud is easy. The tests can be worked into continuous integration processes, which continually check apps against all devices. Currently, there is no simpler or more convenient option--especially for developers who use C#.

Clarity is a Xamarin certified development firm. We've been developing on Xamarin long before Microsoft acquired the technology back in 2014. If you've got a potential Xamarin project and need to talk to someone about planning or costs, please reach out for a free consultation.