Visual and UI Test Automation for Agile Dev

Visual and UI Test Automation for Agile Dev

Achieve optimal results in enterprise dev & test

Moderator: Hello and welcome to today’s web seminar, Visual and UI Test Automation for Agile and Continuous Delivery Processes, sponsored by Info Stretch and featuring speakers Harshal Vora and Sivakumar Anna. Harshal Vora is the Senior Solutions Engineer at Info Stretch. He’s the solution architect and specializes in automation and CI processes. Sivakumar Anna is the Senior Director of Enterprise QA at Info Stretch. He works as a principle QA strategist for Fortune 500 companies. I am your host and moderator, Beth Romanik. Thank you for joining us today.

Before I hand it off to the speakers, let me explain the controls on this console. The console opens with three panels on your screen. All of these can be moved around and resized to your liking. At the bottom of your console, it a widget toolbar. Hover your cursor over the icons and labels identifying each widget. Through the additional resources on the right, you can download the pdf of the speakers’ slides. If you experience any technical issues, please send a note to the production field beneath the panel. We’ll be on hand to help. Our responses can be read in the Q & A panel to the left of the slide. This is also how you will submit questions.

There will be a Q & A session at the end. So feel free to ask questions throughout the presentation. Finally, to optimize your bandwidth for the best viewing experience possible, please close any unnecessary applications you have running in the background. This is being recorded and will be made available for on demand viewing. Once the recording is ready, you’ll receive an email with instruction about how to access the presentation and slides on demand. With that, I’d like to hand it over to our first speaker, Harshal.

Harshal Vora: Thank you, Beth. Good morning. Sorry, guys. Good morning, good afternoon, and good evening to everyone. Thank you again for joining this web seminar. Today, we are going to talk about how to begin implementing the Visual and UI Test Automation for Agile and Continuous Processes. Today, we are going to lay out the whole session. We are going to talk about how we can achieve the Visual and UI Test automation. Why the automated visual testing is very important when the industry is talking about ship left and QA to QA transformation. What are the different tools that deliver to achieve the automated visual testing?

Where automated visual testing falls in this whole CI and continuous delivery process. We’ll also talk about what are the enterprise continuous delivery objective and what Info Stretch has done to achieve those objectives. Before we start and go deep diving into automated testing, I would like to introduce Info Stretch. What we do at Info Stretch? We do a lot of testing, automated testing, implementing using CI/CD across multiple channels of web applications, mobile applications. As well as, we have a lot of patents pending on our frameworks of our tools. We are headquartered right here in Santa Clara, California.

We have a global presence across Europe, as well as, on the Eastern Coast. So let’s start with looking at how the mobile test automation has evolved over a period of time. So this is the timeline since the first IPhone came out which was in early 2006. Let’s see what are the tools and how the test evolution. What are the types of testing that has evolved over a period of time in the last 10 years? So since the early inception of the IPhone, we saw a lot of devices coming into the picture. Some of the first testing that came out of the box was the compatibility test.

Right. Because of the number of devices out there: the Blackberry devices, Nokia devices, symbian devices. It was a lot about compatibility. People wanted to test how that application behaves on each and every device. At that time, Perfecto was the only tool available for a device compatibility test. Then came Soft Labs which also was compatibility testing. As we came towards 2010, the Android became popular. We saw the automation, function automation, taking a forefront in terms of testing. Selenium and Appium Open Source Technologies came into the picture.

Every other tool you see out there is supporting this Open Source technology. Along with mobile getting a lot of attention and the mobile first or mobile only technologies coming into the picture, early 2011 a lot of load performance testing automation came into the picture. So we needed a strong framework or strong integration point between functional test and load and performance test. That’s where Siesta and Blaze Meter came into the picture. They were indicative with a lot of Open Source tools and help people to achieve the whole or more automation.

Then in early 2013, we saw the rise of the topic that we are going to talk about today which is the visual automated testing. There are tools available in the market right now like Applitools and Galen framework which we are going to talk about today. We will see how these tools help us get through the whole life cycle of testing any mobile applications or any responsive applications. How are we going to choose a whole CI/CD pipeline by integrating all of these tools which evolved over a period of time into one to achieve the continuous delivery process.

I’d like to see what is the visual testing, right. Before we go out and define what is visual testing, let’s get a quick poll in how you guys scale your art of visual testing on a scale of 1 to 10. We’ll give like a quick 15 or 20 seconds for this poll. Then, we’ll move on fast. The automated visual testing as we saw in the evolution life cycle, right. It has been evolving since 2013 the framework, the tools and technologies that are out there since 2013. The next evolution that we are going to see is integrating this automation within our CI/CD processes because we already have a functional automation, the regression automation.

If it can get through manual testing which includes automated testing as well at a lower level, then it can help a lot. So I think we have got a good amount of responses. Let me see, it’s about on an average the awareness is between 5 and 7. So on a scale of 1 to 10, the awareness is 5 to 7. So that’s a pretty good number. Let us define what is visual layout testing. What exactly is this process of doing this visual layout testing? In today’s world where we have a number of factor of devices with mobile devices, desktop devices with browsers; it becomes very essential to test my web apps or web applications for the layout of the object or the UI.

For example, the whole QA where we are visually seeing the look and feel of my application of my say textbox. Any of the text on the application is either in a readable format or it’s in the correct order. You can have visual automation testing to verify that any of those objects are present or not. But if you want to verify that visually those objects are arranged properly as it should be as a UX document is defined, then the visual layout you are testing comes into the picture where we are verifying the content, layout, and appearance of the UI on multiple platforms across multiple browsers or multiple devices.

Let’s take a look at an example, right. What exactly … talk about the teslamotors.com. right. So as you can see on the desktop version of teslamotors.com app. There is a mobile version. How this application being responsive to that application from the desktop version to the mobile version? As you can see the whole transformation and order which is on the left hand side of the test drive on my desktop version becomes on top of the test drive on my mobile version.

So if I want to verify this particular layout, then I need to either have a manual process or I need to have an automation process that we are going to talk about today which is the visual layout automation. My personal test automation is going to verify all those things saying that, yes, all text are present. So my Selenium automation or Appium automation is going to say all these things are present. But to verify that these things are in proper order, we need to have this kind of automation. Also, you are seeing the links which are in the desktop version gets into one menu on the mobile version.

So all those things are very important. Let’s take a look at the second example which is … let me get to the … we’ll take a look at cnn.com example. This is a similar fashion, right, with the responsive web design I’m getting a lot of share in the market right now. We are seeing that from the desktop version to the mobile version, it becomes very important to check out even in the mobile version we get for the CNN application … we are getting the weather and location info right within the application, right, which is missing on the desktop version.

So if you want to verify this kind of scenarios, visual testing can help us do these things. Mixing these with the functional automation, it can be a strong vision … or it can be a strong automation feature during our CI/CD. I’ll just browse through one more example which is going to be espn.com. What you are seeing on the desktop version is three layers: a favorite section, new section, and the now section which on the mobile version gets created as three separate pages or is rendered as three separate pages. The top score menu completely vanishes.

So this is the whole layout that gets totally changed on multiple browsers or multiple platforms. Automated visual testing is all about testing the automatically and integrating it with our function testing. So now let’s see why visual layout testing automation essential. Before we do that, let’s take a quick poll again to see what is our audiences’ view? How important is your testing for the mobile apps? So if you guys can rate this visual testing … we are already starting to get answers. We already see the trend. At least I am seeing that 80 percent or 90 percent of people say that visual testing is very important for mobile apps.

As we see the growth of mobile applications or the proliferation, the user base, moving from conventional desktops apps to getting on the mobile apps, it’s becoming very essential to test this visual testing. If we are talking about myself, I would rather be banking on my desktop app or opening my mobile.com on my desktop app. I’d really prefer to browse through on my mobile application. All right. So I think the results are out here. We see at least 80 percent of people saying it is very important to do the mobile app testing. Okay. So let’s see why … this is very important, right.

So UI issues are one of the top reasons that apps are rejected from Apple Apps store. If a developer or enterprise is submitting that app to an app store, the application certification engine then supports multiple devices and multiple phone factors. We have seen that it has been the top 10 reasons that at least 5 percent of apps are rejected due to Apple’s guideline. If the interface is complex or less than very good, then it was rejected. So it becomes very essential at that time that we account for this visual issues early in the cycle so that we get that fixed.

So this is the basic testing parameter on which basically lays out the whole testing landscape. Right. So you start all the way from unit testing of verification and regression testing, API testing, front end testing, and during testing. Then at the end, we have a manual testing which can be extra validating to our divisions where you can see visual UI testing and the business logic testing. Now with the proliferation of devices where you see this Apple Mac book coming out with Apple pencil and all those different fun factors, it becomes very essential to this visual automated testing in an automated fashion.

So up until now, all this four and five layers of parameter have been the core of the test automation. Now taking this one step farther, the top of the peak of the parameter which is the manual testing is extrapolated into business logic testing and visual UI testing. Now if we are able to even eliminate or shrink that top of the page which is going to be a bottleneck during the CI/CD process, then that can exit the whole CI/CD cycle in terms of getting to my applications faster, to my QA teams, or into the production in moments.

So let us look at the whole CI/CD pipeline and where this automated visual testing is seen. Right. So if in my continuous delivery process the whole idea is everything is automated right from the core check in to the unit testing and integration testing. Then, we have the functional automation testing. But right at the end where we have this acceptance test or user acceptance test which becomes a manual effort. If we are able to convert that manual effort to an automated effort, then we can go to the continuous deployment.

There we can readily reduce our release cycles from a couple of weeks to say days or even a couple of hours. If we do the right start of CI/CD structuring. So let’s look at the landscape of two technologies that are available for the whole visual layout testing. In terms of the two technologies that we have available right now expands across all the channels. Right. So we have web applications. We have mobile applications, as well as, we have a mix of mobile and web applications. So there are tools out there. Some are Open Source. Some are proprietor tools which enable us to do a lot of visual layout testing.

One of them we have here is Applitools Eyes. Right. We can get comprehensive automated UI validation solution. It also has a smart image message algorithm. Since it’s a cloud service, it makes integration tests available everywhere and accessible for each and every team. Also, it supports marketable mobile solutions for responsive app, as well as, it also supports multiple OS and languages. Then, we have Perfecto Mobile which is basically has a multi-language support. It is a proprietor tool. You can develop your test score using Java or Java Script or any other language support by Selenium or remote web driver.

They also have Skip One technology where it’s basically leveling the device agnostic test coding method. It basically works on the OCR, object recognition method and object validation method where you can check using your CSS, ID, and X path across multiple found factors. So it can be net browser or a mobile web app or mobile AD App. Then, we are extensively talk about the Galen framework which is an Open Source framework. Galen framework is also designed to do the whole responsive web testing.

To do a deep dive into the responsive app or deep dive of the Galen framework I’d like to invite Siva who is going to take us through the journey of how Galen framework can be incorporated for the whole visual layout testing. It can be integrated with our CI/CD processes using like Jenkins and also integrating with any of the cloud solutions. So with that I’ll hand it over to Siva who is going to take us through the Galen framework.

Sivakumar Anna: Thanks. Good morning everyone. Before I go into the Galen framework, I’ll just kind of reiterate that we are seeing that not many tools and technologies available. We have seen in the industry a lot more important to the visual layout testing. That’s where in the last couple of years we have seen tools popping up to provide support to focus on the visual layout testing. So it’s basically how your application is kind of rendered and in terms of the arrangement of the objects. That’s the core focus of the layout testing that we have seen.

The tools like Applitools and Perfecto Mobile, for example, has been providing visual inspection from the beginning. The Galen framework is a lot more focusing on the responsive web design. So what we will be seeing and what we will be kind of doing today is for the responsive web design because what we are also seeing is the industry is kind of moving towards a lot more towards HTML 5 and responsive web design. So the Galen framework is for now focused for the responsive web design. But there are tools and technologies that we have implemented in a lot of different engagement that supports both the mobile web and desktop web applications. Right.

For today, the focus will be all around the responsive web design. So moving to the next slide. What is Galen framework? The Galen framework has been out there for a little over a year. We are seeing a lot more traction and enterprises start adopting the Galen framework. The very cool features I would say a powerful feature of Galen framework is entirely built on top of Selenium. So as we have been using Selenium. A lot of folks are using Selenium for functional automation for quite a long time.

So for them to adopt the Galen framework is very easy because since they’ve been built on top of that it provides a specific language on top of Selenium to do the UI and visual layout testing. Right. So basically it supports both the desktop browsers as well as mobile web. As we have seen in the responsive web design though it is targeted for the mobile devices with the different found factors, but for the testing purpose you can use it within the desktop itself. You can resize the window automatically the applications that is UI. Right.

So it has you to do the automation within the local environment when you want to test the different UI layout. By nature, it’s built on Selenium. So pretty much you have access to the entire cloud system that is available for Selenium like Soft Labs or Browser stage or open stage or any solution that you want to use for your web for kind of a scalability purpose. Galen framework supports all of it. Right. So moving along to the next slide. It takes a little bit of time to … so basically, how does it work? I mean, it’s very similar to Selenium. It’s not going to be any different from the execution from the architecture perfect.

It goes through the same steps that Selenium regularly would. What Galen framework allows you to do is kind of interject the solution or the validation more specific to the UI layout. So there you can see what is the order of the elements that you are looking for. What is the color? What is the relationship between one object to another object? So it can provide all of that. Galen framework or the Galen specs. So you provide the specifications of the different objects and put the relationship of the association of those objects in a spec file.

So we can see in a real example so you will be able to kind of understand how the specification files are done. If you are to change, you can see how the specification looks like. Right. It has very basic feature that you would expect from any framework in terms of reporting, in terms of the data it collects, and in terms of the reports, in terms of the screen charts, and how you can accurately identify the error and helps you to kind of get down to the specific problems. Right. It has a built in image comparison. You can use that to the image comparison.

You can also verify through a specific part of the screen. That’s the way Galen framework works. You can also go to the Galen framework website to read more about it. Before I go into the details, I just want to kind of get the response. So as you can see, we are also seeing a lot more in the industry the adoption of the responsive web design. It’s kind of slowly we are seeing that the enterprises are moving towards the responsive web design as it becomes a lot more standard in the industry. The applications are kind of moving towards all the responsive web design based architecture.

As opposed to just I agree totally that responsive web design is coming toward a lot more to the standard. Right. So just to go into the actual demo that we will be doing today, right. Using the Galen framework, I’m taking Tesla Models website as an example. So as you can see in the screenshot, this is two issues that they are kind of trying to validate. One is between the desktop version and mobile version. It’s kind of two different screen size. As you can see, there are like you can see two or three major changes between one version to another version. One is the order of the buttons.

So instead of horizontally stacked, it’s vertically stacked between the desktop version and the mobile version. That is one change. So that change … currently there is no way to do that in the standard framework that is out there using the Selenium. That’s where the tools like Perfecto Mobile or Applitools. Galen framework allows you to specify what should be the relationship between the other button and desktop button. One thing that we would be verifying. Also if you look at the top menu that totally changes between the desktop and the mobile.

In the desktop, it is all kind of laid out on the main screen; but in the mobile version it is a menu option that is available for you to click to see the different menu options. Right. Finally, you can also see that several links are missing to optimize the screen properly. So the upload links and other links are kind of removed from the mobile version. Right. So in our automation in our demo, this is where we are going to kind of focus doing this simple demo, but you will be able to kind of understand the concept and understand the power of the Galen framework. Right.

So before I kind of go into the code, I just want to kind of highlight the specifications that we are return to verify what was on the previous screen or slide. One is about the link verification itself. So as we saw the app for leasing. Right. So as you can see this is a couple of samples on the Galen specifications. So what you do is in the specification, you define the object that you are interested. So on the left hand side, put in the specification that we are interested in for missing link. Right. So you want to put what is the page that is associated. As you can see, you can also specify the associations or conditions specific to various found factors.

So here, I’m seeing that on the mobile and tablet I want that a link should be missing. So I can view a specification of association condition that it should be absent. So this is the language that the Galen provides. It is a very detailed robust language specification available, but this is very basic one that you want to specify that you want to see the link is missing on the mobile and tablet. Right. We will see on the next slide what is a mobile and tablet definition means. As you can see at the bottom of this specification on desktop, we are seeing that the text should be visible and the text for that link should be apply for leasing, Right.

So this is a very simple basic example how you can define a specification. As you can see on the right hand side where we are giving the relationship between the button. So we are kind of picking up two buttons. The other button for mobile and tablet. It should be inside a button group called the menu button group. We are also seeing that the mobile should be above the other button. Right. So this is the way that you would be able to define. When it comes to desktop, as you can see once again it has to be inside the menu button group; but it should be left out to the desktop. As you can see, there are two different object identifications.

So that is entirely based on the Selenium X path and Selenium object identification strategy. That is the same strategy Galen uses. Galen basically allows you to kind of extend that and provide the various syntax that you want to verify in terms of the object relationship. Right. So this is really basic specification. We are going to run this for three different configurations. So this is how you define the various configurations. So I kind of took three here in the same test case for three different configurations and see what fails and what works.

But you can pretty much put all different combinations that you want to put, and you can just run it without changing anything. Here, I’m defining mobile, tablet, and desktop. The only thing that is varying here is the size of the window. I’m also going to run this demo on the desktop so you’ll be able to see that it resizes the window automatically. It renders the application to verify all the properties that we are looking for is as expected. So once again, I’ll show the report also. There is like if you look at the Galen framework and Soft Labs documentation, there is a lot more information available how you can integrate this with Soft Labs and other clouds that you want to execute. Right.

So I’m trying to run this through Jenkins so I have a sample project that is set up to run this demo. But before, I just want to show the core base also. So as you can see here … let me show you. This is my test. So it’s a very simple test all I’m saying is load the home page. You run this specification that is defined. This is on the slides. Also I’m seeing that running for all the devices that are different. So if you open up my configurations or I define all the tags it should ask the number of examples that I have, ask the number of configurations that I have. It will try to run that. If I look at the specs, this is how my specs will look like.

So I kind of put everything into one, the button ordering as well as the link. As you can see here, you can put what are the specific insertion that you want for mobile tablets and what are unique about that stuff. Right. As you can see here, you can specify the relationship. You can say that inside particular object should be there. It is also you can use some kind of a range. Like so many times you want to kind of say that it has to be within the space between the two buttons has to be a specific range, you can do a range. There are like a lot of other attributes that it can verify the color of the button should be white.

It can give you a range of color. It can also give a percentage of approximation so that the framework would intelligently use that to verify if it falls within the range or not. Right. So it is very powerful. If somebody wanted to go and look at the detailed specification language, so they would kind of be able to use and perform the testing project effectively. So I’m going to run this example. Once it kind of completes, we can see that running it locally … like I said, it’s going to run just loading the Tesla Motors home page for three different UI. One is for the mobile, and one is for the tablet. One is for the desktop version.

Once it completes the test, the test is a very simple test because it’s just loading the home page and find out whether it is loading correctly or not. Then once it completes, we will look at the execution report. The report also is very cool. Right. So it completed. Let’s look at the report. I’m going to explain the reports. It pretty much gives very detailed information. Let me open up here. So this is a test report out of the box from Galen. Right. So as you can see here we ran three different configurations. One is for the desktop. One is for the mobile, and one is for the tablet.

It gives a very high level dashboard of what all the different combinations that you ran and what is the state of how each of the steps within the configuration. Right. As you can see, there is one test failed out of the different combinations. So if you have to take for desktop, you can see that it highlights where the failure. Any place that it has stopped you can see the checkmarks and see that it indicated pass. Nothing else fails. You can see that you can expand it. If you want to see what is the content, you’ll be able to see the actual insertion.

The very cool thing about that is it shows the screen shot what was our insertion in the topic and see what was the condition. You can also see in the screen shot where the actual insertion happened. Right. So if there is no red indicator, it means that everything is passing. Say, for example, if you wanted to see whether it is … so this is the menu button blue. As you can see, the other button is within the menu button group which is as we expected. Right. Many times when the developers change some of the chip, if you want to enforce that, okay, because this starts the UA.

You’ll be able to kind of catch because if the other button is outside the menu button group, you’ll be able to catch this one. Right. So like I said, the reporting is very cool and really useful because it gives the condition of the insertion that you have. Also, it usually gives how it passed and what verification it passed. Right. Looking into the failure, as you can see our condition was that the button group should be below the header. This is the header. The distance between the header and button group should be 20 to 45. Also, it gives what is the actual distance. Right.

It says that currently it is 15.9 pixel, whereas the range that we gave it was 20 to 45 pixel. Right. So that’s reason that it marked this particular insertion as failure. So you can adjust it. It gives what is the insertion range. It also gives the actual value. So that is a really powerful feature. You want to kind of use it and run the reporting. Right. I just want to kind of restate Galen framework you should look at their website to understand what are all the ways that you can verify and what are the values, what are the layout verifications that might be very important to your application and how that is supported as part of Galen framework.

You can see it. Galen framework is one of the tools that are like a lot of other tools available on the market like we said Perfecto Mobile and Applitools all kind of provide the same level of service to perform the visual automation. Right. So before I just want to also reiterate the fact that as we kind of using this visual layout automation, how do you kind of integrate this part of the continuous delivery? Because the CI/CD has become so important in today’s life cycle of the application development. How you can kind of integrate the visual layout test?

What are the things that you have to take into consideration to make it the continuous delivery very successful? Right. So what we are seeing is this shift left upload, especially moving the QA aspects more early in the cycle. Right. The shift left is very important challenge that we are seeing. Like, for example, they explained automation while having the visual automation as part of the early cycle so that you will be able to start doing your automation well in advance even without the back end necessary components being double up on. Right.

That’s where I think we have seen that the media has become a lot more important because it allows you to be able to enables you to kind of start writing the test cases early in the cycle. You can start putting as part of define the CS cycles. So early in the cycle you will be able to kind of start running your test cases and get the necessary processes to define early the cycle. Right. Unified automation is another key important aspect that you kind of want to take into consideration. You want to have one framework that allows you to do the website cases and mobile automation.

When it comes to whether it is an issue or responsive web design or HDML site or if you are using any other framework like Kony or Phone gap or any other framework that you might be using to build your application. Right. Those are the key things as part of this one. Automating the entire quality process whether it is a CA or deployment like I talked about the virtualization of the test data. Right. All of that is a very important challenge. We have solutions we have implemented successfully in many of our engagements. So we have our framework that supports ability.

We do have integration with Soft Labs, Appium, Galen framework, Perfecto Mobile. We do provide value added frameworks and services on top of these tools. Also, we do provide as part of our service engagement the CS/CD implementation. Our framework supports providing early implementation and early adoption of the automation in your mobile application life cycle management. Right. So before I go into the next section, I just want to kind of quickly get feedback.

If any of you are interested, we would be able to share a follow up session to give more details about automated framework that we can kind of help you to do unified automation like Galen framework and Perfecto Mobile solutions.

Harshal Vora: Yeah, as soon as we have the poll all answered, we’ll do more Q&A session. We’ll open the Q&A for the audience. Myself and Siva will go through all the questions that people have. So as you highlighted, the Info Stretch framework is unified automation framework that will enable you to do functional automation, visual automation, degression automation, as well as, the web services and web channel automation. So it gives you the whole approach where you can integrate your CI/CD. You get very conclusive continuous delivery with one framework rather than having three or four different frameworks for each kind of automation approach.

I think, yes, we are getting in the results for the polls. We’ll keep this poll open for a couple of more seconds. Then, we’ll have the last 15 minutes or so for our Q&A session. As we saw the open source and the way the visual layout automation testing is shaping up, the next step is going to be the whole nonfunctional testing aspect of it. The Info Stretch framework gives us an inside into that nonfunctional aspects of testing as well and how to automate that whole process. So we are going to get a very conclusive testing automation strategy and CI/CD strategy that will enable the enterprise to get to their markets faster. Okay. So let’s go ahead and do the whole Q&A session.

Moderator: All right. First question, does the Info Stretch framework support visual layout testing?

Harshal Vora: Yes, as we indicated the Info Stretch framework integrates … has a very strong integration with Galen framework we leverage. So the whole idea of behind Info Stretch framework is making open source more effective. So the whole framework is built on the open source like Selenium and Galen. So, yes, we do support the visual automated testing.

Moderator: Okay. Great. Thanks. Next question. Is this framework good for verification for whether a link is missing in a mobile version?

Sivakumar Anna: So, yes. Let me take this question. Yes. So like I said in the demo presentation also you can check various aspects of the object that is there on the web page whether it is a link or whether it is an object that is arrangement of the object or color of the object that you are looking for. So there are various elements that you can do. Yes, you can do much more than what we showed in the demo today in terms of the objects that are in the web application and the mobile application.

Moderator: Okay. That’s good to know. Thanks. This person says this type of testing would be fantastic on our Windows app. Do you have any suggestions for an IDE or tool that would allow him to do that?

Sivakumar Anna: So, yeah. I think the Info Stretch framework definitely is a good candidate for this particular requirement because it allows you to kind of do the IDE where you can do the object inspection, the whole object management. You’ll be able to kind of define the parameters like whether you want to run if for Soft Labs or whether you want to run it locally. So that one recommendation I would kind of put it out, but definitely like the Eclipse or any other ID that you might be using today would be a candidate because the Galen framework supports any of the standard ID that are out there on the market.

Moderator: Okay. Thanks. This person wants to know can Galen be used to run browser compares between different browsers on the same desktop?

Sivakumar Anna: Yeah. I think that the model we showed today … we ran it for one browser, but I wish we had more time to show additional demos. But you can pretty much configure to run for various browser combinations. You can take Safari, Firefox, Chrome. You can run it. You can also … I think that’s where you can also leverage some of the Cloud solutions that are out there like Soft Labs or Open stack. So you’ll be able to utilize more effectively so that you can run all these things in parallel to kind of effectively perform your automation.

Moderator: Okay. Cool. Thanks. Can you also check the background color or text color and font details?

Sivakumar Anna: Yes. That is correct. So you can check the color attribute. It can give you a range if you are not sure what is the color. It can give all different attributes of the object that you see in the application.

Moderator: All right. This question is kind of related to one you just asked. Can Galen be integrated with an IDE?

Sivakumar Anna: Yes. The whole Info Stretch framework is an automation studio that we have integrated Selenium, Appium, as well as, Galen framework into one automation studio. The whole IDE integration, we have taken care of it within our Info Stretch framework.

Moderator: All right. Thanks. Can the Galen layout specification be used with fixed client automation tools such as Squish?

Sivakumar Anna: I probably need to kind of get back to you on this question because I don’t think we kind of looked at it, but the Galen framework itself is a very flexible framework. Currently, it supports Java and Java Script. I think the model we saw today was in Java Script, but it does support Java. For other languages and other things, we have to kind of look into it.

Moderator: Okay. Thanks. Maybe you can answer that question more offline. All right. What about broken links?

Sivakumar Anna: Broken link is definitely you can test it. So I think in one of the examples we saw today you can identify what all the links are working. If there is, for example, if you want to see if a link is to an email and you want to see what is that email that it is pointing, you’ll be able to verify that email. So you’ll be able to kind of see what are the links that is failing by clicking whether it is working or not. You’ll be able to kind of verify that as part of Galen framework and some other solutions that we have as part of our framework as well.

Moderator: Okay. Thank you. This person wants to know what is your feedback on visual UI testing for websites hosted on a CMS system.

Sivakumar Anna: So the visual layout testing in general you can use it for any application. So visual layout testing is like how Harshal pointed out in the evolution that it’s been discussed in the last couple of years. We have seen that the tools are popping up. So I don’t think the Galen framework has an intersection for only to specific type of application. You can use for CMS or any other application as long as the objects are accessible through the browser.

Moderator: Okay. Thanks. What languages are supported for writing test automation using Galen or QAS?

Harshal Vora: So using Galen and the QS with the Info Stretch framework, we do support Java and Java Script across the automation cycle. So for Selenium, Appium, as well as, Galen; we do support Java and Java Script.

Sivakumar Anna: Just to add that the Java and Java Script is the language, but we also support the BDD based test altering because that’s where we are seeing the industry adopting BDD a lot more. The Galen framework out of the box doesn’t support BDD. The Info Stretch QAS framework does support BDD on top of Java and Java Script.

Moderator: All right. Thank you. This person wants to know how well would this integrate with my organization’s CI/CD process?

Harshal Vora: As we saw on the slide demo today, CI/CD processes … the whole bottleneck is the visual layout testing which comes right at the end before it goes out to the production and QA environments. If we are able to automate this process, then we can reduce a lot of work on the whole ops team and this gets validated. So the Info Stretch framework has a strong integration with CI like Jenkins or Teamcity or Bamboo or something like that.

Moderator: All right. Thank you. This person says we have a lot of applications developed using grails and groovy for a desktop version. Can Galen be used to completely automate all the test cases like verification and expected page output?

Sivakumar Anna: Yes, very much. Because like I said, it basically sits on top of Selenium. So it will be able to kind of do all the necessary function validations using the Selenium automation. Galen integrates well with Selenium to focus on just the layout part. You will be able to verify the object relationship or the object attributes whether it is the color or links or text. All of that. Right. Yeah, so it definitely supports.

Moderator: Okay. Thanks very much. Well, we had a lot of great questions, but that’s all we have time for today. So that will end our event. I’d like to thank our speakers, Harshal Vora and Sivakumar Anna, for their time. I’d like to thank Info Stretch for sponsoring the web seminar. Of course, a special thank you goes out to you, the audience, for spending this last hour with us. Have a great day. We hope to see you all at a future event.

Sivakumar Anna: Thank you, Beth.

Harshal Vora: Thank you everyone.

Achieve optimal results in enterprise dev & test

Moderator: Hello and welcome to today’s web seminar, Visual and UI Test Automation for Agile and Continuous Delivery Processes, sponsored by Info Stretch and featuring speakers Harshal Vora and Sivakumar Anna. Harshal Vora is the Senior Solutions Engineer at Info Stretch. He’s the solution architect and specializes in automation and CI processes. Sivakumar Anna is the Senior Director of Enterprise QA at Info Stretch. He works as a principle QA strategist for Fortune 500 companies. I am your host and moderator, Beth Romanik. Thank you for joining us today.

Before I hand it off to the speakers, let me explain the controls on this console. The console opens with three panels on your screen. All of these can be moved around and resized to your liking. At the bottom of your console, it a widget toolbar. Hover your cursor over the icons and labels identifying each widget. Through the additional resources on the right, you can download the pdf of the speakers’ slides. If you experience any technical issues, please send a note to the production field beneath the panel. We’ll be on hand to help. Our responses can be read in the Q & A panel to the left of the slide. This is also how you will submit questions.

There will be a Q & A session at the end. So feel free to ask questions throughout the presentation. Finally, to optimize your bandwidth for the best viewing experience possible, please close any unnecessary applications you have running in the background. This is being recorded and will be made available for on demand viewing. Once the recording is ready, you’ll receive an email with instruction about how to access the presentation and slides on demand. With that, I’d like to hand it over to our first speaker, Harshal.

Harshal Vora: Thank you, Beth. Good morning. Sorry, guys. Good morning, good afternoon, and good evening to everyone. Thank you again for joining this web seminar. Today, we are going to talk about how to begin implementing the Visual and UI Test Automation for Agile and Continuous Processes. Today, we are going to lay out the whole session. We are going to talk about how we can achieve the Visual and UI Test automation. Why the automated visual testing is very important when the industry is talking about ship left and QA to QA transformation. What are the different tools that deliver to achieve the automated visual testing?

Where automated visual testing falls in this whole CI and continuous delivery process. We’ll also talk about what are the enterprise continuous delivery objective and what Info Stretch has done to achieve those objectives. Before we start and go deep diving into automated testing, I would like to introduce Info Stretch. What we do at Info Stretch? We do a lot of testing, automated testing, implementing using CI/CD across multiple channels of web applications, mobile applications. As well as, we have a lot of patents pending on our frameworks of our tools. We are headquartered right here in Santa Clara, California.

We have a global presence across Europe, as well as, on the Eastern Coast. So let’s start with looking at how the mobile test automation has evolved over a period of time. So this is the timeline since the first IPhone came out which was in early 2006. Let’s see what are the tools and how the test evolution. What are the types of testing that has evolved over a period of time in the last 10 years? So since the early inception of the IPhone, we saw a lot of devices coming into the picture. Some of the first testing that came out of the box was the compatibility test.

Right. Because of the number of devices out there: the Blackberry devices, Nokia devices, symbian devices. It was a lot about compatibility. People wanted to test how that application behaves on each and every device. At that time, Perfecto was the only tool available for a device compatibility test. Then came Soft Labs which also was compatibility testing. As we came towards 2010, the Android became popular. We saw the automation, function automation, taking a forefront in terms of testing. Selenium and Appium Open Source Technologies came into the picture.

Every other tool you see out there is supporting this Open Source technology. Along with mobile getting a lot of attention and the mobile first or mobile only technologies coming into the picture, early 2011 a lot of load performance testing automation came into the picture. So we needed a strong framework or strong integration point between functional test and load and performance test. That’s where Siesta and Blaze Meter came into the picture. They were indicative with a lot of Open Source tools and help people to achieve the whole or more automation.

Then in early 2013, we saw the rise of the topic that we are going to talk about today which is the visual automated testing. There are tools available in the market right now like Applitools and Galen framework which we are going to talk about today. We will see how these tools help us get through the whole life cycle of testing any mobile applications or any responsive applications. How are we going to choose a whole CI/CD pipeline by integrating all of these tools which evolved over a period of time into one to achieve the continuous delivery process.

I’d like to see what is the visual testing, right. Before we go out and define what is visual testing, let’s get a quick poll in how you guys scale your art of visual testing on a scale of 1 to 10. We’ll give like a quick 15 or 20 seconds for this poll. Then, we’ll move on fast. The automated visual testing as we saw in the evolution life cycle, right. It has been evolving since 2013 the framework, the tools and technologies that are out there since 2013. The next evolution that we are going to see is integrating this automation within our CI/CD processes because we already have a functional automation, the regression automation.

If it can get through manual testing which includes automated testing as well at a lower level, then it can help a lot. So I think we have got a good amount of responses. Let me see, it’s about on an average the awareness is between 5 and 7. So on a scale of 1 to 10, the awareness is 5 to 7. So that’s a pretty good number. Let us define what is visual layout testing. What exactly is this process of doing this visual layout testing? In today’s world where we have a number of factor of devices with mobile devices, desktop devices with browsers; it becomes very essential to test my web apps or web applications for the layout of the object or the UI.

For example, the whole QA where we are visually seeing the look and feel of my application of my say textbox. Any of the text on the application is either in a readable format or it’s in the correct order. You can have visual automation testing to verify that any of those objects are present or not. But if you want to verify that visually those objects are arranged properly as it should be as a UX document is defined, then the visual layout you are testing comes into the picture where we are verifying the content, layout, and appearance of the UI on multiple platforms across multiple browsers or multiple devices.

Let’s take a look at an example, right. What exactly … talk about the teslamotors.com. right. So as you can see on the desktop version of teslamotors.com app. There is a mobile version. How this application being responsive to that application from the desktop version to the mobile version? As you can see the whole transformation and order which is on the left hand side of the test drive on my desktop version becomes on top of the test drive on my mobile version.

So if I want to verify this particular layout, then I need to either have a manual process or I need to have an automation process that we are going to talk about today which is the visual layout automation. My personal test automation is going to verify all those things saying that, yes, all text are present. So my Selenium automation or Appium automation is going to say all these things are present. But to verify that these things are in proper order, we need to have this kind of automation. Also, you are seeing the links which are in the desktop version gets into one menu on the mobile version.

So all those things are very important. Let’s take a look at the second example which is … let me get to the … we’ll take a look at cnn.com example. This is a similar fashion, right, with the responsive web design I’m getting a lot of share in the market right now. We are seeing that from the desktop version to the mobile version, it becomes very important to check out even in the mobile version we get for the CNN application … we are getting the weather and location info right within the application, right, which is missing on the desktop version.

So if you want to verify this kind of scenarios, visual testing can help us do these things. Mixing these with the functional automation, it can be a strong vision … or it can be a strong automation feature during our CI/CD. I’ll just browse through one more example which is going to be espn.com. What you are seeing on the desktop version is three layers: a favorite section, new section, and the now section which on the mobile version gets created as three separate pages or is rendered as three separate pages. The top score menu completely vanishes.

So this is the whole layout that gets totally changed on multiple browsers or multiple platforms. Automated visual testing is all about testing the automatically and integrating it with our function testing. So now let’s see why visual layout testing automation essential. Before we do that, let’s take a quick poll again to see what is our audiences’ view? How important is your testing for the mobile apps? So if you guys can rate this visual testing … we are already starting to get answers. We already see the trend. At least I am seeing that 80 percent or 90 percent of people say that visual testing is very important for mobile apps.

As we see the growth of mobile applications or the proliferation, the user base, moving from conventional desktops apps to getting on the mobile apps, it’s becoming very essential to test this visual testing. If we are talking about myself, I would rather be banking on my desktop app or opening my mobile.com on my desktop app. I’d really prefer to browse through on my mobile application. All right. So I think the results are out here. We see at least 80 percent of people saying it is very important to do the mobile app testing. Okay. So let’s see why … this is very important, right.

So UI issues are one of the top reasons that apps are rejected from Apple Apps store. If a developer or enterprise is submitting that app to an app store, the application certification engine then supports multiple devices and multiple phone factors. We have seen that it has been the top 10 reasons that at least 5 percent of apps are rejected due to Apple’s guideline. If the interface is complex or less than very good, then it was rejected. So it becomes very essential at that time that we account for this visual issues early in the cycle so that we get that fixed.

So this is the basic testing parameter on which basically lays out the whole testing landscape. Right. So you start all the way from unit testing of verification and regression testing, API testing, front end testing, and during testing. Then at the end, we have a manual testing which can be extra validating to our divisions where you can see visual UI testing and the business logic testing. Now with the proliferation of devices where you see this Apple Mac book coming out with Apple pencil and all those different fun factors, it becomes very essential to this visual automated testing in an automated fashion.

So up until now, all this four and five layers of parameter have been the core of the test automation. Now taking this one step farther, the top of the peak of the parameter which is the manual testing is extrapolated into business logic testing and visual UI testing. Now if we are able to even eliminate or shrink that top of the page which is going to be a bottleneck during the CI/CD process, then that can exit the whole CI/CD cycle in terms of getting to my applications faster, to my QA teams, or into the production in moments.

So let us look at the whole CI/CD pipeline and where this automated visual testing is seen. Right. So if in my continuous delivery process the whole idea is everything is automated right from the core check in to the unit testing and integration testing. Then, we have the functional automation testing. But right at the end where we have this acceptance test or user acceptance test which becomes a manual effort. If we are able to convert that manual effort to an automated effort, then we can go to the continuous deployment.

There we can readily reduce our release cycles from a couple of weeks to say days or even a couple of hours. If we do the right start of CI/CD structuring. So let’s look at the landscape of two technologies that are available for the whole visual layout testing. In terms of the two technologies that we have available right now expands across all the channels. Right. So we have web applications. We have mobile applications, as well as, we have a mix of mobile and web applications. So there are tools out there. Some are Open Source. Some are proprietor tools which enable us to do a lot of visual layout testing.

One of them we have here is Applitools Eyes. Right. We can get comprehensive automated UI validation solution. It also has a smart image message algorithm. Since it’s a cloud service, it makes integration tests available everywhere and accessible for each and every team. Also, it supports marketable mobile solutions for responsive app, as well as, it also supports multiple OS and languages. Then, we have Perfecto Mobile which is basically has a multi-language support. It is a proprietor tool. You can develop your test score using Java or Java Script or any other language support by Selenium or remote web driver.

They also have Skip One technology where it’s basically leveling the device agnostic test coding method. It basically works on the OCR, object recognition method and object validation method where you can check using your CSS, ID, and X path across multiple found factors. So it can be net browser or a mobile web app or mobile AD App. Then, we are extensively talk about the Galen framework which is an Open Source framework. Galen framework is also designed to do the whole responsive web testing.

To do a deep dive into the responsive app or deep dive of the Galen framework I’d like to invite Siva who is going to take us through the journey of how Galen framework can be incorporated for the whole visual layout testing. It can be integrated with our CI/CD processes using like Jenkins and also integrating with any of the cloud solutions. So with that I’ll hand it over to Siva who is going to take us through the Galen framework.

Sivakumar Anna: Thanks. Good morning everyone. Before I go into the Galen framework, I’ll just kind of reiterate that we are seeing that not many tools and technologies available. We have seen in the industry a lot more important to the visual layout testing. That’s where in the last couple of years we have seen tools popping up to provide support to focus on the visual layout testing. So it’s basically how your application is kind of rendered and in terms of the arrangement of the objects. That’s the core focus of the layout testing that we have seen.

The tools like Applitools and Perfecto Mobile, for example, has been providing visual inspection from the beginning. The Galen framework is a lot more focusing on the responsive web design. So what we will be seeing and what we will be kind of doing today is for the responsive web design because what we are also seeing is the industry is kind of moving towards a lot more towards HTML 5 and responsive web design. So the Galen framework is for now focused for the responsive web design. But there are tools and technologies that we have implemented in a lot of different engagement that supports both the mobile web and desktop web applications. Right.

For today, the focus will be all around the responsive web design. So moving to the next slide. What is Galen framework? The Galen framework has been out there for a little over a year. We are seeing a lot more traction and enterprises start adopting the Galen framework. The very cool features I would say a powerful feature of Galen framework is entirely built on top of Selenium. So as we have been using Selenium. A lot of folks are using Selenium for functional automation for quite a long time.

So for them to adopt the Galen framework is very easy because since they’ve been built on top of that it provides a specific language on top of Selenium to do the UI and visual layout testing. Right. So basically it supports both the desktop browsers as well as mobile web. As we have seen in the responsive web design though it is targeted for the mobile devices with the different found factors, but for the testing purpose you can use it within the desktop itself. You can resize the window automatically the applications that is UI. Right.

So it has you to do the automation within the local environment when you want to test the different UI layout. By nature, it’s built on Selenium. So pretty much you have access to the entire cloud system that is available for Selenium like Soft Labs or Browser stage or open stage or any solution that you want to use for your web for kind of a scalability purpose. Galen framework supports all of it. Right. So moving along to the next slide. It takes a little bit of time to … so basically, how does it work? I mean, it’s very similar to Selenium. It’s not going to be any different from the execution from the architecture perfect.

It goes through the same steps that Selenium regularly would. What Galen framework allows you to do is kind of interject the solution or the validation more specific to the UI layout. So there you can see what is the order of the elements that you are looking for. What is the color? What is the relationship between one object to another object? So it can provide all of that. Galen framework or the Galen specs. So you provide the specifications of the different objects and put the relationship of the association of those objects in a spec file.

So we can see in a real example so you will be able to kind of understand how the specification files are done. If you are to change, you can see how the specification looks like. Right. It has very basic feature that you would expect from any framework in terms of reporting, in terms of the data it collects, and in terms of the reports, in terms of the screen charts, and how you can accurately identify the error and helps you to kind of get down to the specific problems. Right. It has a built in image comparison. You can use that to the image comparison.

You can also verify through a specific part of the screen. That’s the way Galen framework works. You can also go to the Galen framework website to read more about it. Before I go into the details, I just want to kind of get the response. So as you can see, we are also seeing a lot more in the industry the adoption of the responsive web design. It’s kind of slowly we are seeing that the enterprises are moving towards the responsive web design as it becomes a lot more standard in the industry. The applications are kind of moving towards all the responsive web design based architecture.

As opposed to just I agree totally that responsive web design is coming toward a lot more to the standard. Right. So just to go into the actual demo that we will be doing today, right. Using the Galen framework, I’m taking Tesla Models website as an example. So as you can see in the screenshot, this is two issues that they are kind of trying to validate. One is between the desktop version and mobile version. It’s kind of two different screen size. As you can see, there are like you can see two or three major changes between one version to another version. One is the order of the buttons.

So instead of horizontally stacked, it’s vertically stacked between the desktop version and the mobile version. That is one change. So that change … currently there is no way to do that in the standard framework that is out there using the Selenium. That’s where the tools like Perfecto Mobile or Applitools. Galen framework allows you to specify what should be the relationship between the other button and desktop button. One thing that we would be verifying. Also if you look at the top menu that totally changes between the desktop and the mobile.

In the desktop, it is all kind of laid out on the main screen; but in the mobile version it is a menu option that is available for you to click to see the different menu options. Right. Finally, you can also see that several links are missing to optimize the screen properly. So the upload links and other links are kind of removed from the mobile version. Right. So in our automation in our demo, this is where we are going to kind of focus doing this simple demo, but you will be able to kind of understand the concept and understand the power of the Galen framework. Right.

So before I kind of go into the code, I just want to kind of highlight the specifications that we are return to verify what was on the previous screen or slide. One is about the link verification itself. So as we saw the app for leasing. Right. So as you can see this is a couple of samples on the Galen specifications. So what you do is in the specification, you define the object that you are interested. So on the left hand side, put in the specification that we are interested in for missing link. Right. So you want to put what is the page that is associated. As you can see, you can also specify the associations or conditions specific to various found factors.

So here, I’m seeing that on the mobile and tablet I want that a link should be missing. So I can view a specification of association condition that it should be absent. So this is the language that the Galen provides. It is a very detailed robust language specification available, but this is very basic one that you want to specify that you want to see the link is missing on the mobile and tablet. Right. We will see on the next slide what is a mobile and tablet definition means. As you can see at the bottom of this specification on desktop, we are seeing that the text should be visible and the text for that link should be apply for leasing, Right.

So this is a very simple basic example how you can define a specification. As you can see on the right hand side where we are giving the relationship between the button. So we are kind of picking up two buttons. The other button for mobile and tablet. It should be inside a button group called the menu button group. We are also seeing that the mobile should be above the other button. Right. So this is the way that you would be able to define. When it comes to desktop, as you can see once again it has to be inside the menu button group; but it should be left out to the desktop. As you can see, there are two different object identifications.

So that is entirely based on the Selenium X path and Selenium object identification strategy. That is the same strategy Galen uses. Galen basically allows you to kind of extend that and provide the various syntax that you want to verify in terms of the object relationship. Right. So this is really basic specification. We are going to run this for three different configurations. So this is how you define the various configurations. So I kind of took three here in the same test case for three different configurations and see what fails and what works.

But you can pretty much put all different combinations that you want to put, and you can just run it without changing anything. Here, I’m defining mobile, tablet, and desktop. The only thing that is varying here is the size of the window. I’m also going to run this demo on the desktop so you’ll be able to see that it resizes the window automatically. It renders the application to verify all the properties that we are looking for is as expected. So once again, I’ll show the report also. There is like if you look at the Galen framework and Soft Labs documentation, there is a lot more information available how you can integrate this with Soft Labs and other clouds that you want to execute. Right.

So I’m trying to run this through Jenkins so I have a sample project that is set up to run this demo. But before, I just want to show the core base also. So as you can see here … let me show you. This is my test. So it’s a very simple test all I’m saying is load the home page. You run this specification that is defined. This is on the slides. Also I’m seeing that running for all the devices that are different. So if you open up my configurations or I define all the tags it should ask the number of examples that I have, ask the number of configurations that I have. It will try to run that. If I look at the specs, this is how my specs will look like.

So I kind of put everything into one, the button ordering as well as the link. As you can see here, you can put what are the specific insertion that you want for mobile tablets and what are unique about that stuff. Right. As you can see here, you can specify the relationship. You can say that inside particular object should be there. It is also you can use some kind of a range. Like so many times you want to kind of say that it has to be within the space between the two buttons has to be a specific range, you can do a range. There are like a lot of other attributes that it can verify the color of the button should be white.

It can give you a range of color. It can also give a percentage of approximation so that the framework would intelligently use that to verify if it falls within the range or not. Right. So it is very powerful. If somebody wanted to go and look at the detailed specification language, so they would kind of be able to use and perform the testing project effectively. So I’m going to run this example. Once it kind of completes, we can see that running it locally … like I said, it’s going to run just loading the Tesla Motors home page for three different UI. One is for the mobile, and one is for the tablet. One is for the desktop version.

Once it completes the test, the test is a very simple test because it’s just loading the home page and find out whether it is loading correctly or not. Then once it completes, we will look at the execution report. The report also is very cool. Right. So it completed. Let’s look at the report. I’m going to explain the reports. It pretty much gives very detailed information. Let me open up here. So this is a test report out of the box from Galen. Right. So as you can see here we ran three different configurations. One is for the desktop. One is for the mobile, and one is for the tablet.

It gives a very high level dashboard of what all the different combinations that you ran and what is the state of how each of the steps within the configuration. Right. As you can see, there is one test failed out of the different combinations. So if you have to take for desktop, you can see that it highlights where the failure. Any place that it has stopped you can see the checkmarks and see that it indicated pass. Nothing else fails. You can see that you can expand it. If you want to see what is the content, you’ll be able to see the actual insertion.

The very cool thing about that is it shows the screen shot what was our insertion in the topic and see what was the condition. You can also see in the screen shot where the actual insertion happened. Right. So if there is no red indicator, it means that everything is passing. Say, for example, if you wanted to see whether it is … so this is the menu button blue. As you can see, the other button is within the menu button group which is as we expected. Right. Many times when the developers change some of the chip, if you want to enforce that, okay, because this starts the UA.

You’ll be able to kind of catch because if the other button is outside the menu button group, you’ll be able to catch this one. Right. So like I said, the reporting is very cool and really useful because it gives the condition of the insertion that you have. Also, it usually gives how it passed and what verification it passed. Right. Looking into the failure, as you can see our condition was that the button group should be below the header. This is the header. The distance between the header and button group should be 20 to 45. Also, it gives what is the actual distance. Right.

It says that currently it is 15.9 pixel, whereas the range that we gave it was 20 to 45 pixel. Right. So that’s reason that it marked this particular insertion as failure. So you can adjust it. It gives what is the insertion range. It also gives the actual value. So that is a really powerful feature. You want to kind of use it and run the reporting. Right. I just want to kind of restate Galen framework you should look at their website to understand what are all the ways that you can verify and what are the values, what are the layout verifications that might be very important to your application and how that is supported as part of Galen framework.

You can see it. Galen framework is one of the tools that are like a lot of other tools available on the market like we said Perfecto Mobile and Applitools all kind of provide the same level of service to perform the visual automation. Right. So before I just want to also reiterate the fact that as we kind of using this visual layout automation, how do you kind of integrate this part of the continuous delivery? Because the CI/CD has become so important in today’s life cycle of the application development. How you can kind of integrate the visual layout test?

What are the things that you have to take into consideration to make it the continuous delivery very successful? Right. So what we are seeing is this shift left upload, especially moving the QA aspects more early in the cycle. Right. The shift left is very important challenge that we are seeing. Like, for example, they explained automation while having the visual automation as part of the early cycle so that you will be able to start doing your automation well in advance even without the back end necessary components being double up on. Right.

That’s where I think we have seen that the media has become a lot more important because it allows you to be able to enables you to kind of start writing the test cases early in the cycle. You can start putting as part of define the CS cycles. So early in the cycle you will be able to kind of start running your test cases and get the necessary processes to define early the cycle. Right. Unified automation is another key important aspect that you kind of want to take into consideration. You want to have one framework that allows you to do the website cases and mobile automation.

When it comes to whether it is an issue or responsive web design or HDML site or if you are using any other framework like Kony or Phone gap or any other framework that you might be using to build your application. Right. Those are the key things as part of this one. Automating the entire quality process whether it is a CA or deployment like I talked about the virtualization of the test data. Right. All of that is a very important challenge. We have solutions we have implemented successfully in many of our engagements. So we have our framework that supports ability.

We do have integration with Soft Labs, Appium, Galen framework, Perfecto Mobile. We do provide value added frameworks and services on top of these tools. Also, we do provide as part of our service engagement the CS/CD implementation. Our framework supports providing early implementation and early adoption of the automation in your mobile application life cycle management. Right. So before I go into the next section, I just want to kind of quickly get feedback.

If any of you are interested, we would be able to share a follow up session to give more details about automated framework that we can kind of help you to do unified automation like Galen framework and Perfecto Mobile solutions.

Harshal Vora: Yeah, as soon as we have the poll all answered, we’ll do more Q&A session. We’ll open the Q&A for the audience. Myself and Siva will go through all the questions that people have. So as you highlighted, the Info Stretch framework is unified automation framework that will enable you to do functional automation, visual automation, degression automation, as well as, the web services and web channel automation. So it gives you the whole approach where you can integrate your CI/CD. You get very conclusive continuous delivery with one framework rather than having three or four different frameworks for each kind of automation approach.

I think, yes, we are getting in the results for the polls. We’ll keep this poll open for a couple of more seconds. Then, we’ll have the last 15 minutes or so for our Q&A session. As we saw the open source and the way the visual layout automation testing is shaping up, the next step is going to be the whole nonfunctional testing aspect of it. The Info Stretch framework gives us an inside into that nonfunctional aspects of testing as well and how to automate that whole process. So we are going to get a very conclusive testing automation strategy and CI/CD strategy that will enable the enterprise to get to their markets faster. Okay. So let’s go ahead and do the whole Q&A session.

Moderator: All right. First question, does the Info Stretch framework support visual layout testing?

Harshal Vora: Yes, as we indicated the Info Stretch framework integrates … has a very strong integration with Galen framework we leverage. So the whole idea of behind Info Stretch framework is making open source more effective. So the whole framework is built on the open source like Selenium and Galen. So, yes, we do support the visual automated testing.

Moderator: Okay. Great. Thanks. Next question. Is this framework good for verification for whether a link is missing in a mobile version?

Sivakumar Anna: So, yes. Let me take this question. Yes. So like I said in the demo presentation also you can check various aspects of the object that is there on the web page whether it is a link or whether it is an object that is arrangement of the object or color of the object that you are looking for. So there are various elements that you can do. Yes, you can do much more than what we showed in the demo today in terms of the objects that are in the web application and the mobile application.

Moderator: Okay. That’s good to know. Thanks. This person says this type of testing would be fantastic on our Windows app. Do you have any suggestions for an IDE or tool that would allow him to do that?

Sivakumar Anna: So, yeah. I think the Info Stretch framework definitely is a good candidate for this particular requirement because it allows you to kind of do the IDE where you can do the object inspection, the whole object management. You’ll be able to kind of define the parameters like whether you want to run if for Soft Labs or whether you want to run it locally. So that one recommendation I would kind of put it out, but definitely like the Eclipse or any other ID that you might be using today would be a candidate because the Galen framework supports any of the standard ID that are out there on the market.

Moderator: Okay. Thanks. This person wants to know can Galen be used to run browser compares between different browsers on the same desktop?

Sivakumar Anna: Yeah. I think that the model we showed today … we ran it for one browser, but I wish we had more time to show additional demos. But you can pretty much configure to run for various browser combinations. You can take Safari, Firefox, Chrome. You can run it. You can also … I think that’s where you can also leverage some of the Cloud solutions that are out there like Soft Labs or Open stack. So you’ll be able to utilize more effectively so that you can run all these things in parallel to kind of effectively perform your automation.

Moderator: Okay. Cool. Thanks. Can you also check the background color or text color and font details?

Sivakumar Anna: Yes. That is correct. So you can check the color attribute. It can give you a range if you are not sure what is the color. It can give all different attributes of the object that you see in the application.

Moderator: All right. This question is kind of related to one you just asked. Can Galen be integrated with an IDE?

Sivakumar Anna: Yes. The whole Info Stretch framework is an automation studio that we have integrated Selenium, Appium, as well as, Galen framework into one automation studio. The whole IDE integration, we have taken care of it within our Info Stretch framework.

Moderator: All right. Thanks. Can the Galen layout specification be used with fixed client automation tools such as Squish?

Sivakumar Anna: I probably need to kind of get back to you on this question because I don’t think we kind of looked at it, but the Galen framework itself is a very flexible framework. Currently, it supports Java and Java Script. I think the model we saw today was in Java Script, but it does support Java. For other languages and other things, we have to kind of look into it.

Moderator: Okay. Thanks. Maybe you can answer that question more offline. All right. What about broken links?

Sivakumar Anna: Broken link is definitely you can test it. So I think in one of the examples we saw today you can identify what all the links are working. If there is, for example, if you want to see if a link is to an email and you want to see what is that email that it is pointing, you’ll be able to verify that email. So you’ll be able to kind of see what are the links that is failing by clicking whether it is working or not. You’ll be able to kind of verify that as part of Galen framework and some other solutions that we have as part of our framework as well.

Moderator: Okay. Thank you. This person wants to know what is your feedback on visual UI testing for websites hosted on a CMS system.

Sivakumar Anna: So the visual layout testing in general you can use it for any application. So visual layout testing is like how Harshal pointed out in the evolution that it’s been discussed in the last couple of years. We have seen that the tools are popping up. So I don’t think the Galen framework has an intersection for only to specific type of application. You can use for CMS or any other application as long as the objects are accessible through the browser.

Moderator: Okay. Thanks. What languages are supported for writing test automation using Galen or QAS?

Harshal Vora: So using Galen and the QS with the Info Stretch framework, we do support Java and Java Script across the automation cycle. So for Selenium, Appium, as well as, Galen; we do support Java and Java Script.

Sivakumar Anna: Just to add that the Java and Java Script is the language, but we also support the BDD based test altering because that’s where we are seeing the industry adopting BDD a lot more. The Galen framework out of the box doesn’t support BDD. The Info Stretch QAS framework does support BDD on top of Java and Java Script.

Moderator: All right. Thank you. This person wants to know how well would this integrate with my organization’s CI/CD process?

Harshal Vora: As we saw on the slide demo today, CI/CD processes … the whole bottleneck is the visual layout testing which comes right at the end before it goes out to the production and QA environments. If we are able to automate this process, then we can reduce a lot of work on the whole ops team and this gets validated. So the Info Stretch framework has a strong integration with CI like Jenkins or Teamcity or Bamboo or something like that.

Moderator: All right. Thank you. This person says we have a lot of applications developed using grails and groovy for a desktop version. Can Galen be used to completely automate all the test cases like verification and expected page output?

Sivakumar Anna: Yes, very much. Because like I said, it basically sits on top of Selenium. So it will be able to kind of do all the necessary function validations using the Selenium automation. Galen integrates well with Selenium to focus on just the layout part. You will be able to verify the object relationship or the object attributes whether it is the color or links or text. All of that. Right. Yeah, so it definitely supports.

Moderator: Okay. Thanks very much. Well, we had a lot of great questions, but that’s all we have time for today. So that will end our event. I’d like to thank our speakers, Harshal Vora and Sivakumar Anna, for their time. I’d like to thank Info Stretch for sponsoring the web seminar. Of course, a special thank you goes out to you, the audience, for spending this last hour with us. Have a great day. We hope to see you all at a future event.

Sivakumar Anna: Thank you, Beth.

Harshal Vora: Thank you everyone.