Another day, another problem. Test of new SalesForce company release is ahead of me and I don’t want to spend my lifetime in repetitive tasks. Hopefully, the solution is within test automation software. I know it’s difficult to change to different software, once it’s filled with data and configured. So my first thought’s are about what capabilities will be (probably) needed, to pick the right one. I did some quick research around the tools and made a picture. Let’s order them by priorities, to make things easier later:
Must have (discrimination criteria):
- I want it to be free, so that my company won’t spend any money on test automation topic before proving it useful
- I want to have UX friendly environment, so that I don’t need to create test cases in code and deliver as fast as possible. (recording function)
- I want web based testing, so that I can test in SalesForce (Lightning)
Should have:
- I want support of multiple browsers, so that I can prevent user complaints when using mostly used browsers
- I want to support multiple environments, so that same tests could run on DEV, INT, UAT, PROD
- I want to have choice to pick concrete cases for testing, so that I can pick which ones will be used in relation with multiple environments
- I want reusability / test development cooperation with my colleagues, so that I can pass this topic / scenarios later on to somebody else
Nice to have:
- I want to have support for continuous integration, so that never ending development will not bring me grey hair
- I want to have some repository, so that I have backup I can share easily when I have lots of scenarios (ideally GIT connection)
- I want to have reports, so that I don’t have to interpret the results to management all by myself each time
Of course, there can be more capabilities, which could help, but for now and for my purpose, I will start with these. Now we can look deeply into the available tools. Best tool for comparison is simple table, as always. When I ran to features which are must have and the tool did not have it, I discriminated the tool from further research and that made it really quick:
| Tool | RedWood HQ | Selenium IDE | Selenium WebDriver | Provar | Tricentis | Katalon Studio | UFT | TestComplete |
| Free | x | x | x | – | – | x | – | – |
| Web based | x | x | x | x | x | x | x | x |
| Record test case | x | x | – | N/A | x | x | x | x |
| Multiple browsers | x | only FF | x | x | x | FF, Chrome | N/A | N/A |
| Multiple environements | x | – | N/A | N/A | N/A | variables | N/A | N/A |
| Select concrete cases | x | – | N/A | N/A | N/A | variables | N/A | N/A |
| Test dev cooperation | x | – | N/A | N/A | N/A | x | N/A | N/A |
| Support for continuous integration | x | – | x | N/A | N/A | x | N/A | N/A |
| Repository | x | – | N/A | N/A | N/A | GIT | N/A | N/A |
| Test analytics | – | – | N/A | N/A | N/A | x | N/A | N/A |
To do some really “hard-number” decision, I could score the table and put the weights on the capabilities which concerns me the most.
Conclusion
So at first I was fan of Selenium, but the WebDriver version is code-only and IDE version lacks multiple environment function – for each test case I need to record login. At the end, I chose Katalon Studio. I will share the experience in next blog.
To save you some googling time, other comparisons can be found for example here and here.
Game Changer 