Every now and then, customers ask us for additional support for new CI tools.
Currently, Xray provides two free add-ons for Bamboo and Jenkins. These add-ons interact with Xray using its built-in REST API. Even if you're not using these tools, it's quite easy to submit your automated test results from other tools, including TeamCity.
We'll show you how you can integrate TeamCity with Xray and have your automated testing results up in Jira in no time.
This approach can be applied to any other tool - the steps are essentially the same.
Let's go through two use cases: JUnit and Cucumber — since their workflows are a bit different.
In this scenario, we want to gain visibility of the automated test results from some tests implemented in Java, using the JUnit framework. This recipe could also be applied to other frameworks such as NUnit or Robot.
In order to submit those results, we need to invoke the REST API (as detailed in Import Execution Results - REST).
Our project is Maven-based, therefore, the first Build Step compiles and runs the JUnit automated tests.
We add a Build Step of type "Command Line", where we'll invoke the REST API, submitting the JUnit XML report generated in the previous step.
The complete script is as follows:
curl -H "Content-Type: multipart/form-data" -u %jira_user%:%jira_password% -F "file=@java-junit-calc/target/surefire-reports/TEST-com.xpand.java.CalcTest.xml" "%jira_base_url%/rest/raven/1.0/import/execution/junit?projectKey=CALC&fixVersion=v3.0&revision=1234"
We use the "curl" utility that comes in Unix-based operating systems, but you can easily use another tool to make the HTTP request.
Notice that we're using some parameters for storing Jira's base URL, along with the credentials to be used in the REST API.
These parameters can actually be defined at multiple levels. In our example, we defined them at the "Build Configuration" level, but they could also be defined at the project level.
The parameters can also be hidden, if you defined them as being of type "Password".
In this scenario, we are managing the specification of Cucumber Scenarios/Scenario Outline(s) based tests in Jira, as detailed in the "standard workflow" mentioned in Testing with Cucumber. This is clearly different from the JUnit case above.
We need to extract this specification from Jira (i.e., generate related Cucumber .feature files), and run it in TeamCity against the code that actually implements each step of those scenarios.
We can then submit the results back to Jira and they'll be reflected on the related entities.
Overall, our Build Configuration is composed of 3 basic steps:
We start by extracting the tests specification out of Jira and generate the proper .feature files.
The export can take as input the issue keys of requirements, Test Executions, Test Plans or a filter id, which we'll use in this example.
We invoke the REST API in order to obtain a zip file containing the feature files.
We use a Build Step of type "Command Line" for this purpose, along with "curl" utility to make the HTTP request.
The complete script is as follows:
curl -u %jira_user%:%jira_password% "%jira_base_url%/rest/raven/1.0/export/test?filter=11400&fz=true" -o features/features.zip
unzip -o features/features.zip -d features/
Notice that we're unzipping the .feature files to a local directory, so we're able to run them.
The exact syntax for running the Cucumber scenarios depends on the Cucumber implementation being used. In this case, we're using Ruby's variant.
We invoke the "cucumber" command with an option to generate a JSON report (e.g., "data.json").
You may have noticed a trick at the end of the Cucumber line above, i.e., ".... || :". That ensures that Cucumber returns with exit code 0 (i.e., success), so the build may proceed.
In order to submit the results, we add a Build Step of type "Command Line", where we'll invoke the REST API, submitting the Cucumber JSON report generated in the previous step.
We also make sure this step is always called.
The complete script is as follows:
curl -v -H "Content-Type: application/json" -X POST -u %jira_user%:%jira_password% --data @data.json "%jira_base_url%/rest/raven/1.0/import/execution/cucumber"
Notice that we used some parameters related to the Jira server, which we configured at the project level.
Even though there is no specific add-on for a tool like TeamCity, implementing a continuous integration process with Xray is quite easy and this recipe can be simply adapted to other tools. Now you can focus on writing your automated tests and improve your builds. What are you waiting for? 😀
