XRAY Test Management System Implementation on a Health Cloud Project: Turning Chaos into Traceability
Introduction
In the world of Salesforce projects, joining a product that’s already live can be trickier than building one from scratch. The codebase has its own history, Jira is full of legacy tickets, and documentation — if it exists — may tell only half the story.
That’s exactly what the Jet BI QA team faced when they took over a Health Cloud project that had already been deployed to several customers. Our mission was to maintain and improve quality through structured testing, while ensuring each release remained stable across all client instances.
But there was a problem: traceability didn’t exist.
The Situation Before Implementation
At the start, the QA process looked more like archaeology than engineering.
- Jira contained hundreds of unrelated tickets: stories, epics, bugs, and subtasks, often without consistent naming or linking.
- Requirements were stored in multiple places — spreadsheets, documents, old wikis.
- Test cases existed, but no one could tell which requirement they validated or which defects they were tied to.
- Coverage metrics were invisible — no single dashboard showed what had been tested and what hadn’t.
In short: the team spent more time searching than testing.
This lack of structure was a clear risk to product quality — not because the testers weren’t skilled, but because they lacked visibility. The decision was made: we needed a single, unified test management system.
The Solution: XRAY Test Management for Jira
Why XRAY
After evaluating multiple tools, XRAY was chosen for its native Jira integration, flexible configuration, and robust reporting capabilities. Unlike standalone systems, XRAY doesn’t require leaving Jira — it builds traceability directly into the project’s existing workflow.
Its strength lies in simplicity: it links requirements, test cases, test executions, and defects into one continuous chain of quality evidence.
In essence, it transforms Jira from a ticket tracker into a QA intelligence hub.
Implementation Journey: Step by Step
Step 1 — Bringing Order to Requirements
To regain control over documentation, Jet BI introduced a new custom Jira issue type — Requirement.
This decision was crucial: it gave the QA team ownership over how requirements were defined, validated, and linked to tests.
Instead of trying to force uniformity across existing stories and epics, we created a clean, structured layer where all requirements would live. Each requirement was reviewed, clarified, and approved by stakeholders before any testing began.
This eliminated one of the biggest bottlenecks — unclear or missing expectations.
Step 2 — Structuring Test Assets
Once requirements were centralized, all existing test cases were imported into XRAY. Each test case became a separate Jira ticket, linked to its corresponding requirement.
In XRAY’s Test Repository, the team organized these test cases into folders reflecting product features — patient management, appointment scheduling, notifications, integrations, and so on.
This structure immediately improved navigation and allowed the team to see coverage per feature at a glance.
Step 3 — Creating Test Sets and Reusable Suites
Next, the QA team grouped test cases into Test Sets — collections that could mix different functionalities.
For example:
- Regression Test Set — tests across all critical workflows.
- Smoke Test Set — high-priority checks for each release.
- Integration Test Set — covering cross-system interactions.
This modular approach gave flexibility: one test case could live in multiple sets without duplication.
As a result, running targeted test executions — for instance, regression after Salesforce updates — became a matter of minutes instead of hours.
Step 4 — Automating Test Executions
Each sprint now begins with creating Test Executions from the relevant sets.
XRAY provides a visual execution window showing:
- step-by-step progress of each test,
- comments, attachments, and evidence,
- direct links to associated bugs.
All test run data — passed, failed, or blocked — is visible in real time.
Reports can be exported to PDF or Word in a format configurable for stakeholders, ensuring complete transparency between QA and management.
Achieving Traceability: From Requirement to Release
The true success of this project was achieving end-to-end test traceability.
Now, with one click, Jet BI testers can see:
Requirement → Linked Test Cases → Executions → Bugs → Fix Validation.
The Requirement Traceability Matrix in XRAY visualizes all these relationships on a single page, providing instant answers to key questions:
- Which requirements are covered by tests?
- Which ones failed validation in the last run?
- Which bugs remain unresolved?
For example, if test case TRAC-36 failed in execution TRAC-50 and created bug TRAC-59, but later passed in TRAC-61, it’s clear that the bug has been fixed and the requirement is verified.
This level of insight was previously impossible.
Beyond the Matrix: Test Coverage and Environment Control
XRAY also provides additional reports such as the Test Coverage Report, showing how many requirements are marked “OK” versus “NOT OK”.
This simple metric gave the team a way to track progress over time and present measurable quality indicators to stakeholders.
Moreover, XRAY supports:
- running the same tests across multiple environments (sandbox, staging, production simulation),
- managing preconditions as reusable Jira objects,
- tracking automation tickets for future integration with CI/CD pipelines.
These features laid the groundwork for introducing automated tests later, transforming the manual structure into a scalable hybrid QA framework.
Results: Order Restored
After full XRAY implementation, the QA process shifted from reactive to proactive.
Key outcomes:
- Every requirement has a clear, testable definition.
- All tests and bugs are traceable through a single Jira hierarchy.
- QA engineers reduced time spent searching for information by more than half.
- Product quality increased — more defects were caught earlier in the cycle.
- Reporting became effortless — stakeholders see real-time quality status at any moment.
In short: Jet BI turned Jira from a cluttered issue tracker into a living quality management system.
Lessons Learned: What This Project Taught Us
- Traceability begins with ownership.
Giving QA control over requirement definition ensures long-term consistency.
- Tools only work when supported by process.
XRAY is powerful, but its success came from disciplined setup and maintenance.
- Small structural changes have big effects.
Adding one custom “Requirement” type brought clarity to hundreds of existing tickets.
- Visibility equals confidence.
When quality data is accessible to everyone, decision-making becomes faster and less risky.
Conclusion
The Health Cloud project proves that traceability is not bureaucracy — it’s visibility.
By adopting XRAY and structuring the testing process, Jet BI not only improved efficiency but also built a foundation for continuous quality improvement.
Now, every test tells a story, every requirement has proof, and every bug has context.
That’s what true QA maturity looks like.
Jet BI continues to help organizations transform their QA operations on Salesforce — implementing intelligent tools, sustainable frameworks, and clear governance that ensure every release meets the highest standards.
