As part of the Excellence in Quality blog, we met with Richelle Paulsen, the QA Director at Cetera Financial Group, to discuss the role of QA in supporting Cetera’s growth.

Context: Over the past five years, Cetera has grown from approximately 7,000 advisors and 1,300 employees supporting $242 billion of assets under administration (“AUA”) in 2018 to approximately 12,000 advisors and 2,800 employees supporting $475 billion of AUA today. In October 2023, Cetera announced a reinvestment from its main investor – Genstar Capital, a leading private equity firm, to expand its business further. “This reinvestment from Genstar affirms Cetera’s proven ability to create value,” said Cetera Holdings CEO Mike Durbin.

Richelle, thank you for finding time for this interview. During the last five years, Cetera made several acquisitions in the wealth management business (Avantax, Securian, Voya, to name a few). At the same time, the firm has been actively developing its IT solutions to improve the experience of its clients. To me, it looks like upgrading a plane in mid-flight while adding more passengers. How do you manage to maintain top-quality standards in such a dynamic environment?

Richelle: We operate in a very challenging environment. I would like to start with the part of your question about the tech upgrade of Cetera platforms.

There are several challenges that we face when shipping our updates to production:

• High competition on the market
• Highly regulated environment
• Specific business domain that requires a deep understanding of the FinTech industry
• Complex architecture with multiple integrations, upstream and downstream systems
• A mix of legacy and new systems
• High requirements for the performance and resiliency of applications
• We need to be very efficient

Wow, that’s a huge list! How do you address all these challenges?

You must have several things in place:

• The IT team members should have a deep understanding of the FinTech industry and regulatory and compliance requirements. There are many industry examples when a lack of this knowledge dramatically reduces the productivity of teams and the quality of applications.
• The QA team needs a strong technical background, hands-on experience with databases, integrations, APIs, automation and performance tools, and monitoring tools, and an understanding of architecture, dependencies, ETL, and messaging protocols. The team must also be able to review code if needed.
• High level of automation. In a FinTech company, it is quite a challenge to build efficient automation with all these data dependencies and integrations. So, your QA team needs to be creative and use the proper automation instruments at the right levels (e.g., unit, integration, E2E, etc) to maximize the automation efficiency.
• Shift Left philosophy. In our case, it is much broader than just shifting testing to earlier stages. We help business analysts to write technical requirements, document architecture and legacy systems, analyze quality risks at the project initiation phase, create production workload models, participate in performance engineering, and help increase unit test coverage. We have also helped to build a chain of environments that allows us to do a wide variety of testing while keeping UAT and integration environments stable.

You should have quite a strong QA team to fulfill all these requirements. How is it structured? Do you rely on your employees? Do you use vendors?

At Cetera, we use a mix of our staff and third-party resources. Allied Testing has been our main QA partner since 2014. We’ve come a long way together, starting with building QA processes and driving all the “Shift Left” initiatives I mentioned earlier.

With a strong team and the right methodology, we significantly reduced the time to market for our applications and plan to improve it even more.

I imagine gathering the right team and building all processes took plenty of time. Let’s talk a bit more about the M&A challenges. Cetera has recently made several acquisitions of other wealth management companies, with the subsequent migration of their advisors to its platforms. Every M&A is a complicated story—from the IT perspective, it only starts when the deal is done. Each deal has many hidden technological risks. How do you approach them and ensure successful delivery from a quality perspective?

Richelle: This is an interesting question. M&A projects, by nature, have their complexities:

• Acquisitions can be unexpected and have very strict deadlines. You know how it happens – due to the confidential nature of such transactions, until the deal is closed, not everyone knows about it. When it is closed, you are typically given a deadline by which the new business should be integrated.
• M&A affects multiple company systems. For every acquisition, you must properly integrate the new business and its clients into your processes.
• M&A consumes resources from all teams. M&A requires involvement from everyone — from sales reps and compliance teams to customer service and technical teams.
• Acquisitions can bring additional risks to the organization. A poorly implemented acquisition can impact a newly-purchased business and its existing users.

So, what is your secret recipe for successfully delivering an M&A project?

Richelle: This recipe has many ingredients, but if you gather all of them, a positive outcome is inevitable. Here is what you need:

• Vision that the quality and success of the company go hand in hand. Our senior IT and Business Executives have this vision, which drives the organization’s iterative improvement processes.
• Strong Subject Matter Experts in each team. Without people with a deep understanding of your business flows and typical user scenarios, you shouldn’t even think about stepping into these waters.
• Deep understanding of the M&A scope. Before upgrading or integrating any systems, you need to identify the full scope of the project, all dependencies, and potential technical and business risks. If you prepare properly, this will increase your chances of successfully delivering an M&A project.
• Efficient collaboration between teams (PMO, BA, Dev, QA, Security, Infra, and others). We invested much time and effort in building strong relationships between all teams. This has always been strongly supported at the Executive level. And now, when we do multiple acquisitions per year, we benefit from this investment.
• Strong project management skills. To successfully deliver an M&A project, you must have very organized and proactive teams. This requires organizational skills and the ability to manage technical initiatives. For example, the QA team helps coordinate data migrations, integration projects, and others.
• People with a deep understanding of architecture, integrations, upstream and downstream systems, and technical dependencies. In our case, the QA team fully meets this need — we have been testing our apps for years and know both business requirements and what’s under the hood of the systems. This knowledge allows them to identify and address risks in advance.

After several acquisitions, we formed a so-called M&A SWAT team. This team consists of the most knowledgeable subject matter experts in each field. Every time a new acquisition comes, they jump on it and start analyzing, planning, assessing risks, forming teams, and doing everything needed for successful delivery. We organize our work in a very efficient way – we can inflate our QA team during acquisitions and release excessive resources when the migration is finished.

Based on what I hear, you have already gained much acquisition experience and know all the ins and outs of this process. I have a question, however. I doubt you had all this knowledge during your first M&A. How did you develop this methodology?

Richelle: After each M&A, we analyze the lessons learned and consider how to do it more efficiently next time. More importantly, I have very strong support from the Executives to modify the QA team as needed and make adjustments whenever I see any inefficiencies in the processes. This helps us to be very flexible and quickly learn from our previous transactions. As a result, our organization is getting stronger with every new acquisition. Our processes are improving; our teams collaborate better, we have new integrations with broker-dealers, and our range of services for the clients is growing.

That was very insightful. Thank you! You mentioned using Allied Testing as a QA partner. Can you elaborate more on your requirements for the QA vendor and how you build the relationships that allow you to deliver such complex projects?

Richelle: Sure. In my career, I have used several vendors, and with time I came to certain principles that allow me to build very productive relationships with a vendor:

• High requirements and focus on the result. I don’t need just a manual testing vendor. I need a proactive vendor that will constantly offer improvements and, more importantly, be capable of delivering them. I need a vendor with professionals capable of wearing multiple hats: PM, System Analyst, UAT Coordinator, Technical Analyst, Data Scientist, and Developer.
• Full transparency. If I don’t like something about the vendor’s performance level, they will know it. But it works in both directions — if a vendor sees a way to improve our processes and needs my help, I will support it and do my best to achieve it.
• Flexibility. For me, having a certain flexibility from a vendor is very important. This includes free bench resources and the ability to supply strong resources with FinTech knowledge quickly. This requirement is very important when you work in a constantly changing environment.
• Clear and proactive communication. When you work with complex projects, any delay in identifying a risk or problem can cost you money, so you’d better have people around you who can clearly articulate all issues.
• Focus on maintaining reasonable attrition. Having people on the team who know how things work is very important. When a group of people with the right skills work together for 3-5 years (or 10 years in our case!), their collaboration significantly improves. They work as a single organism, and everyone can start outperforming.

And the final question for today. Given the current excitement around AI, I’m curious to know if your team has begun experimenting with AI technologies?

Yes, we already have several working groups analyzing how AI can improve the productivity of the SDLC and the QA team. The results of this analysis look very promising. Still, given that we operate in a very regulated environment, we collaborate with Legal, Security, Infrastructure, and other teams to assess all solutions.

Richelle, thank you for this interview! This was very insightful. I wish you and Cetera all the best with future acquisitions and further business growth!

Cetera Financial Group, which is owned by Cetera Holdings (collectively, Cetera), is the premier financial advisor Wealth Hub where financial advisors and institutions optimize their control and value creation. Breaking away from a commoditized and homogenous IBD model, Cetera offers financial professionals and institutions the latest solutions, support, and services to grow, scale, or transition with a merger, sale, investment, or succession plan. Cetera proudly serves independent financial advisors, tax professionals, licensed administrators, large enterprises, as well as institutions, such as banks and credit unions, providing an established and repeatable blueprint for scalable growth.

Allied Testing, a company founded in 2000, is a leading QA and testing specialist firm with the main focus on the capital markets, trading and finance industry. Over the years we have earned a reputation of premiere application quality management specialists. We work with CIOs, VPs of Development and Quality Directors to improve stability and resilience of their systems, develop or fine-tune QA and sourcing strategies, and take on non-trivial testing tasks, while driving down costs.

Starting from February 15, 2024, Atlassian will no longer support or update its server products. Users of Jira, Confluence, Bitbucket, and Bamboo should plan to move to Cloud or Data Center (DC) solutions.

Even though Atlassian offers detailed guides for moving to the new systems (example), large organizations with thousands users, high security requirements, multiple add-ons and integrations, comprehensive customizations, and workflows, may find this change challenging.

In this article, we share valuable tips based on the Allied Testing experience helping big financial companies make this switch.

Form a team with the right skills

To ensure a successful migration project, it’s vital to form a team with the right set of skills. Depending on the project size, you may need more than one expert in each role, or some team members may take on multiple roles.

• Project manager. Keep in mind that this project may not be a top priority for everyone in your organization. Therefore, the project manager needs to be persuasive and able to get people to clearly outline their migration needs.
• Product expert / administrator. This person is key to the operation. They need to understand all workflows, configurations, integrations, and more. They should also have basic knowledge of infrastructure and security, working closely with the relevant teams. If you’re moving multiple products, you’ll need an expert for each.
• Database engineer. An expert on databases saves a lot of time during migration.
• Atlassian Migration Assistants Expert.Although the Atlassian tools are designed to simplify the process, knowing their specific features and limitations is important to streamline the migration and avoid problems.

Dedicate substantial time to analysis and preparation before starting the migration

When planning your migration, start by creating a project plan and analyzing key elements like data types, user permissions, workflows, custom fields, add-ons, and integrations. These steps are generally recommended by Atlassian.

Here are specific points to keep in mind:

• Make sure all user emails are unique and valid. The Cloud system uses emails for user identification, so it’s important to sort these out before migrating.
• Clean up any outdated data, like finished projects or old JIRA filters and dashboards. This helps simplify and speed up the migration.
• If you’re using Atlassian products integrated with other internally hosted tools (like TestRail), discuss with your Infrastructure and Security teams how to manage this integration in the Cloud. Sometimes security policies restrict certain integrations, so it’s better to determine this early.
• If your server version was restricted to internal access and you’re moving to the cloud, coordinate with your Security team to determine which access permissions should be enabled in the Cloud. This will help you select the right Cloud version.
• Migrating JIRA workflows with custom fields can be tricky. Identify these workflows early and handle them separately – you may need to remove custom fields, migrate, and then add them back in.
• If you’re moving from Bamboo to Bitbucket Pipelines, be prepared to rewrite your build scripts. Plan accordingly for the time and resources this will take.
• If you migrate Bamboo to Bitbucket Pipelines and plan to deploy your code to local servers, consult with your Security and Infrastructure teams to ensure there are no security issues.

Set up a migration PoC, identify any issues, and determine the best solution

Set up a Proof of Concept (PoC) by migrating a small amount of data to both the Cloud and DC versions. This PoC will help you spot any missing features or technical problems in each version of the Atlassian product.

For instance, during our PoC, we encountered an issue with the Azure AD integration. User names were changed from display names to legal names during the single sign-on process, making users only searchable by their legal names. While not a major problem, it did cause confusion among users with name discrepancies.

You may also run into issues with the Migration Assistant during the PoC. In our case, when moving priority schemes for several projects, the Assistant mixed all the values into one scheme, requiring us to sort them out manually. However, it’s important to note that Atlassian regularly updates its Migration Assistants, so this specific issue may be resolved by the time you’re reading this.

Organize User Acceptance Testing with the right user groups

The success of User Acceptance Testing (UAT) hinges on selecting the right users to evaluate the migration project. For JIRA/Confluence, the best candidates are Quality Assurance (QA) and Business Analysts (BA) who use these tools daily and have specific expectations. For Bitbucket and Bamboo, involve Developers, Development Leads/Managers, and DevOps specialists. These are the individuals who regularly work with code, manage repository and branch permissions and rules, and set up build and deployment jobs. Their feedback can be crucial in deciding whether to choose the Cloud or DC versions of Bitbucket.

Follow up with the post-migration tasks and user training

If all previous steps are correctly executed, the live migration should proceed without any new problems. Remember to allocate time for the post-migration tasks, like setting up filters in JIRA or enabling custom fields.

It’s also important to recognize that you’re transitioning to a completely new product. Dedicate ample time to train users on its features, hold demonstrations, and develop guides for administrators.

Allied Testing has extensive experience in assisting its clients with cloud migration. Should you require assistance with the migration, don’t hesitate to reach out to us. We’re ready to guide you through the process.