Research Methodologies

On this page:

• Research process
• Evaluation Criteria
• Quadrant descriptions

Inclusion in the RiskTech Quadrant®

Chartis seeks to include risk technology vendors that have a significant presence in a given target market. The significance may be due to market penetration (e.g., large client base) or innovative solutions. Chartis does not give preference to its own clients and does not request compensation for inclusion in a RiskTech Quadrant® report. Chartis utilizes detailed and domain-specific ‘vendor evaluation forms’ and briefing sessions to collect information about each vendor. If a vendor chooses not to respond to a Chartis vendor evaluation form, Chartis may still include the vendor in the report. Should this happen, Chartis will base its opinion on direct data collated from risk technology buyers and users, and from publicly available sources.

Research process

The findings and analyses in the RiskTech Quadrant® reports reflect our analysts’ considered opinions, along with research into market trends, participants, expenditure patterns and best practices. The research lifecycle usually takes several months, and the analysis is validated through several phases of independent verification. Figure 1 below describes the research process.

Figure 1

Chartis typically uses a combination of sources to gather market intelligence. These include (but are not limited to):

• Chartis vendor evaluation forms. A detailed set of questions covering functional and non-functional aspects of vendor solutions, as well as organizational and market factors. Chartis’ vendor evaluation forms are based on practitioner-level expertise and input from real-life risk technology projects, implementations and requirements analysis.
• Risk technology user surveys. As part of its ongoing research cycle, Chartis systematically surveys risk technology users and buyers, eliciting feedback on various risk technology vendors, satisfaction levels and preferences.
• Interviews with subject matter experts. Once a research domain has been selected, Chartis undertakes comprehensive interviews and briefing sessions with leading industry experts, academics and consultants on the specific domain to provide deep insight into market trends, vendor solutions and evaluation criteria.
• Customer reference checks. These are telephone and/or email checks with named customers of selected vendors to validate strengths and weaknesses, and to assess post-sales satisfaction levels.
• Vendor briefing sessions. These are face-to-face and/or web-based briefings and product demonstrations by risk technology vendors. During these sessions, Chartis experts ask in-depth, challenging questions to establish the real strengths and weaknesses of each vendor.
• Other third-party sources. In addition to the above, Chartis uses other third-party sources of information such as conferences, academic and regulatory studies, and collaboration with leading consulting firms and industry associations.

Evaluation criteria

The RiskTech Quadrant® (see Figure 2) evaluates vendors on two key dimensions: 

1. Completeness of offering 
2. Market potential

Figure 2

We develop specific evaluation criteria for each piece of quadrant research from a broad range of overarching criteria, outlined below. By using domain-specific criteria relevant to each individual risk, we can ensure transparency in our methodology and allow readers to fully appreciate the rationale for our analysis.

Completeness of offering

• Depth of functionality. The level of sophistication and number of detailed features in the software product (e.g., advanced risk models, detailed and flexible workflow, domain-specific content). Aspects assessed include: innovative functionality, practical relevance of features, user-friendliness, flexibility and embedded intellectual property. High scores are given to firms that achieve an appropriate balance between sophistication and user-friendliness. In addition, functionality linking risk to performance is given a positive score.
• Breadth of functionality. The spectrum of requirements covered as part of an enterprise risk management system. This varies for each subject area, but special attention is given to functionality covering regulatory requirements, multiple risk classes, multiple asset classes, multiple business lines and multiple user types (e.g., risk analyst, business manager, CRO, CFO, compliance officer). Functionality within risk management systems and integration between front office (customer-facing) and middle/back office (compliance, supervisory and governance) risk management systems are also considered.
• Data management and technology infrastructure. The ability of risk management systems to interact with other systems and handle large volumes of data is considered to be very important. Data quality is often cited as a critical success factor and ease of data access, data integration, data storage and data movement capabilities are all important factors. Particular attention is given to the use of modern data management technologies, architectures and delivery methods relevant to risk management (e.g., in-memory databases, complex event processing, component-based architectures, cloud technology, software-as-a-service). Performance, scalability, security and data governance are also important factors.
• Risk analytics. The computational power of the core system, the ability to analyze large amounts of complex data in a timely manner (where relevant in real time), and the ability to improve analytical performance are all important factors. Particular attention is given to the difference between ‘risk’ analytics and standard ‘business’ analytics. Risk analysis requires such capabilities as non-linear calculations, predictive modeling, simulations, scenario analysis, etc.
• Reporting and presentation layer. The ability to present information in a timely manner, the quality and flexibility of reporting tools, and ease of use are important for all risk management systems. Particular attention is given to the ability to do ad hoc ‘on-the-fly’ queries (e.g., what-if analysis), as well as the range of ‘out-of-the-box’ risk reports and dashboards.

Market potential

• Business model. Includes implementation and support and innovation (product, business model and organizational). Important factors include size and quality of implementation team, approach to software implementation and post-sales support and training. Particular attention is given to ‘rapid’ implementation methodologies and ‘packaged’ services offerings. Also evaluated are new ideas, functionality and technologies to solve specific risk management problems. Speed to market, positioning and translation into incremental revenues are also important success factors in launching new products.
• Market penetration. Volume (i.e., number of customers) and value (i.e., average deal size) are considered important. Rates of growth relative to sector growth rates are also evaluated. Also covers brand awareness, reputation and the ability to leverage current market position to expand horizontally (with new offerings) or vertically (into new sectors).
• Financials. Revenue growth, profitability, sustainability and financial backing (e.g., the ratio of license to consulting revenues) are considered key to scalability of the business model for risk technology vendors.
• Customer satisfaction. Feedback from customers is evaluated, regarding after-sales support and service (e.g., training and ease of implementation), value for money (e.g., price to functionality ratio) and product updates (e.g., speed and process for keeping up to date with regulatory changes).
• Growth strategy. Recent performance is evaluated, including financial performance, new product releases, quantity and quality of contract wins, and market expansion moves. Also considered are the size and quality of the sales force, sales distribution channels, global presence, focus on risk management, messaging and positioning. Finally, business insight and understanding, new thinking, formulation and execution of best practices, and intellectual rigor are considered important.

Quadrant descriptions

Point solutions

Point solutions providers focus on a small number of component technology capabilities, meeting a critical need in the risk technology market by solving specific risk management problems with domain-specific software applications and technologies. They are often strong engines for innovation, as their deep focus on a relatively narrow area generates thought leadership and intellectual capital. By growing their enterprise functionality and utilizing integrated data management, analytics and BI capabilities, vendors in the point solutions category can expand their completeness of offering, market potential and market share.

Best-of-breed

Best-of-breed providers have best-in-class point solutions and the ability to capture significant market share in their chosen markets. They are often distinguished by a growing client base, superior sales and marketing execution, and a clear strategy for sustainable, profitable growth. High performers also have a demonstrable track record of R&D investment, together with specific product or ‘go-to-market’ capabilities needed to deliver a competitive advantage. Focused functionality will often see best-of-breed providers packaged together as part of a comprehensive enterprise risk technology architecture, co-existing with other solutions.

Enterprise solutions

Enterprise solutions providers typically offer risk management technology platforms, combining functionally rich risk applications with comprehensive data management, analytics and BI. A key differentiator in this category is the openness and flexibility of the technology architecture and a ‘toolkit’ approach to risk analytics and reporting, which attracts larger clients. Enterprise solutions are typically supported with comprehensive infrastructure and service capabilities, and best-in-class technology delivery. They also combine risk management content, data and software to provide an integrated ‘one-stop-shop’ for buyers.

Category leaders

Category leaders combine depth and breadth of functionality, technology and content with the required organizational characteristics to capture significant share in their market. Category leaders demonstrate a clear strategy for sustainable, profitable growth, matched with best-in-class solutions and the range and diversity of offerings, sector coverage and financial strength to absorb demand volatility in specific industry sectors or geographic regions. Category leaders will typically benefit from strong brand awareness, global reach and strong alliance strategies with leading consulting firms and systems integrators.

Copyright Infopro Digital Limited. All rights reserved. You may share this content using our article tools. As outlined in our terms and conditions, https://www.infopro-digital.com/terms-and-conditions/subscriptions/ (clause 2.4), an Authorised User may only make one copy of the materials for their own personal use. You must also comply with the restrictions in clause 2.5. If you would like to purchase additional rights please email info@chartis-research.com