What is responsible AI – how can it address concerns within financial services?

Why having a responsible AI policy is critical for organizations, customers, and society as a whole  

What is responsible AI?

Responsible AI refers to the development and implementation of artificial intelligence (AI) systems in a manner that meets behavioral expectations and is transparent and beneficial to society. It involves creating AI technologies that are not only technically proficient but also remain fair and free from bias, respect individual rights, and adhere to legal and regulatory standards.  

To achieve a responsible approach to AI, organizations and researchers must consider the entire lifecycle of AI systems, from design and development to deployment and ongoing maintenance. This includes being proactive by taking measures to mitigate risks and unintended consequences. Examples of careful risk management include selecting and preprocessing data to avoid perpetuating existing biases, implementing robust testing and validation processes, and establishing clear governance frameworks. It also requires clear and auditable reporting to ensure that any outcomes can be fully understood.  

It is important to note that although some may use ethical AI and responsible AI interchangeably, ethical AI is just one part of responsible AI, and ethics may vary from company to company (a small community bank will think very differently from a global hedge fund, for example). As such, responsible AI is the principle term used within the industry.

Recent regulatory developments in AI  

Recent regulatory developments across the globe concerning the use of AI in organizations, particularly in financial services, reflect growing awareness and efforts to address the ethical, legal, and operational challenges posed by AI technologies.  

The most significant thus far is the EU AI Act, the world’s first comprehensive AI law, which came into effect in August 2024. Categorizing AI systems based on risk, it has a significant impact on AI use in financial services as high-risk AI systems include those used in critical infrastructure, employment, creditworthiness assessments, and law enforcement. Financial services’ AI management systems require strict compliance with safety, transparency, and accountability standards if they are operating within the EU. Alongside the Act, it’s important to note that GDPR continues to have implications for AI use, particularly concerning data privacy, transparency, and consent in processing personal data. 

Other countries are also looking at bringing in AI regulations. The UK is developing governance frameworks and ethical guidelines, its Financial Conduct Authority and the Bank of England are also exploring how it is best used in financial services, and there are proposals on algorithmic transparency. The UK has also signed the first international treaty addressing the risks of AI. Meanwhile, the US published a Blueprint for an AI Bill of Rights in late 2022, inspired by the EU’s approach as ‘best practice’. Other countries including Canada, China, Singapore, and Australia are looking at AI regulations with global trends also emerging including discussions at events like the G20 and Organisation for Economic Co-operation and Development (OECD). 

When it comes to practising responsible AI, organizations should perhaps attempt to follow ISO/IEC 42001 – the world’s first AI management system standard, providing guidance for organizations to establish, implement, maintain, and continually improve AI Management Systems. The standard addresses the unique challenges AI poses, such as ethical considerations, transparency, and continuous learning. Its primary purpose is to ensure that AI systems are developed, deployed, and managed in a responsible, secure, and accountable manner. 

The concerns of using AI in financial services 

Implementing AI (generative, predictive, and agentic AI) and using it for financial crime prevention in banks and other financial institutions offers several advantages but also comes with understandable concerns. Here are some potential concerns banks might have: 

• Data privacy and security: AI systems require access to a huge amount of data to function effectively. With this in mind, banks may be concerned about the potential for data breaches or misuse of sensitive customer information, which could lead to regulatory fines and reputational damage. 

• Bias and fairness: AI models can inadvertently learn and perpetuate biases present in training data. Banks need to ensure that their AI systems do not unfairly target specific groups or individuals, which could lead to discrimination and legal challenges. A list of biases has been included below. 

• Model explainability and transparency: Many AI models make it difficult to understand how they arrive at specific decisions. Financial institutions must be able to explain AI-driven decisions to both regulators and customers. 

• Regulatory compliance: Financial institutions are highly regulated. Banks must ensure that AI systems comply with existing regulations and are able to adapt quickly to new regulatory requirements within the AI and financial crime prevention spaces. 

• Over-reliance on technology: There is a risk of banks becoming over-reliant on AI for detecting and preventing financial crimes, potentially undermining human judgment and expertise. This might lead to complacency and a failure to detect new tactics that criminals are using, which AI may not yet be trained to recognize.

Initiating a responsible AI process can help organizations address these concerns while ensuring that they take a balanced and cautious approach to AI implementation.  

The importance of responsible AI  

Responsible AI is extremely important and encompasses various principles such as fairness, accountability, privacy protection, and safety. This approach aims to mitigate potential risks and negative impacts associated with AI, such as bias in decision-making (see below), job displacement, or the misuse of personal data.  

Additionally, responsible AI is important because it advocates for inclusive development practices that involve diverse perspectives and stakeholders in the creation and deployment of AI technology. It also emphasizes the importance of explainable AI, where the decision-making processes of AI systems can be understood and interpreted by humans, fostering trust and enabling meaningful human oversight.  

Examples of potential AI biases   

There are many potential biases that need to be avoided when implementing responsible AI. These biases are a principal reason that organizations may be concerned about using the technology. These include:

• Sampling bias: If the AI is being trained on data that is not representative, the model may be biased.  

• Implicit bias: Algorithms may unintentionally reflect biases present in training data.  

• Temporal bias: Models may be trained on outdated data, leading to biases that are unrepresentative of the current climate.  

• Racial bias: Algorithms may favor certain racial groups. Famous cases of racial bias in AI have been seen in Google software and other AI image generators.  

• Gender bias: Gender bias may be present in AI applications.  

• Ageism bias: Bias related to age can impact model predictions.  

• Ableism bias: Models may discriminate against people with disabilities.

Alongside these biases, models may also struggle with outliers and edge cases, omitting such data results in their findings.  

These biases must be avoided as they could be the difference between banks and other financial institutions making key decisions that hugely affect customers’ lives and have a significant societal impact. These include approving a loan application, accepting a customer, or insuring a business.  

How responsible AI benefits society

Responsible AI can help society in innumerable ways, beginning with contributing to a fairer society, with big decisions that impact the everyday lives of all of us done through transparent, unbiased, and safe AI models. Some of these might include AI taking on menial, repetitive tasks such as text-based customer service or data collection, freeing up human potential for higher-value work, improving mental health and life fulfilment as a result.  

Almost every industry can benefit from AI in some form whether using AI-powered tools to enhance healthcare or using AI to optimize important infrastructure. In fact, Google published a report on how AI helps drive progress in all 17 of the UN’s Sustainable Development Goals (SDGs).  

With industries able to use AI responsibly, it is society that benefits. From using machine learning (ML) analytics on emissions data to help improve the environment through to combining ML analytics with natural language processing to create a multilingual engagement platform for those that speak different languages, the opportunities are plentiful.  

It is understandable that some might fear for the future of their current jobs, but great advancements alter how resources are used, how hiring happens, and how skillsets change to adapt to new ways of working. The evolution of AI in business is no exception to this age-old trend. Though some jobs will be replaced, AI is more likely to enhance current roles and create more jobs than are lost.

Why responsible AI is important for financial services

With generative AI and predictive AI becoming used in various industries, and the number of companies using it set to grow more than tenfold (one estimate believes $85.7 billion will be spent on generative AI by banks in 2030), putting responsible AI policies in place sooner rather than later is paramount.

The purpose of having a responsible AI policy in place is to establish legal and behavioral guidelines and standards for AI development. It is intended to foster a culture of responsibility and continuous improvement in the AI ecosystem that meets regulatory requirements such as the EU AI Act and any future laws. Such a policy must cover five main areas to provide the oversight and understanding that regulators will require: accountability, transparency, reliability and safety, privacy, and security.

These areas can then be broken up into smaller sections that ensure responsible design and development, the promotion of trustworthiness, the mitigation of risks, the encouragement of transparency and explainability, the driving of continuous improvement, and the ensuring of compliance with laws and regulations.

In part two of this series, look at how SymphonyAI is practicing responsible AI in financial services using these five pillars.