Opinion: Why Algorithmic Decision-Making Needs to Be Managed

The next time you get a credit limit increase, remember to thank the algorithm because that’s likely who granted it. 

Four out of 5 such increases were initiated automatically by banks, not borrowers, according to new research from King’s Business School and the Federal Reserve Bank.

In theory, this is a classic use case for Artificial Intelligence: Taking a task that’s really just analyzing numbers and removing the cumbersome (and potentially biased) element of human decision-making.

In practice, though, this is a potentially dangerous development — both for individual borrowers and also for lenders who are assuming more risk as a result. Seen in that light, this is actually an ideal scenario for man and machine to interact.

By pairing algorithmic analysis with a human touch — and establishing policy guardrails to minimize negative outcomes — banks can still service core customers and innovate without risk spinning out of control and detonating balance sheets.

Why Moderation is Good Business

Consider the metaphor of someone being served at a bar. Alcohol isn’t necessarily good or bad in and of itself. But if a customer is continuously being overserved, they are being tipped from healthy moderation into risk — and that risk can eventually rebound on whoever is serving them.

Four out of 5 credit limit increases are prompted by bank algorithms, according to new research from King’s Business School and the Federal Reserve Bank.

In a similar way, take the case of someone who has been responsibly paying their monthly bills, within whatever credit limits they happen to enjoy. If they keep being offered more and more opportunity to go into debt, they might very well take it — and eventually find themselves in deep waters, unable to keep up with payments. 

That’s not good for anyone — not for the individual, and not for the institution. This isn’t small change we’re talking about, either.

These algorithmic decisions amount to $40 billion in additional credit every single quarter, according to the new study, with most of the increase going to those who are already carrying balances month to month.

This is essentially playing with fire.

Machines Are Tools, Not Masters

It’s not surprising that algorithms are operating the way they do. They have been designed to identify opportunities and make money; nothing nefarious about that.

And one of the easiest ways to do so, as opposed to the laborious and expensive process of acquiring new customers, is to maximize relationships with the ones you already have. 

Indeed, it’s hard to argue with the math of lending established clients more money, especially at a current average interest rate of around 20%. For an industry facing all kinds of economic pressures at the moment, that’s low-hanging fruit that is difficult to resist.

The challenge with algorithms is that — as we have seen with their application elsewhere in society, such as with social media — they are very good at exploiting human vulnerabilities and not great at diagnosing or preventing the eventual fallout. 

Algorithms are good at exploiting human vulnerabilities but not great at diagnosing the eventual fallout.

That’s an example of why some modest policy restraints, as those already used in some other countries, can be a wise compromise that saves borrowers (and corporations, to be honest) from their own worst tendencies.

For example, in Canada and the U.K., banks are required to obtain customer consent for credit increases. That’s an example of smart policy: robust consumer protections, but not terribly burdensome, and not overly restrictive of business expansion, since most consumers would presumably be very happy to accept a credit increase.

But it does introduce enough friction in the process to slow things down, so people and lenders can be more deliberative about the extension of credit. Think of it like the automatic braking process in cars: When software sees a potential crash developing, it swings into action to protect everyone involved.

Gasoline on a Fire

After all, this trend affects vulnerable borrowers disproportionately. The study found that among those with lower credit scores, 60% of revolving balances came as a result of credit limit increases instituted after the account was already opened.

There is no doubt that AI can be a useful tool in evaluating credit limits or anything else. But if you are trusting machines to behave in a way that won’t exploit human vulnerabilities, that’s not a smart bet to make. Eventually, loading up higher-risk borrowers with more and more debt will come back to bite you.

That’s where human beings have an essential role to play, on both ends of any credit limit increase. By all means, marshal all the possibilities of what dataset analysis can do. But being thoughtful about the use of that power, and not simply letting automatic approvals run amok, makes smart business sense.

Because when a bill comes due and isn’t getting paid, it won’t do any good to point fingers at an algorithm.