Can I Speak To The Manager

Klarna’s AI assistant, powered by @OpenAI has in its first 4 weeks handled 2.3 m customer service chats and the data and insights are staggering:

- Handles 2/3 rd of our customer service enquires

- On par with humans on customer satisfaction

- Higher accuracy leading to a 25% reduction in repeat inquiries

- Customer resolves their errands in 2 min vs 11 min

- Live 24/7 in over 23 markets, communicating in over 35 languages

It performs the equivalent job of 700 full time agents...

So while we are happy about the results for our customers, our employees who have developed it and our shareholders, it raises the topic of the implications it will have for society. In our case, customer service has been handled by on average 3000 full time agents employed by our customer service / outsourcing partners. Those partners employ 200 000 people, so in the short term this will only mean that those agents will work for other customers of those partners.

This is like a £20m saving. For one company. Wild!

I have tried a few and it very frustrating to deal with it for non standard queries or when you're stressed out because money is at stake

When something sounds too good to be true - maybe it is?

Did Klarna really fully replace 700 customer support agents overnight?

I did what few retweeting this do: tried Klarna’s AI assistant.

It’s… underwhelming. It recites exact docs and passes me on to human support fast.

It acts basically as a filter to get to customer support.

Good job on the team for making hallucination not possible - because it seems to spit out the same responses however I ask it, and refuses to go “out of bounds.”

It replaces the “RTFM” step, which could be win, yes.

As soon as I ask or instruct anything that is not a doc, I’m boom talking with a human agent.

(I tried to get it to hallucinate, and to tell me it’s system prompt, model and other stuff, after pressuring it to tell me how to repay less than what I borrowed.

Every time - passed to human support)

Also, almost all questions I ask about payment terms or problems the chatbot tells me - in various ways - to talk to the merchant if I have a problem, not to Klarna.

Assuming this is what most support tickets were - sure, the chatbot reduces tickets by rerouting these ppl.

And yes, this is more likely. No one reads the docs. These assistants turn docs into chat text, that people read!

My sense is this is what is happening, and every company is already doing it (eg Vercel did this recently after a hackathon project!) or will do so soon:

> @rogueboykilla: Well maybe 2/3 of the people that uses it don’t actually need real support, they just need stuff that already exists in the docs

THIS is what the real story should be btw.

> @uditranasaria Bilt's customer support is similar. I think this story is potentially more a story about the simplicity/stupidity of the average support request rather than the impressiveness of AI agents.

Nice one!

By filtering out stuff that has zero need to reach a human (as it’s long as it’s a standard, generic question with a standard answer) we did something simple, and not radical.

Also: context matters!

Klarna is a middleman.

The customer buys from the merchant and Klarna sells defaulted payments to collections agencies!! Customers think they have a relationship w Klarna. But they don’t really! Most support is redirecting the customer to another party.

> Professional Opinion Flipper: Its an IPO-hype play 😉

And if you’re wondering what the goal of sharing this detail ~a year before a planned IPO: yes, it’s likely Klarna wants potential investors to believe they are buying into an “AI edge” company. If that would work, they could trade at a premium.

A good try, we can give it that?

Ok, one underrated area with even simpler chatbots is how they can communicate in other languages.

Klarna's AI bot talking to me in Hungarian was not anything I've experienced before with a commerce site. Yes, it's ChatGPT (that can do this) but I now see how this is v powerful.