3 very disturbing things happened this week in AI — and you need to pay attention

Overall I’ve been pretty positive on generative AI so far in its fairly young history. I really like what Google has done with Magic Editor, for example, to make your photos look their best. And I often turn to ChatGPT or Copilot to get answers to common questions that I’d normally go to Google for.

I’m also encouraged by Apple Intelligence. Even though it won’t be out until this fall, the developer preview of iOS 18 shows that Apple is finally making Siri smarter while also saving us time with features like recording and transcribing audio within notes and phone calls.

But there’s a darker side to AI that has emerged this week — no, I’m not talking about hallucinations — that has me very worried about whether companies are thinking things through before they sprinkle the proverbial AI pixie dust over everyone and everything. And it’s overzealous humans, not AI, that’s the problem.

Google Gemini Olympics ad disaster

A screenshot from a Google Gemini ad airing during the Olympics

(Image credit: Google)

How do you make an ad go from heartwarming to borderline horrifying? Google has shown us the way with its Gemini Olympics ad, which has already been widely panned for favoring algorithms over human connection.

In the commercial (whose comments have been turned off on YouTube), we see a father using AI to help her daughter gain an advantage in her training to be a great hurdler. Seems pretty harmless at first, but then the dad turns to Gemini to help his daughter write a letter to Olympic athlete Sydney McLaughlin-Levrone.

Google + Team USA – Dear Sydney – YouTube
Google + Team USA - Dear Sydney - YouTube

Watch On

The father explains that he’s “pretty good with words,” but he wants the fan letter to be “just right.” This is a huge insult to the daughter, who should be writing this letter in her own words. This is not something that needs to be just right; it just needs to come from the heart.

In this instance Gemini isn’t just a creativity crutch, it sucks the humanity right out of the room.

Google issued the following statement: “Our goal was to create an authentic story celebrating Team USA. It showcases a real-life track enthusiast and her father, and aims to show how the Gemini app can provide a starting point, thought starter, or early draft for someone looking for ideas for their writing.”

Now I could see the father himself helping as a starting point, but I don’t think AI should be the helper in a case like this.

Create an AI version of you with Instagram!

AI Studio screenshot

(Image credit: Meta)

You might not have noticed but this week Meta shut down its celebrity AI chatbots. And I wouldn’t blame you because it appears no one was using them.

Seriously, why would I want to talk to the AI version of Tom Brady or Paris Hilton? It’s not them. But don’t worry. Meta has already pivoted to AI Studio, which enables creators to customize AI avatars of themselves to answer questions from fans. The feature works with Instagram, WhatsApp, Messenger and the web.

The aim is to let creators and business owners talk to their followers on their behalf. In the blog post Meta says AI Studio can help creators build an “extension of themselves to reach more fans.”

Some of the AI characters that have been made so far seem somewhat helpful. For example, chef Marc Murphy has created an Eat Like You Live There! AI that offers personalized tips for embracing local dining when traveling.

But overall there seems something very empty about using an AI avatar of yourself to engage fans. What’s next, fake fans engaging with fake creators?

Meet your next Friend

The Friend AI wearable in use

(Image credit: Friend)

2024 will be remembered as the year of the AI hardware disaster, with poster children like the useless Rabbit R1 and Humane AI Pin. But that’s not stopping 21-year-old entrepreneur Avi Schiffman from jumping on the bandwagon.

To be fair, the Friend is a $99 device that’s not designed to help you be more productive — unlike the above gadgets already heading for the AI graveyard. This always-listening Bluetooth pendant is supposed to be a companion.

Friend Reveal Trailer – YouTube
Friend Reveal Trailer - YouTube

Watch On

“Productivity is over, no one cares,” Schiffmann told Wired. “No one is going to beat Apple or OpenAI or all these companies that are building Jarvis. The most important things in your life really are people.”

Let’s ignore for just a moment that the Friend is not a person. Actually, let’s not do that.

Yes, there are lots of lonely people out there, and something like the Friend could help. But there’s a huge difference between tech helping us find companionship and being a wholesale replacement for it and for humans.

The Friend is designed to be encouraging, texting you positive affirmations after you “killed it” in that meeting or consoling you after you get blown away by a friend in a game. And the founder promises that the company will not store audio recordings or transcripts.

But I am very concerned that we are going down the wrong path with AI gadgets like this one. Have we forgotten that our very reliance on and addiction to tech has been the catalyst of loneliness and depression for millions?

Don’t worry. As Schiffmann told The Verge, “I don’t think this should be the only person you should talk to.”