Google’s AI tool helps us add disasters and corpses to our photos

as it turns out, A rabbit wearing a top hat created by AI This was just a small part of the iceberg.

After Samsung, Google is the latest phone company to announce an AI photo editing tool this year. A somewhat annoying, mostly enjoyable sketch-to-image feature And Apple is much quieter Image Playground Coming this fall. The Pixel 9’s answer is a new tool called “Reimagine,” and after using it for a week with a few of my coworkers, I’m more convinced than ever that none of us are ready for what’s to come.

Reimagine is a logical extension of last year’s Magic Editor tool, which let you select and erase parts of a scene or change the sky to look like a sunset. That wasn’t surprising. But Reimagine doesn’t just take it a step further — it breaks the whole door down. You can select any non-human object or part of a scene and type a text prompt to draw something in that spot. The results are often Very Convincing and even funny. The lighting, shadows and perspective generally match the original photo. You can add fun things, sure, like wildflowers or rainbows or something. But that’s not the problem.

Some of my colleagues helped me test the limits of Reimagine with our Pixel 9 and 9 Pro review units, and we were able to get it to generate some pretty disturbing stuff. Some of these required some creative prompting to work around the obvious protections; if you chose your words carefully, you could get it to produce a reasonably believable body under a blood-stained sheet.

In our week of testing, we added car wrecks, smoking bombs in public places, sheets covering bloodied corpses, and drug paraphernalia to the images. It looks bad. As a reminder, this isn’t special software we’ve done something on our own to use – it’s all built into a phone my dad can go to Verizon and buy.

When we asked Google for comment on the issue, company spokesperson Alex Moriconi responded with the following statement:

Pixel Studio and Magic Editor are helpful tools meant to unlock your creativity with text-to-image generation and advanced photo editing on the Pixel 9 device. We design our generative AI tools to respect the intent of user prompts and that means they may create content that may be offensive if the user instructs them to do so. That being said, it’s nothing. We have a clear Policies And terms of Service We discuss what kind of content we allow and what we don’t, and build in security measures to prevent abuse. At times, certain signals may challenge the security measures of these tools and we are committed to continually enhancing and refining the security measures we implement.

Of course, our creative motivation to work around the filters is a clear violation of these policies. It’s also a violation of Safeway’s policies to mark your organic peaches as conventionally grown peaches at self-checkout, not that I know anyone who would do that. And most people with the worst intentions don’t even care about Google’s terms and conditions. What’s most troubling about all of this is the lack of robust tools to identify this kind of content on the web. Our ability to create problematic images is far outpacing our ability to identify them.

When you edit an image with Reimagine, there is no watermark or any other obvious way to know the image was created by AI – the metadata simply consists of a tag. That’s all fine, but standard metadata can easily be removed from an image simply by taking a screenshot. Moriconi tells us that Google uses this A more robust tagging system called SynthID Tags are used for images created by Pixel Studio because they are 100 percent synthetic. But images edited with Magic Editor do not get those tags.

Of course, manipulating photos is nothing new. People have been adding weird and misleading things to photos for a long time since the beginning of photographyBut the difference now is that adding these things to your photos realistically has never been easier. A year or two ago, adding a believable car crash to an image required time, expertise, an understanding of Photoshop layers, and access to expensive software. Those barriers have gone away; now all it takes is a bit of text, a few moments, and a new Pixel phone.

It’s also easier than ever to quickly disseminate deceptive images. The tools to reliably alter your photos are right there on the device you use to capture it and publish it for the world to see. We uploaded one of our “reimagined” images to an Instagram Story as a test (and promptly deleted it). Meta didn’t automatically tag it as AI-generated, and I’m sure no one would have been the wiser if someone saw it.

Who knows, maybe everyone will read and follow Google’s AI policies and use Reimagine to put wildflowers and rainbows in their photos. That would be awesome! But if they NoSo, it’s best to apply a little extra skepticism to photos you see online.