Apple working on a new AI-powered editing tool and you can try out the demo now

Apple says it plans on introducing generative AI features to iPhones later this year. It’s unknown what they are, however, a recently published research paper indicates one of them may be a new type of editing software that can alter images via text prompts.

It’s called MGIE, or MLLM-Guided (multimodal large language model) Image Editing. The tech is the result of a collaboration between Apple and researchers from the University of California, Santa Barbara. The paper states MGIE is capable of “Photoshop-style [modifications]” ranging from simple tweaks like cropping to more complex edits such as removing objects from a picture. This is made possible by the MLLM (multimodal large language model), a type of AI capable of processing both “ text and images” at the same time.

VentureBeat in their report explains MLLMs show “remarkable capabilities in cross-model understanding”, although they have not been widely implemented in image editing software despite their supposed efficacy.

Public demonstration

The way MGIE works is pretty straightforward. You upload an image to the AI engine and give it clear, concise instructions on the changes you want it to make. VentureBeat says people will need to “provide explicit guidance”. As an example, you can upload a picture of a bright, sunny day and tell MGIE to “make the sky more blue.” It’ll proceed to saturate the color of the sky a bit, but it may not be as vivid as you would like. You’ll have to guide it further to get the results you want. 

MGIE is currently available on GitHub as an open-source project. The researchers are offering “code, data, [pre-trained models]”, as well as a notebook teaching people how to use the AI for editing tasks. There’s also a web demo available to the public on the collaborative tech platform Hugging Face. With access to this demo, we decided to take Apple’s AI out for a spin.

Image 1 of 3

Cat picture new background on MGIE

(Image credit: Cédric VT/Unsplash/Apple)
Image 2 of 3

Cat picture lightning background on MGIE

(Image credit: Cédric VT/Unsplash/Apple)
Image 3 of 3

Cat picture on MGIE

(Image credit: Cédric VT/Unsplash/Apple)

In our test, we uploaded a picture of a cat that we got from Unsplash and then proceeded to instruct MGIE to make several changes. And in our experience, it did okay. In one instance, we told it to change the background from blue to red. However, MGIE instead made the background a darker shade of blue with static-like texturing. On another, we prompted the engine to add a purple background with lightning strikes and it created something much more dynamic.

Inclusion in future iPhones

At the time of this writing, you may experience long queue times while attempting to generate content. If it doesn’t work, the Hugging Face page has a link to the same AI hosted over on Gradio which is the one we used. There doesn't appear to be any difference between the two.

Now the question is: will this technology come out to a future iPhone or iOS 18? Maybe. As alluded to at the beginning, company CEO Tim Cook told investors AI tools are coming to its devices later on in the year but didn’t give any specifics. Personally, we can see MGIE morph into the iPhone version of Google’s Magic Editor; a feature that can completely alter the contents of a picture. If you read the research paper on arXiv, that certainly seems to be the path Apple is taking with its AI.

MGIE is still a work in progress. Outputs are not perfect. One of the sample images shows the kitten turn into a monstrosity. But we do expect all the bugs to be worked out down the line. If you prefer a more hands-on approach, check out TechRadar's guide on the best photo editors for 2024.

You might also like

TechRadar – All the latest technology news

Read More

Apple Vision Pro blasts out of mixed reality and into real stores – here’s how to sign up for a demo

It felt almost odd to be standing in the rain outside of Apple's glassy Fifth Avenue flagship store on Groundhog Day and not be wearing my Apple Vision Pro. I'd barely removed the mixed reality headset in my first two days of testing the Vision Pro and the real world felt a bit flat. Until, that is, Apple CEO Tim Cook opened the swinging glass doors and opened the proverbial floodgates to new and soon-to-be-new Apple Vision Pro owners.

It is something of a tradition for Cook to usher in every new product at Apple's Central Park-adjacent location but this moment was different, maybe bigger. It has been almost a decade since Apple launched a new product category (see the Apple Watch) and so expectations were high.

The crowd gathered outside was not what I'd call iPhone size – the miserable weather might have been a factor there – but there were dozens of people somewhat evenly split between media and customers.

A cluster of blue-shirted Apple employees poured out of the store, which featured the giant white outline of a Vision Pro on the storefront, and started clapping and cheering (I'd heard them practicing cheers and getting amped up from inside the store), doing their best to substitute any enthusiasm the crowd might've been lacking. This, too, is tradition and I find it almost endearing but also just a tiny bit cringe-worthy. It's just a gadget – a very expensive one – after all.

At precisely 8AM ET, Cook appeared behind the glass doors (someone had previously double-checked and triple-checked that the doors were not locked so Cook didn't have to bend down and release a latch). He swung open the door and gave a big wave.

Soon customers who had preordered the $ 3,499 (to start) spatial reality computer were filing into the store (many pausing to take a selfie with Cook), while I waited outside, getting drenched and wondering if the Vision Pro is waterproof (it's not).

Image 1 of 5

Apple Vision Pro store launch

Tim Cook acknowledges the crowd. (Image credit: Future / Lance Ulanoff)
Image 2 of 5

Apple Vision Pro store launch

Cook pops out and waves. (Image credit: Future / Lance Ulanoff)
Image 3 of 5

Apple Vision Pro store launch

Tim Cook was in his element. (Image credit: Future / Lance Ulanoff)
Image 4 of 5

Apple Vision Pro store launch

Waiting for the launch. (Image credit: Future / Lance Ulanoff)
Image 5 of 5

Apple Vision Pro store launch

First guy on line. (Image credit: Future / Lance Ulanoff)

Inside the store, which sits below ground level, the floor was packed. Vision Pros were lined up on stands similar to what I'd seen at launch. Below each one was an iPad, describing the experience you were about to have. Some people were seated on wooden benches near the back of the store, wearing Vision Pro headsets and gesturing to control the interfaces.

Oddly, though, not a lot of people were trying Vision Pros, but that was probably because Tim Cook was still in the room.

The scrum around him was dense, so much so that I noticed some nervous-looking Apple employees trying to gently clear a path and give the Apple leader some air. Cook, ever the gracious southern gentleman, smiled for countless photos with fans. He even signed a few things.

I stepped forward and Cook's eyes caught mine. He smiled broadly and said hello. We shook hands and I congratulated him on a successful launch. Then I gave him my brief assessment of the product: “It's incredible.” He brightened even further, “I know!” he shouted back over the din.

Apple Vision Pro store launch

(Image credit: Future / Lance Ulanoff)
Image 1 of 4

Apple Vision Pro store launch

They put some of the Vision Pros on stands. (Image credit: Future / Lance Ulanoff)
Image 2 of 4

Apple Vision Pro store launch

You cna see people in the back wearing them. (Image credit: Future / Lance Ulanoff)
Image 3 of 4

Apple Vision Pro store launch

Tim Cook is surrounded. (Image credit: Future / Lance Ulanoff)
Image 4 of 4

Apple Vision Pro store launch

Hi, Mr. Cook. (Image credit: Future / Lance Ulanoff)

There wasn't much more to say, really, and I left him to get sucked back into the crowd while I took another look at the Vision Pro sales setup. In the meantime, customers were leaving with large Vision Pro boxes they'd pre-ordered. Thousands of the mixed reality headsets are in stores and arriving at people's homes (in the US only). This will be their first experience with Vision Pro.

The good news is, as I told someone else today, there is no learning curve. The setup is full of hand-holding and using the system generally only requires your gaze and very simple gestures.

There will be comments about the weight and getting the right, comfortable fit on your head, and some may be frustrated with the battery pack and that they have to keep Vision Pro plugged in if they want to use it for more than two hours at a time.

Still, the excitement I saw at the store this morning and in Tim Cook's eyes may be warranted. This is not your father's mixed reality.

Booking your demo

For the next few days, all demos will be first-come-first-serve in the stores. However, if you can wait until after Feb 5, you can book your in-store demo by visiting the Apple Store site, navigating to the Vision Pro section, and selecting “Book a demo.” Apple will guide you to sign in with your Apple ID. You must also be at least 13 years old to go through the experience.

Demos take about 30 minutes. An Apple specialist will guide you through the setup processes, which is fairly straightforward.

You'll choose a store near you, a date, and an available time. If you wear glasses, Apple should be able to take your lenses and do a temporary measurement to give you the right lenses for the demonstration (you'll be buying your own Zeiss inserts if you buy a headset.).

After that, you can go home and figure out how to save up $ 3,500.

@techradar

♬ Epic Inspiration – DM Production

You might also like

TechRadar – All the latest technology news

Read More

That mind-blowing Gemini AI demo was staged, Google admits

Earlier this week, Google unveiled its new Gemini artificial intelligence (AI) model, and it’s safe to say the tool absolutely wowed the tech world. That was in part due to an impressive “hands on” video demo (below) that Google shared, yet it’s now emerged that all was not as it seemed.

According to Bloomberg, Google modified interactions with Gemini in numerous ways in order to create the demonstration. It raises questions over the chatbot’s abilities, as well as how much Google has been able to catch up with rival OpenAI and its own ChatGPT product.

For instance, the video’s YouTube description explains that “for the purposes of this demo, latency has been reduced and Gemini outputs have been shortened for brevity.” In other words, it probably takes a lot longer for Gemini to respond to queries than the demo suggested.

And even those queries have come under scrutiny. It turns out that the demo “wasn’t carried out in real time or in voice,” says the Bloomberg report. Instead, the real demo was constructed from “still image frames from the footage, and prompting via text.” 

This means that Gemini wasn’t responding to real-world prompts quickly in real time – it was simply identifying what was being shown in still images. To portray it as a smooth, flowing conversation (as Google did) feels a little misleading.

A long way to go

That’s not all. Google claimed that Gemini could outdo the rival GPT-4 model in almost every test the two tools took. Yet looking at the numbers, Gemini is only ahead by a few percentage points in many benchmarks – despite GPT-4 being out for almost a year. That suggests Gemini has only just caught up to OpenAI’s product, and things might look very different next year or when GPT-5 ultimately comes out.

It doesn’t take much to find other signs of discontent with Gemini Pro, which is the version currently powering Google Bard. Users on X (formerly Twitter) have shown that it is prone to many of the familiar “hallucinations” that other chatbots have experienced. For instance, one user asked Gemini to tell them a six-letter word in French. Instead, Gemini confidently produced a five-letter word, somewhat confirming the rumors from before Gemini launched that Google’s AI struggled with non-English languages.

Other users have expressed frustration with Gemini’s inability to create accurate code and its reluctance to summarise sensitive news topics. Even simple tasks – such as naming the most recent Oscar winners – resulted in flat-out wrong responses.

This all suggests that, for now, Gemini may fall short of the lofty expectations created by Google’s slick demo, and is a timely reminder not to trust everything you see in a demo video. It also implies that Google still has a long way to go to catch up with OpenAI, despite the enormous resources at the company’s disposal.

You might also like

TechRadar – All the latest technology news

Read More