What is Project Astra? Google’s futuristic universal assistant explained

Almost everyone in tech is investing heavily in artificial intelligence right now, and Google is among those most committed to an AI future. Project Astra, unveiled at Google I/O 2024, is a big part of that – and it could end up being one of Google's most important AI tools.

Astra is being billed as “a universal AI agent that is helpful in everyday life”. It's essentially something like a blending of Google Assistant and Google Gemini, with added features and supercharged capabilities for a natural, conversational experience.

Here, we're going to explain everything you need to know about Project Astra – how it works, what it can do, when you can get it, and how it might shape the future. 

What is Project Astra?

In some ways, Project Astra isn't any different to the AI chatbots we've already got: you ask a question about what's in a picture, or about how to do something, or request some creative text to be generated, and Astra gets on with it.

What elevates this particular AI project is its multimodal functionality (the way text, images, video, and audio can all be combined), the speed that the bot works at, and how conversational it is. Google's aim, as we've already mentioned, is to create “a universal AI agent” that can do anything and understand everything.

Google IO 2024

Project Astra in action (Image credit: Google)

Think about the Hal 9000 bot in Kubrick's 2001: A Space Odyssey, or the Samantha assistant in the movie Her: talking to them is like talking to a human being, and there isn't much they can't do. (Both those AIs eventually got too big for their creators to control, but let's ignore that for the time being.)

Project Astra has been built to understand context and to take actions, to be able to work in real time, and to remember conversations from the past. From the demos we've seen so far, it works on phones and on smart glasses, and is powered by the Google Gemini AI models – so it may eventually be part of the Gemini app, rather than something that's separate and standalone.

When is Project Astra coming out?

Project Astra is in its early stages: this isn't something that's going to be available to the masses for a few months at least. That said, Google says that “some of these agent capabilities will come to Google products like the Gemini app later this year”, so it looks as though elements of Astra will appear gradually in Google's apps as we go through 2024.

When we were given some hands-on time with Project Astra at I/O 2024, these sessions were limited to four minutes each – so that gives you some idea of how far away this is from being something that anyone, anywhere can make use of. What's more, the Astra kit didn't look particularly portable, and the Google reps were careful to refer to it as a prototype.

Project Astra demonstration room at Google I/O showing large display and toys

We’ve already tried Project Astra (Image credit: Philip Berne / Future)

Taking all that together, we get the impression that some of the Project Astra tricks we've seen demoed might appear in the Google Gemini app sooner rather than later. At the same time, the full Astra experience – perhaps involving some dedicated hardware – is probably not going to be rolling out until 2025 at the earliest.

Now that Google has shared what Project Astra is and what it's capable of, it's likely that we're going to hear a whole lot more about it in the months ahead. Bear in mind that ChatGPT and Dall-E developer OpenAI is busy pushing out major upgrades of its own, and Google isn't going to want to be left behind.

What can I do with Project Astra?

One of Google's demos shows Astra running on a phone, using its camera input and talking naturally to a user: it's asked to flag up something in view that can play sounds, and correctly identifies a speaker. When an arrow is drawn on screen, Astra then recognizes and talks about the speaker component highlighted by the arrow.

In another demo, we see Astra correctly identifying world landmarks from drawings in a sketchbook. It's also able to remember the order of objects in a list, identify a neighborhood from an image, understand the purpose of sections of code that are shown to it, and solve math problems that are written out.

There's a lot of emphasis on recognizing objects, drawings, text, and more through a camera system – while at the same time understanding human speech and generating appropriate responses. This is the multimodal part of Project Astra in action, which makes it a step up from what we already have – with improvements in caching, recording, and processing key to the real time responsiveness.

In our hands-on time with Project Astra, we were able to get it to tell a story based on objects that we showed to the camera – and adapt the story as we went on. Further down the line, it's not difficult to imagine Astra applying these smarts as you explore a city on vacation, or solve a physics problem on a whiteboard, or provide detailed information about what's being shown in a sports game.

Which devices will include Project Astra?

In the demonstrations of Project Astra that Google has shown off so far, the AI is running on an unidentified smartphone and an unidentified pair of smart glasses – suggesting that we might not have heard the last of Google Glass yet.

Google has also hinted that Project Astra is going to be coming to devices with other form factors. We've already mentioned the Her movie, and it's well within the realms of possibility that we might eventually see the Astra bot built into wireless earbuds (assuming they have a strong enough Wi-Fi connection).

Google Pixel 8 Pro back in porcelain in front of animal print

Expect to see Project Astra turn up on Pixel phones, eventually (Image credit: Future / Philip Berne)

In the hands-on area that was set up at Google I/O 2024, Astra was powered through a large camera, and could only work with a specific set of objects as props. Clearly, any device that runs Astra's impressive features is going to need a lot of on-board processing power, or a very quick connection to the cloud, in order to keep up the real-time conversation that's core to the AI.

As time goes on and technology improves, though, these limitations should slowly begin to be overcome. The next time we hear something major about Project Astra could be around the time of the launch of the Google Pixel 9 in the last few months of 2024; Google will no doubt want to make this the most AI-capable smartphone yet.

You might also like

TechRadar – All the latest technology news

Read More

Google’s Project Astra could supercharge the Pixel 9 – and help Google Glass make a comeback

I didn't expect Google Glass to make a minor comeback at Google I/O 2024, but it did thanks to Project Astra. 

That's Google's name for a new prototype of AI agents, underpinned by the Gemini multimodal AI, that can make sense of video and speech inputs, and smartly react to what a person is effectively looking at and answer queries about it. 

Described as a “universal AI” that can be “truly helpful in everyday life”, Project Astra is designed to be proactive, teachable, and able to understand natural language. And in a video ,Google demonstrated this with a person using what looked like a Pixel 8 Pro with the Astra AI running on it. 

By pointing the phone's camera at room, the person was able to ask Astra to “tell me when you see something that makes sound”, to which the AI will flagged a speaker it can see within the camera's viewfinder. From there the person was able to ask what a certain part of the speaker was, with the AI replying that the part in question is a tweeter and handles high frequencies. 

But Astra does a lot more: it can identify code on a monitor and explain what it does, and it can work out where someone is in a city and provide a description of that area. Heck, when promoted, it can even make an alliterative sentence around a set of crayons in a fashion that's a tad Dr Zeus-like.

It can can even recall where the user has left a pair of glasses, as the AI remembers where it saw them last. It was able to do the latter as AI is designed to encode video frames of what it's seen, combine that video with speech inputs and put it all together in a timeline of events, caching that information so it can recall it later at speed. 

Then flipping over to a person wearing the Google Glass 'smart glasses', Astra could see that the person was looking at a diagram of a system on a whiteboard, and figure out where optimizations could be made when asked about them. 

Such capabilities suddenly make Glass seem genuinely useful, rather than the slightly creepy and arguably dud device it was a handful of years ago; maybe we'll see Google return to the smart glasses arena after this. 

Project Astra can do all of this thanks to using multimodal AI, which in simple terms is a mix of neural network models that can process data and inputs from multiple sources; think mixing information from cameras and microphones with knowledge the AI has already been trained on.

Google didn't say when Project Astra will make it into products, or even into the hands of developers, but Google's DeepMind CEO Demis Hassabis said that “some of these capabilities are coming to Google products, like the Gemini app, later this year.” I'd be very surprised if that doesn't mean the Google Pixel 9, which we're expecting to arrive later this year.

Now it's worth bearing in mind that Project Astra was shown off in a very slick video, and the reality of such onboard AI agents is they can suffer from latency. But it's a promising look at how Google will likely integrate actually useful AI tools into its future products.

You might also like

TechRadar – All the latest technology news

Read More

Adobe’s next big project is an AI that can upscale low-res video to 8x its original quality

A group of Adobe researchers recently published a paper on a new generative AI model called VideoGigaGAN and we believe it may launch on a future product. What it does is upscale low-quality videos by up to eight times their original resolution without sacrificing stability or important aspects of the source material. Several demo clips can be found on the project’s website showing off its abilities. It can turn a blurry 128×128 pixel resolution video of a waterfall into footage running at a resolution of 1,024×1,024 pixels.

Post by @luokai
View on Threads

What’s noteworthy about the AI is it doesn’t skimp out on the finer details. Skin texture, wrinkles, strands of hair, and more are visible on the faces of human subjects. The other demos also feature a similar level of quality. You can better make out a swan swimming in a pond and the blossom on a tree thanks to this tech. It may seem bizarre to be focusing so much on skin wrinkles or feathers. However, it is this level of detail that companies like Adobe must nail down if they aim to implement image-enhancing AI on a wide scale.

Improving AI

You probably have a couple of questions about the platform’s latest project like how does it work? Well, it’s complicated. 

The “GAN” in VideoGigaGAN stands for generative adversarial network, a type of AI capable of creating realistic images. Adobe’s version is specifically based on GigaGAN which specializes in upscaling generated content as well as real photos. The problem with this tech, as TheVerge points out, is that it can’t improve the quality of videos without having multiple problems crop up like weird artifacts. To solve this issue, Adobe researchers used a variety of techniques.

The research paper explains the whole process. You can read it yourself to get the full picture although it is dense material. Basically, they introduced a “flow-guided propagation module” to ensure consistency among a video’s frames, anti-aliasing to reduce artifacts, and a “high-frequency feature shuttle” to make up for sudden drops in detail. There is more to VideoGigaGAN than what we just described, but that’s the gist of it.

Potential inclusion

Will we see this on an upcoming Adobe product or roll out as a standalone app? Most likely – at least we think so. 

In the past year, the company has been focusing heavily on implementing artificial intelligence into its software from the launch of Firefly to Acrobat’s new assistant. A few months ago during Adobe MAX 2023, a video upscaler referred to as Project Res Up was previewed at the event and its performance resembles what we see in the VideoGigaGAN demos. An old movie from the 1940s goes from running at a 480 x 360 image resolution to a crisp 1,280 x 960. Blurry footage of an elephant in a river becomes crystal clear. The presenter even mentions how the software can upscale a clip to four times the original quality. 

Admittedly, this is conjecture, but it’s entirely possible VideoGigaGAN may be the engine behind Res-Up. Adobe’s future product could give people a way to upscale old family videos or low-quality footage into the movie we envision in our minds. Perhaps, the recent preview is a hint at an imminent release.

VideoGigaGAN is still deep in development so it’s unknown when or if it’ll come out. There are several obstacles in the way. The AI can’t properly process videos beyond 200 frames or render small objects, but we'll definitely be keeping an eye on it.

In the meantime, check out TechRadar's list of the best AI image upscalers for 2024.

You might also like

TechRadar – All the latest technology news

Read More

Hidden code points to Google reviving its AR glasses project

Google Glass was one of the first devices to build augmented reality tech into spectacles, but that device came and went without making much of a real impact. Now it looks like Google is once again interested in this particular product category.

Based on hidden code discovered by 9to5Google in the Google app for Android, there's a new reference to “iris”, as well as to launching Google Assistant with a tap on the right temple – which sounds like a pair of AI specs to us.

Now Project Iris was the codename of the specs that Google briefly showed off at Google I/O in 2022: their main job was to translate dialog spoken in a foreign language into text that would appear before your eyes.

It was only a short demo, and we haven't heard much about the glasses since – except in June of this year, when a report appeared that suggested the project had been shelved. Four months later, it could be back on the table once again.

AR and MR

Considering the brevity of the initial demo, and the lack of official information about these AR specs, we don't have much to go on in terms of what they can do – or indeed why they might have fallen down Google's list of priorities.

We were certainly impressed by what Google showed off last year, but it seems likely that these specs would go beyond instant translation and cover a variety of other features too. We might be talking about everything Google Assistant can do, and then some.

Even while Google has been relatively quiet on this topic – at least since Google Glass Enterprise was discontinued –  the market category continues to trundle on, as our Ray-Ban Meta Smart Glasses review shows.

We also know that Samsung and Google are currently working on a mixed reality headset to take on the likes of the Meta Quest 3 and the Apple Vision Pro. Based on this report, there could also be something more lightweight in the pipeline again.

You might also like

TechRadar – All the latest technology news

Read More

OpenAI’s CEO to get $100 million to scan everyone’s eyes for new crypto project

After going mainstream with ChatGPT, OpenAI's CEO is now embarking on a new challenge online.

Sam Altman actually co-founded Worldcoin in 2019 with the mission of “building the world’s largest identity and financial network.” Now, he seems to be close to securing $ 100 million of funds to kickstart the next step of the project: scanning everyone's eyeball to grant them free access to the new global cryptocurrency.

Some commentators have already expressed concerns about the ethical and privacy issues that could arise from it. So, will this end up being another privacy nightmare very much like his AI-powered bot?  

Iris-scanning ID verification system

According to the official website, Worldcoin is a new global cryptocurrency that aims to “create universal access to the global economy regardless of country or background, accelerating the transition to an economic future that welcomes and benefits every person on the planet.”

Quite an ambitious mission, but how do its founders plan to do that?

The key to the whole project seems to be what they refer to as the Orb. This is software that “uses iris biometrics to establish an individual’s unique personhood.” Once users have been verified, they can create their digital World ID and start receiving the crypto tokens. 

The company ensures that the World ID, which was released last week in Beta together with the World App, “can be used pseudonymously in a wide variety of everyday applications without revealing the user’s identity.”

This technology, the so-called proof of personhood protocol, is also believed to tackle some of the biggest issues raised by the quick development of AI-powered tools. It will discern between a real person and a bot, for example. Developers even believe that it could help provide a universal basic income to those affected by job cuts caused by AI.

Not everyone seems to be thrilled by the idea, though. Famous US whistleblower Edward Snowden raised concerns about the practice back in 2021. At the time, he pointed out how Worldcoin would de-facto build a global database of people's iris scans, keeping them in the form of hashes able to “match with future scans.”

See more

The company ensures that it will not store eye scans. It also says that the device is safe to use and will not hurt people's irises. 

Three people with knowledge of the deal have said to the Financial Times that Wordlcoin is now in “advanced talks to raise fresh cash as it prepares to launch in the next few weeks.”

The startup seems to be attracting new investors, too, alongside previous names like FTX founder Sam Bankman-Fried and internet entrepreneur Reid Hoffman.

Despite still operating on Beta, Worldcoin counts over 1.7 million sign-ups across the world so far, but the numbers are very likely to get higher soon.  

TechRadar – All the latest technology news

Read More

Opera launch Crypto Browser Project on iOS

After releasing its Crypto Browser Project on Windows, Mac and Android back in January, Opera has now launched a version of its cryptocurrency-focused browser on iOS.

For those unfamiliar, the company’s new browser is based around cryptocurrency and will provide users with an easier way to browse decentralized apps (dApps), games and metaverse platforms for a more seamless cross-platform experience. 

Share your thoughts on Cybersecurity and get a free copy of the Hacker’s Manual 2022. Help us find how businesses are preparing for the post-Covid world and the implications of these activities on their cybersecurity plans. Enter your email at the end of this survey to get the bookazine, worth $ 10.99/£10.99.

Opera’s Crypto Browser Project also features a news and data aggregator named “Crypto Corner”, a bespoke start page with live crypto information and updates, crypto asset prices and gas fees as well as crypto events, airdrops and even podcasts.

While Web3 is gaining momentum on the developer side with 34k developers joining the space in 2021 alone, the Web3-experience for users is still far from intuitive and is not optimized for iPhone users.

For these reasons, Opera has decided to add support for iOS to its Crypto Browser which is a dedicated Web3 browser with a built-in non-custodial crypto wallet.

Opera Crypto Browser on iOS 

In addition to giving users access to Web3 and dApps, Opera’s Crypto Browser on iOS also includes cryptocurrency mining protection that can block any ‘cryptojacking’ scripts that could compromise a user’s iPhone and decrease its performance.

The browser on iOS even features the ability to restore any Ethereum Virtual Machine (EVM) compatible  crypto wallet with the native Opera Wallet so that users can integrate their existing assets and balances into their Crypto Browser setup.

At the same time, Opera’s Crypto Browser is designed to address crypto’s growing pains with support for more efficient and environmentally-friendly PoS and Layer 2 chains which enable cheaper transactions and consume far less energy than Proof-of-Work (PoW) Blockchains. As such, the company has partnered with Polygon and will be integrating more PoS chains in its browser going forward.

EVP of mobile at Opera, Joregne Arnesen provided further insight in a press release on how the company’s Crypto Browser makes Web3 more accessible for users, saying:

“The interest in Web3 is continuing to grow. The Opera Crypto Browser Project was built to simplify the Web3 user experience that has often been bewildering for mainstream users. Opera believes Web3 has to be easy to use in order to reach its full potential and a mass adoption.”

iOS users interested in cryptocurrency or checking out Web3 and dApps for themselves can now download Opera’s Crypto Browser from Apple’s App Store.

TechRadar – All the latest technology news

Read More

Opera launch Crypto Browser Project on iOS

After releasing its Crypto Browser Project on Windows, Mac and Android back in January, Opera has now launched a version of its cryptocurrency-focused browser on iOS.

For those unfamiliar, the company’s new browser is based around cryptocurrency and will provide users with an easier way to browse decentralized apps (dApps), games and metaverse platforms for a more seamless cross-platform experience. 

Share your thoughts on Cybersecurity and get a free copy of the Hacker’s Manual 2022. Help us find how businesses are preparing for the post-Covid world and the implications of these activities on their cybersecurity plans. Enter your email at the end of this survey to get the bookazine, worth $ 10.99/£10.99.

Opera’s Crypto Browser Project also features a news and data aggregator named “Crypto Corner”, a bespoke start page with live crypto information and updates, crypto asset prices and gas fees as well as crypto events, airdrops and even podcasts.

While Web3 is gaining momentum on the developer side with 34k developers joining the space in 2021 alone, the Web3-experience for users is still far from intuitive and is not optimized for iPhone users.

For these reasons, Opera has decided to add support for iOS to its Crypto Browser which is a dedicated Web3 browser with a built-in non-custodial crypto wallet.

Opera Crypto Browser on iOS 

In addition to giving users access to Web3 and dApps, Opera’s Crypto Browser on iOS also includes cryptocurrency mining protection that can block any ‘cryptojacking’ scripts that could compromise a user’s iPhone and decrease its performance.

The browser on iOS even features the ability to restore any Ethereum Virtual Machine (EVM) compatible  crypto wallet with the native Opera Wallet so that users can integrate their existing assets and balances into their Crypto Browser setup.

At the same time, Opera’s Crypto Browser is designed to address crypto’s growing pains with support for more efficient and environmentally-friendly PoS and Layer 2 chains which enable cheaper transactions and consume far less energy than Proof-of-Work (PoW) Blockchains. As such, the company has partnered with Polygon and will be integrating more PoS chains in its browser going forward.

EVP of mobile at Opera, Joregne Arnesen provided further insight in a press release on how the company’s Crypto Browser makes Web3 more accessible for users, saying:

“The interest in Web3 is continuing to grow. The Opera Crypto Browser Project was built to simplify the Web3 user experience that has often been bewildering for mainstream users. Opera believes Web3 has to be easy to use in order to reach its full potential and a mass adoption.”

iOS users interested in cryptocurrency or checking out Web3 and dApps for themselves can now download Opera’s Crypto Browser from Apple’s App Store.

TechRadar – All the latest technology news

Read More