Google explains why AI Overviews couldn’t understand a joke and told users to eat one rock a day – and promises it’ll get better

If you’ve been keeping up with the latest developments in the area of generative AI, you may have seen that Google has stepped up the rollout of its ‘AI Overviews’ section in Google Search to all of the US.

At Google I/O 2024, held on May 14, Google confidently presented AI Overviews as the next big thing in Search that it expected to wow users, and when the feature finally began rolling out the following week it received a less than enthusiastic response. This was mainly due to AI Overviews returning peculiar and outright wrong information, and now, Google has responded by explaining what happened and why AI Overviews performed the way it did (according to Google). 

The feature was intended to bring more complex and better-verbalized answers to user queries, synthesizing a pool of relevant information and distilling it into a few convenient paragraphs. This summary would then be followed by the listed blue links with brief descriptions of the websites that we’re used to. 

Unfortunately for Google, screenshots of AI Overviews that provided strange, nonsensical, and downright wrong information started circulating on social media shortly after the rollout. Google has since pulled the feature, and published an explanatory post on its ‘Keyword’ blog to explain why AI Overviews was doing this, as mentioned – being quick to point out that many of these screenshots were faked. 

What AI Overviews were intended to be

Keynote speech at Google i/o 2024

(Image credit: Future)

In the blog post, Google first explains that the AI Overviews were designed to collect and present information that you would have to dig further via multiple searches to find out otherwise, and to prominently include links to credit where the information comes from, so you could easily follow up from the summary. 

According to Google, this isn’t just its large language models (LLMs) assembling convincing-sounding responses based on existing training data. AI Overviews is powered by its own custom language model that integrates Google’s core web ranking systems, which are used to carry out searches and integrate relevant and high-quality information into the summary. Accuracy is one of the cornerstones that Google prides itself on when it comes to search, the company notes, saying that it built AI Overviews to show information that’s sourced only from the web results it deems the best. 

This means that AI Overviews are generally supposed to hallucinate less than other LLM products, and if things happen to go wrong, it’s probably for a reason that Google also faces when it comes to search, giving the possible issues as “misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.”

What actually happened during the rollout

Windows 10 dual screen

(Image credit: Shutterstock / Dotstock)

Google goes on to state that AI Overviews was optimized for accuracy and tested extensively before its wider rollout, but despite these seemingly robust testing efforts, Google does admit that’s not the same as having millions of people trying out the feature with a flood of novel searches. It also points out that some people were trying to provoke its search engine into producing nonsensical AI Overviews by carrying out ridiculous searches. 

I find this part of Google’s explanation a bit odd, seeing as I’d imagine that when building such a feature as AI Overviews, the company would appreciate that folks are likely to try to break it, or send it off the rails somehow, and that it should therefore be designed to handle silly or nonsense searches in its stride.

At any rate, Google then goes on to call out fake screenshots of some of the nonsensical and humorous AI Overviews that made their way around the web, which is fair I think. It reminds us we shouldn’t believe everything we see online, of course, although the faked screenshots looked pretty good if you didn't scrutinize them too closely (and all this underscores the need to check AI-generated features, anyway).

Google does admit, though, that sometimes AI Overviews did produce some odd, inaccurate, or unhelpful responses. It elaborates by explaining that there are multiple reasons why these happened, and that this whole episode has highlighted specific areas where AI Overviews could be improved.

The tech company further observes that these questionable AI Overviews would appear on searches for queries that didn’t happen often. A Threads user, @crumbler, posted an AI Overviews screenshot that went viral after they asked Google: “how many rocks should i eat?” This returned an AI Overview that recommended eating at least one small rock per day. Google’s explanation is that before this screenshot circulated online, this question had rarely been asked in search (which is certainly believable enough). 

A screenshot of an AI Overview recommending that humans should eat one small rock a day

(Image credit: Google/@crumbler on Threads)

Google continues to explain that there isn’t a lot of quality source material to answer that question seriously, either, calling instances when this happens a “data void” or an “information gap.” Additionally, in the case of the query above, some of the only content that was available was satirical by nature, and was linked in earnest as one of the only websites that addressed the query. 

Other nonsensical and silly AI Overviews pulled details from sarcastic or humorous content sources, and the likes of troll posts from discussion forums.

Google's next steps and the future of AI Overviews

When explaining what it’s doing to fix and improve AI Overviews, or any part of its Search results, Google notes that it doesn’t go through Search results pages one by one. Instead, the company tries to implement updates that affect whole sets of queries, including possible future queries. Google claims that it’s been able to identify patterns when analyzing the instances where AI Overviews got things wrong, and that it’s put in a whole set of new measures to continue to improve the feature.

You can check out the full list in Google’s post, but better detection capabilities for nonsensical queries trying to provoke a weird AI Overview are being implemented, and the search giant is looking to limit the inclusion of satirical or humorous content.

Along with the new measures to improve AI Overviews, Google states that it’s been monitoring user feedback and external reports, and that it’s taken action on a small number of summaries that violate Google’s content policies. This happens pretty rarely – in less than one in seven million unique queries, according to Google – and it’s being addressed.

The final reason Google gives for why AI Overviews performed this way is just the sheer scale of the billions of queries that are performed in Search every day. I can’t say I fault Google for that, and I would hope it ramps up the testing it does on AI Overviews even as the feature continues to be developed.

As for AI Overviews not understanding sarcasm, this sounds like a cop-out at first, but sarcasm and humor in general is a nuance of human communication that I can imagine is hard to account for. Comedy is a whole art form in itself, and this is going to be a very thorny and difficult area to navigate. So, I can understand that this is a major undertaking, but if Google wants to maintain a reputation for accuracy while pushing out this new feature – it’s something that’ll need to be dealt with.

We’ll just have to see how Google’s AI Overviews perform when they are reintroduced – and you can bet there’ll be lots of people watching keenly (and firing up yet more ridiculous searches in an effort to get that viral screenshot).

YOU MIGHT ALSO LIKE…

TechRadar – All the latest technology news

Read More

Windows 11 23H2 update is real, we’re told – but it could disappoint

Windows 11’s big update for later this year, known as 23H2, is reportedly real and inbound for PCs – though how much impact it’ll make is another question (that we’ll come back to shortly).

Windows Latest has been doing some digging and tells us that it has spotted references to ‘23H2’ in a couple of documents, as well as Windows preview builds.

Furthermore, those references have also been seen in Windows 11 itself, in Settings, and Winver, a command that displays the current version of Windows (and labels a test build as 23H2, presumably).

Windows Latest underlines something else in its report, namely that the 23H2 update will be triggered via an enablement package, something we’ve already heard from the rumor mill in recent times.

This means that in theory – we need to take all of this with a fair old sprinkling of salt – Microsoft will preload the 23H2 update before it comes around to release. So Windows 11 users will only need to download a small update – the trigger, or enablement package – to receive the preloaded features.

This also suggests that the update will be a more minor affair, as generally this is the approach Microsoft takes with upgrades that are, shall we say, a little less ambitious in their scope – they are effectively quick and easy updates (relatively speaking).


Analysis: Making way for Windows 12?

This also marries with what we’ve seen thus far in preview builds, namely that there aren’t any huge Windows 11 features appearing in the pipeline thus far. Don’t get us wrong, there’s definitely some solid stuff present in the preview – some key interface changes, and the revamp of File Explorer (complete with a new photo gallery feature) – but the meaty changes appear to be somewhat thin on the ground.

Now, that could change, as there’s still some time before the release of 23H2 – perhaps as much as five months even. But the reality is the upgrade will probably arrive before November, and given the time taken to test larger bits of functionality, there isn’t much breathing room left to get that kind of testing in.

It also makes sense that Microsoft hasn’t officially said anything about 23H2 yet, simply because there’s not all that much to shout about, perhaps.

In reality, as Windows Latest points out, bigger moves are at this point probably being reserved for Windows 12. After all, Microsoft needs to make a splash with a new incarnation of Windows – something in all honesty it failed to do with Windows 11, which initially felt like more of a reskin of Windows 10 than anything else. (Albeit with some good changes on the design front, no doubt – but also frustrations).

So, Microsoft will likely be saving much of the juicier stuff for Windows 12 – or whatever next-gen Windows is called – and that’s quite possibly going to turn up later in 2024, so Windows Latest suggests. And that’s a believable prospect, given that Windows 10 will be pushed out of support in 2025, plus it also aligns with other chatter from the rumor mill, too.

If true, this means that next year’s annual update for Windows 11 (24H2) will likely also be a more minor affair – given that Microsoft will have shifted its attention to Windows 12. Then it’ll only be a matter of time before Microsoft ceases any meaningful feature updates for Windows 11, which is what just happened with Windows 10.

TechRadar – All the latest technology news

Read More

The inside story of the browser wars, told by a veteran

The story of Brendan Eich is in many ways the story of the evolution of the internet and the technologies we use to access it. It is also a story of battles won, lost and soon to be played out.

Eich is best known as the creator of programming language JavaScript, which he developed over a sleepless period of ten days in 1995. At the time, he was working for Netscape, whose web browser dominated the market before Internet Explorer spoiled the party.

Recognizing that Netscape had lost its way, Eich spun out another project he had been working on, leading to the formation of the Mozilla Foundation. The organization went on to pioneer the concept of browser extensions with Firefox, which quickly became a household name, before it was crushed under the weight of Google Chrome.

Since departing Mozilla, Eich has focused his attention on a new company called Brave Software, which he believes will help usher in the next landmark period in the history of the internet.

Founded in 2015, Brave is the maker of a privacy-centric web browser by the same name, which blocks both ads and tracking cookies. It is also the proving ground for a novel opt-in advertising model, whereby users are paid for their attention.

With these building blocks, Eich is aiming to bring to fruition an internet characterized not by monopoly and unfettered surveillance, but rather decentralization, disintermediation and individual privacy.

Brendan Eich

Brendan Eich, creator of JavaScript, Mozilla and Brave Software. (Image credit: Brendan Eich)

JavaScript is born

Today, JavaScript is deployed across practically all websites, as part of a famous trifecta that also includes HTML and CSS. It allows web developers to code in all the rich features and dynamic content users interact with on the web, and can also be deployed server-side and for various other purposes.

Given its ubiquity and the extent of its influence over the web, it’s easy to forget JavaScript is the creation of a single person, who put it together in under a fortnight. “The web was evolving almost in a biological fashion [in the 1990s],” Eich told TechRadar Pro. “My big contribution to the process was JavaScript.”

When Eich joined Netscape in 1995, he says there was a “feeling of doom” hanging over the company, because Microsoft was breathing down its neck. The infamous Microsoft strategy was to “embrace, extend and extinguish”; it would embrace a new type of software, extend it with proprietary facilities that only functioned inside Windows, and harness these new capabilities to extinguish the competition.

The previous year, Netscape had spurned a low-ball acquisition offer from Microsoft, so the company knew it was next in line for the typical treatment. The plan to shield itself relied heavily on the integration of Java into the browser and the opportunities made possible by JavaScript, which was designed as a dynamic companion language that non-expert developers could use to add interactivity to their websites.

Netscape Navigator 2.0

Netscape Navigator 2.0, released in 1995. (Image credit: Wikipedia)

“We knew Microsoft was coming after us and we wanted to get Netscape Navigator 2.0 out the door, with Java as the big programming language and JavaScript as the sidekick that let the average scripter glue things together,” Eich explained.

To some extent, the endeavor was a success. Microsoft, which had previously stated its intentions to make VBScript the go-to language for building web applications, was ultimately forced to build support for JavaScript into Internet Explorer. This meant the company did not have control over the favorite scripting language of the web.

Until 1996, Eich was the sole developer working full-time on the JavaScript engine, which was plagued by technical debt (in other words, messy code) that resulted from the speed with which it was first composed. It was also apparent that a specification needed to be created, to guarantee web pages would function as intended across the multiple browsers that now supported JavaScript.

After rewriting the core, Eich helped build out a vendor-neutral specification in collaboration with Microsoft and other players, which was then left under the stewardship of a standards body called Ecma International.

History shows that Netscape was ultimately unsuccessful in fending off the advances of Microsoft, which eventually captured 95% of the browser market with Internet Explorer. Bill Gates had not only made his browser free, but also packaged it with Windows machines, which left Netscape no room to maneuver.

“A bunch of us started to see the writing was on the wall,” said Eich. “The feeling that we were doomed had been fulfilled, and the question became: what next?”

Bill Gates

Under Bill Gates, Microsoft went in pursuit of Netscape with Internet Explorer.  (Image credit: Shutterstock / Paolo Bona)

Mozilla breaks free

Knowing itself beaten, Netscape decided to open source its browser code. Eich says the idea was to create a community like the one that surrounded Linux, which would engender new browser innovation.

The company tasked Eich and a group of other developers with setting in motion the project, which came to be called Mozilla. The team worked “inside a fishbowl” at Netscape, at a remove from the rest of the company, Eich explained.

In the coming years, however, the relationship between the Mozilla team and Netscape executives soured. The two groups traded blows over product design, release timelines, the toxic working culture and other topics. 

“It was a difficult time,” said Eich. “I stopped using my Netscape email as much as I could and used an ISP email instead. I acted as if I were completely outside the firewall.”

“I also set up a proxy for the developers we were trying to bring in, who didn’t have the advantage of being employees. Meanwhile, Netscape kept regressing to the mean when it came to the strength of its programmers.”

After a series of layoffs in 2003, Eich and the Mozilla team broke out of their fishbowl to form a standalone non-profit called the Mozilla Foundation, which was tasked with carrying the project forward.

Meanwhile, wounded by the famous antitrust ruling over the bundling of Windows and Internet Explorer, Microsoft had grown lazy with its web browser. The strength of the company’s grip on the market meant it no longer had reason to innovate, creating a window of opportunity for a plucky newcomer.

Despite the politicking inside Netscape, the Mozilla team had managed to build a browser capable of rising to this challenge. The first ever to support extensibility, Firefox (as it came to be known after a series of name changes) rose quickly to prominence and reignited competition in the browser space, says Eich.

Firefox

Firefox was the first ever web browser to support extensibility, paving the way for Google Chrome and other modern browsers. (Image credit: Shutterstock / tanuha2001)

Mozilla reaped the rewards of its tenacity in the years that followed, attracting many millions of users to Firefox and netting a lucrative deal that saw Google become the browser’s default search engine. Ultimately, though, this level of momentum proved unsustainable and Mozilla was caught out by developments elsewhere in the technology world.

The rise of the smartphone in the late aughts transformed the way people engaged with the internet, and Mozilla failed to spot the danger. While the iPhone shipped with Safari pre-installed and Android devices came with Chrome, Firefox was left out in the cold. Once again, Eich found himself on the wrong end of the platform effect.

Locked out of the mobile market and unable to compete with Google’s marketing spend, Mozilla could do little to stop the numbers tumbling. Once responsible for roughly 30% of web activity, Firefox now holds just a 4% market share, the latest figures suggest.

Although there were efforts to limit the damage with Firefox OS and other projects, Mozilla never managed to regain a proper foothold and has now pivoted towards other products, including a new VPN.

Eich eventually left the organization under a cloud of controversy in 2015. After less than two weeks as CEO, it emerged he had made a donation in support of a ban on same-sex marriage, and the backlash was fierce. We were told an NDA signed between Eich and Mozilla precluded discussions of this chapter of his life.

Brave new world

While no rational person would dispute the importance of Eich’s contribution to the web, it is also true that he has been on the losing side of both so-called browser wars; first at Netscape, then at Mozilla.

This is a pattern he is hoping to break with Brave, which is pitched as the antidote to the threats posed by Google and its stranglehold on the browser and search markets.

Brave’s browser blocks all advertising by default and has a no-tolerance policy towards third-party cookies, which track users across the web to help inform highly-targeted advertising efforts.

Although there are now plenty of browsers that block invasive tracking techniques, Brave stands apart for its ambitions to rewire the economics that underpin the digital advertising industry.

“The plan was that Brave would be faster, easier on the battery and more private,” said Eich. “And with the help of blockchain technology, we also wanted to replumb the economic engine [of the web].”

The company’s unusual model is built around its Basic Attention Token (BAT), which was launched in 2017 via an initial coin offering (ICO). When an advertiser signs up for a campaign, Brave uses 70% of the fee to purchase BAT from the open market, and these tokens are then distributed to users who have opted-in to the ads program.

Brave browser

The Brave Software logo. (Image credit: Brave)

Once in the user’s possession, BAT tokens can either be donated to favorite content creators, used for microtransactions with Brave partners, or flipped into regular currency via an exchange.

Which specific ads are served to which specific users is determined by browsing data that is stored on-device and run through a machine learning (ML) model. Apparently, this approach actually yields markedly higher clickthrough rates than the 2% industry average.

Unlike the Google system, which is based on tracking users indiscriminately across the web, the Brave model is opt-in only, compensates the user for their participation and does not involve the transport of browsing data to the cloud.

The signs suggest this strategy is paying off for Brave, which has benefited from increasing awareness of the importance of privacy among consumers. The latest figures show the browser now attracts 50 million monthly users, which is double the figure from a year ago, and quadruple the year before that.

While these numbers are modest in the context of the total volume of web users, Eich predicts that enthusiasm for the service among developers, crypto fans and privacy evangelists is bound to spill over.

“There are privacy nihilists out there that we’ll never convince, but people have generally become more conscious about their privacy as a result of security breaches and events like the Cambridge Analytica scandal,” he said.

“But to breach the chasm to mass market, reaching people who are aware of these kinds of problems matters, because they convert their friends and family, and this creates rolling thunder.”

Brave

Brave’s browser on mobile. (Image credit: Brave)

Empire building

Over the last couple of years, Brave has quietly set about expanding its empire with new privacy-centric products. Orbiting its browser, there is now a Brave VPN, firewall, encrypted video conferencing service, crypto wallet, news aggregator and search engine.

Asked which areas Brave will look to expand into next, Eich declined to provide any concrete information, but did concede the company is “looking at the larger space”. We wouldn’t be surprised to see an encrypted email service from Brave in the near future, for example.

All of these technologies will be foundational to Web 3.0, a new generation of the internet defined by decentralization, disintermediation and greater user privacy. Among those attempting to bring Web 3.0 into being, many believe blockchain and cryptocurrency will play a fundamental role in the transition.

Naturally, Brave has attracted a large number of cryptocurrency enthusiasts, whose ambitions with regards to economic freedom align closely with the company’s attempts to create a more equitable and private web. 

The long-term success of the project, however, will be determined by how effectively Brave is able to sell itself to a wider and less technical audience. It has a lot of ground to make up before it can hope to challenge the dominance of Google and other incumbents.

However, as Eich’s story demonstrates, the internet is littered with the corpses of fallen giants. The cry for greater privacy on the web is growing louder and louder, and Brave has put itself in a position to ride the zeitgeist.

TechRadar – All the latest technology news

Read More