Travel and Shop

🔷 Subscribe to get breakdowns of the most important developments in AI in your inbox every morning.

Here’s today at a glance:

• Unbundling Google

• Cleanup in Aisle 2

• Things happen

• AI artwork of the day

🧶 Unbundling Google

When you book a trip:

• 25% of your hotel room price gets paid to an OTA

• Online Travel Agents like Expedia take that revenue and use it to advertise on Google

• 20% of Google’s revenue comes from the travel sector

This relationship is so significant that small changes on the Search Engine Results Page have a huge impact, as evinced by Expedia owner Barry Diller’s rant at such a change.

AI that plans a trip is one of the holy grails of product. And now the team at Ohio State University has finally created a benchmark to specifically measure how close we are to creating a Google killer, an AI travel planner.

The benchmark consists of a dataset of training set of 5 queries, with human-annotated solutions, and a test set of 1,000 queries.

The Agent has presumed access to 6 search tools:

• City

• Flights

• Distance

• Attraction

• Accommodation

• Restaurant

And the dataset includes 4 million entries for possible results from the 6 tools. In effect, this is the first closed-world dataset for travel planning which should be large enough to create a planner.

The findings using this dataset and the benchmark were.. interesting:

• LLMs are terrible at planning right now - GPT-4 had a success rate of 0.6%, while all other models totally failed

• Existing planning strategies fail at this level of complexity.

Google certainly doesn’t seem to need to worry yet… However, in AI, the creation of a good benchmark and dataset is often the first step towards finding the path to a solution… And I‘m sure we’ll be talking about state-of-the-art performance on this benchmark within a year.

Share this story

🧹 Cleanup in Aisle 2

What happens when a machine gets much better than a human at doing something? When it’s an intellectual task?

This paper measures the performance of stock pickers before and after the introduction of AI car parking lot counting.

Car counting is important as it directly correlates to foot traffic, which is a key financial input into retailers such as shopping malls. Knowing this number allows more accurate predictions of quarterly sales, and hence stock prices.

The net result of the introduction of AI car counting was:

• Poor stock pickers performed the same before and after

• Good stock pickers lost their edge and performed worse

In effect, AI leveled the playing field between the good and the bad, by eliminating the advantage enjoyed by the good (intuition, etc). This is likely a precursor to what will happen in many zero-sum contests.

Share this story

🗞️ Things Happen

• The Apple Vision Pro was launched. I have nothing to say except the memes are excellent. Hat/tip @tunguz

• Kleiner Perkins says more than 80% of pitches now involve AI. Well, I certainly hope everyone has some matrix multiplication going on.. but really the question should be what data do you have access to that no one else does, and what can you do with it? The matrix multiplication is the least of it

• The EU’s AI Act nears passage. While we are interested to see what actually makes it into the final bill, we are really in uncharted territory here. Unless Year Zero measures are taken, in 3 or more years a GPT-5 equivalent model will be on every cell phone in the world. Little chance of stopping it without getting rid of most of tech altogeher.

🖼️ AI Artwork Of The Day

That’s it for today! Become a subscriber for daily breakdowns of what’s happening in the AI world: