Hello, everyone, and welcome to our weekly newsletter on AI from TechCrunch. To have this digest sent directly to your email each Wednesday, make sure to subscribe here.
Generative AI is a hot topic, and it seems to be moving towards being more widely available and affordable.
In a notable move during early August, industry giants Google and OpenAI announced significant price reductions for their most economical text generation models. Google slashed the processing cost for its Gemini 1.5 Flash model by 78% and the text generation fee by 71%. Concurrently, OpenAI cut the input and output costs for GPT-4o by 50% and 33%, respectively.
A recent analysis suggests that the expense of running these AI models, or inference costs, is decreasing annually by 86%. The question is, what’s spurring this drop?
For starters, the leading AI models don’t differ significantly in what they can do.
Andy Thurai from Constellation Research mentioned to me that ongoing price reductions across all AI models are likely if they fail to distinguish themselves. If consumer uptake is low or competition tightens, providers will have to adopt aggressive pricing strategies to retain their clientele.
John Lovelock from Gartner points out that both commodification and rivalry are driving prices down. Historically, prices were based on a cost-plus strategy — meaning, the aim was to recover the extensive costs dedicated to training these models (e.g., GPT-4’s training costed around $78.4 million) as well as to cover server operational costs (ChatGPT’s operational costs reached approximately $700,000 every day). However, with Data centers expanding their capability and efficiency, it’s now possible to offer these services at discounted rates.
In a bid to further reduce costs, companies like Google, Anthropic, and OpenAI are utilizing techniques such as prompt caching and batching. Prompt caching allows for the reuse of certain prompt information in future API calls, while batching handles a collection of non-urgent model inference requests at a lower cost.
The advent of open model releases such as Meta’s Llama 3 is also playing a role in driving down vendor prices. Although the top-tier models are not exactly cheap to operate, they present a competitive cost alternative when deployed on a company’s private servers.
However, it remains to be seen if these price reductions are sustainable in the long term.
The economic reality for generative AI companies is stark; OpenAI is reportedly on course to lose $5 billion this year, while its competitor Anthropic might find itself nearly $2.7 billion in debt by 2025.
Lovelock speculates that the high capital and operational costs may force these vendors to rethink their pricing models.
“What implications will the cost-plus pricing model have for consumers, considering the upcoming generation of models will incur costs in the hundreds of millions?” he questioned.
Time will certainly tell.
News Highlights
Musk Throws Support Behind SB 1047: Elon Musk, the CEO of X, Tesla, and SpaceX, has shown support for California’s SB 1047, legislation that mandates large AI model creators to implement and record measures preventing serious harm.
AI Overviews’ Hindi Hiccups: Ivan reports on the shortcomings of Google’s AI Overviews in Hindi, pointing out errors like suggesting inappropriate summer snacks.
AI Watermarking Gets a Nod: A California bill that mandates the labeling of AI-created content has garnered support from OpenAI, Adobe, and Microsoft, Max tells us. The bill is awaiting its final decision this August.
Inflection Limitations: AI venture Inflection will soon restrict free access to its chatbot Pi as the company pivots towards enterprise solutions, following a significant portion of its team being acquired by Microsoft.
Stephen Wolfram’s AI Insights: In a conversation with Ron Miller, Stephen Wolfram, creator of Wolfram Alpha, discusses how AI is ushering philosophy into a new era of prominence and inquiry.
Waymo’s Youth Transport Initiative: Waymo, a subsidiary of Alphabet, is reportedly exploring a subscription service that would allow teenagers to use its vehicles unaccompanied, providing parents with real-time updates on their trips.
Discontent at DeepMind: A section of employees at DeepMind, Google’s AI research division, has expressed discontent with Google’s defense contracts, internally sharing a letter to voice their concerns.
AI Startups Spark SVP Interest: Venture capitalists are increasingly investing in secondary shares of cutting-edge AI startups through special purpose vehicles (SVPs), aiming to stake a claim in the sector’s most promising companies, according to Rebecca.
Featured Research Paper
We’ve previously discussed how many AI benchmarks fall short, being either too simplistic, niche, or flawed.
In an effort to enhance the evaluation of vision-language models (VLMs), which understand both imagery and text, experts from the Allen Institute for AI (AI2) and other institutions have developed a new testing environment known as WildVision.
WildVision is a comprehensive platform hosting around 20 models, such as Google’s Gemini Pro Vision and OpenAI’s GPT-4o, along with a leaderboard showcasing user preferences during interactions with these models.
Through WildVision, AI2’s team identified areas where even the most advanced VLMs falter, including imaginations and spatial reasoning capabilities. Their findings, documented in an accompanying research paper, suggest directions for future advancements in VLMs.
Highlighted Model
This week features Anthropic’s launch of its Artifacts function for all users, which transforms interactions with its Claude models into a variety of digital outputs including apps, visuals, dashboards, and websites.
Initially previewed in June, Artifacts now freely available on the web and via Anthropic’s Claude mobile applications, provides a space to view and engage with creations made through Claude. The feature allows for publishing and community collaboration on artifacts, with Anthropic’s Team plan subscribers accessing more exclusive sharing capabilities.
Michael Gerstenhaber, Anthropic’s product lead, explained to TechCrunch, “Artifacts change how generated content is used, giving users the opportunity to refine it through continuous dialogue. For example, if your goal is to create code, the artifact becomes part of the UI for iterative improvement, allowing you to adjust and execute the code.”
It’s noteworthy that Poe, an AI model aggregator by Quora, offers a similar feature named Previews. However, unlike Artifacts, access to Previews comes with a subscription fee of $20 per month under Poe’s premium plan.
Extra Tidbits
OpenAI may soon introduce Strawberry, a project believed to enhance problem-solving capabilities beyond its current models.
As per The Information, Strawberry, previously known as Q*, aims to tackle complex challenges across mathematics, programming, and word puzzles with improved reasoning. However, this added intelligence might require more processing time than GPT-4o.
Plans are in place to integrate Strawberry’s capabilities with ChatGPT this fall. Moreover, OpenAI is experimenting with Strawberry to produce synthetic data for training models, including an upcoming model codenamed Orion.
The AI community has high expectations for Strawberry. Whether OpenAI will fulfill these expectations remains to be seen, but at the very least, an enhancement in spelling accuracy for ChatGPT would be welcomed.
Compiled by Techarena.au.
Fanpage: TechArena.au
Watch more about AI – Artificial Intelligence
