Subtitle: Navigating the Path to General Artificial Intelligence and Meta’s Strategic Shift
Mark Zuckerberg, the visionary behind Meta, recently announced a paradigm-shifting move towards the pursuit of General Artificial Intelligence (AI). General AI refers to a form of artificial intelligence that can comprehend, learn, and apply its intelligence across a broad spectrum of tasks, mirroring and even surpassing human capabilities. Unlike Specific or Limited-Purpose AI, which is designed and trained for a particular task (e.g., DALL-E, exclusively generating images), General AI could tackle any intellectual task a human can.
Zuckerberg highlights the distinctive feature of this technology, emphasizing its “breadth, which is that intelligence has all these different capabilities where you have to be able to reason and have intuition.”
Meta’s Plans for General AI: Inspired by Llama2
This renewed focus on General AI has been influenced by Meta’s latest Large Language Model (LLM), the Llama2. Despite acknowledging that the ability to generate code might not be directly relevant to their platform users, Meta decided to develop this function in pursuit of achieving a more intelligent AI.
“A hypothesis was that coding is not that important because it’s not like a lot of people are going to ask coding questions on WhatsApp. It turns out that coding is structurally very important for LLMs to understand the rigor and hierarchical structure of knowledge and, overall, have a more intuitive sense of logic.”
To facilitate a more direct impact on users, Meta has relocated its AI research team to the same area as the team creating its generative AI products.
Challenges and Resources: Meta’s Bold Move
It’s unsurprising that the company has chosen to invest in the development of General AI, a technology that, while promising an astronomical leap, is considered a futuristic endeavor due to its technical challenges. In fact, Zuckerberg refrains from providing a specific timeline for achieving this milestone.
On the flip side, the complexity of General AI has led to an increased demand for AI experts by major companies, engaging in intense talent wars. “We’re used to there being pretty intense talent wars. But here, there are different dynamics, with several companies looking for the same profile, and many venture capitalists and people investing money in different projects, making it easier for people to start different things externally,” remarked Zuckerberg in an interview.
In terms of material resources, Meta aims to have over 340,000 Nvidia H100 GPUs by the end of the year, a powerful graphics accelerator with 800 billion transistors, poised to propel their project forward. Zuckerberg expresses optimism, stating, “We’ve built the capability to do this at a scale that can be larger than any other individual company.”
The Significance of Open Source Models
The prospect of a potent technology like General AI coming into existence also raises ethical and social risks. Concentrating control over this technology under a single company does not paint a promising picture. To allay concerns on this front, Zuckerberg affirmed Meta’s commitment to open source.
“I tend to think that one of the biggest challenges here will be that if you build something that is really valuable, it will end up becoming very concentrated. Whereas if it’s more open, you’re addressing a big class of problems that could arise from unequal access to opportunities and value. That’s a big part of the whole vision of open source.”
In a clear reference to OpenAI (though not directly naming it), he adds, “all these companies that used to be open, used to publish all their work and used to talk about how they were going to open all their work in open source. I think you see the dynamic of people just realizing, ‘Hey, this is going to be really valuable, let’s not share it.'”
Meta’s Commitment to the Metaverse
Finally, Zuckerberg underscores that Meta has not abandoned its plans for the metaverse. “I don’t know how to state more unequivocally that we will continue to focus on Reality Labs and the metaverse.”
In fact, over $15 billion is being invested annually in this project, and the advancement in AI is expected to serve as a catalyst for its progress.