Microsoft Copilot

microsoft-copilot-will-watch-you-play-minecraft,-tell-you-what-you’re-doing-wrong

Microsoft Copilot will watch you play Minecraft, tell you what you’re doing wrong

gaming, Microsoft Copilot, minecraft / /

Parasocial gaming department —

Microsoft demo is like chatting with GameFAQs when you don’t have a friend to hang with.

–

In the recent past, you'd have to rely on your kid sibling to deliver <em>Minecraft</em> commentary like “Oh no, it’s a zombie. Run!”” src=”https://cdn.arstechnica.net/wp-content/uploads/2024/05/zombie-800×429.png”></img><figcaption> <p><a data-height=Enlarge / In the recent past, you’d have to rely on your kid sibling to deliver Minecraft commentary like “Oh no, it’s a zombie. Run!”

Longtime gamers (and/or Game Grumps fans) likely know that even single-player games can be a lot more fun with a friend hanging out nearby to offer advice, shoot the breeze, or just offer earnest reactions to whatever’s happening on screen. Now, Microsoft is promising that its GhatGPT-4o-powered Copilot system will soon offer an imitation of that pro-social experience even for Minecraft players who don’t have any human friends available to watch them play.

In a pair of social media posts Monday, Microsoft highlighted how “real-time conversations with your AI companion copilot” can enhance an otherwise solitary Minecraft experience. In the first demo, the disembodied copilot voice tells the player how to craft a sword, walking him through the process of gathering some wood or stone to go with the sticks sitting in his inventory. In another, the AI identifies a zombie in front of the player and gives the (seemingly obvious) advice to run away from the threat and “make sure it can’t reach you” by digging underground or building a tower of blocks.

Real time conversations with your AI companion Copilot, powered by OpenAI’s GPT-4o. pic.twitter.com/Ug7EWv2sah

— Microsoft Copilot (@MSFTCopilot) May 20, 2024

These kinds of in-game pointers aren’t the most revolutionary use of conversational AI—even a basic in-game tutorial/reference system or online walkthrough could deliver the same basic information, after all. Still, the demonstration stands out for just how that information is delivered to the player through a natural language conversation that doesn’t require pausing the gameplay even briefly.

The key moment highlighting this difference is near the end of one of the video demos, when the Copilot AI offers a bit of encouragement to the player: “Whew, that was a close one. Great job finding shelter!” That’s the point when the system transitions from a fancy voice-controlled strategy guide to an ersatz version of the kind of spectator that might be sitting on your couch or watching your Twitch stream. It creates the real possibility of developing a parasocial relationship with the Copilot guide that is not really a risk when consulting a text file on GameFAQs, for instance (though I think the Copilot reactions will have to get a bit less inane to really feel like a valued partner-in-gaming).

Just hanging out with my AI buddy

It’s unclear from the video clips whether Copilot is reading data directly from the Minecraft instance or simply reacting to the same information the player is seeing. But the social media posts came the same day as Microsoft’s announcement of “Recall,” a coming feature that “take[s] images of your active screen every few seconds” to provide a persistent “memory” of everything you do on the computer. That feature will be exclusive to Copilot+ PCs, which use an integrated Neural Processing Unit for on-device processing of many common generative AI tasks.

Microsoft’s Minecraft copilot demo brings to mind some of the similar conversations that OpenAI showed off during last week’s live demo of ChatGPT-4o. But the artificial game-adjacent conversation here sounds significantly more robotic and direct than the lifelike, emotional responses in ChatGPT’s presentation. Then again, ChatGPT has come under fire from actress Scarlett Johansson for using a voice that sounds too much like her performance in a 2013 movie about a conversational AI. Microsoft might be on safer ground sticking with a voice that is more obviously artificial here.

The casual cursing is what really makes an AI gaming buddy feel real.

Speaking of Her, we can’t help but think of one particular scene in that movie where Joaquin Phoenix’s Theodore asks for gaming advice both from the titular AI and a hilariously potty-mouthed NPC. Maybe Microsoft can add a casual cursing module to its Copilot gaming companion to really capture the feeling of hanging out with a dorm room buddy over a late-night gaming session.

Microsoft Copilot will watch you play Minecraft, tell you what you’re doing wrong Read More »

llms-keep-leaping-with-llama-3,-meta’s-newest-open-weights-ai-model

LLMs keep leaping with Llama 3, Meta’s newest open-weights AI model

AI, AI assistants, Biz & IT, chatgpt, chatgtp, copilot, Google Gemini, LLaMA, Llama 2, Llama 3, machine learning, Meta, Meta AI Assistant, microsoft, Microsoft Copilot, Open Source, open weights, openai / /

computer-powered word generator —

Zuckerberg says new AI model “was still learning” when Meta stopped training.

–

A group of pink llamas on a pixelated background.

On Thursday, Meta unveiled early versions of its Llama 3 open-weights AI model that can be used to power text composition, code generation, or chatbots. It also announced that its Meta AI Assistant is now available on a website and is going to be integrated into its major social media apps, intensifying the company’s efforts to position its products against other AI assistants like OpenAI’s ChatGPT, Microsoft’s Copilot, and Google’s Gemini.

Like its predecessor, Llama 2, Llama 3 is notable for being a freely available, open-weights large language model (LLM) provided by a major AI company. Llama 3 technically does not quality as “open source” because that term has a specific meaning in software (as we have mentioned in other coverage), and the industry has not yet settled on terminology for AI model releases that ship either code or weights with restrictions (you can read Llama 3’s license here) or that ship without providing training data. We typically call these releases “open weights” instead.

At the moment, Llama 3 is available in two parameter sizes: 8 billion (8B) and 70 billion (70B), both of which are available as free downloads through Meta’s website with a sign-up. Llama 3 comes in two versions: pre-trained (basically the raw, next-token-prediction model) and instruction-tuned (fine-tuned to follow user instructions). Each has a 8,192 token context limit.

A screenshot of the Meta AI Assistant website on April 18, 2024.

Enlarge / A screenshot of the Meta AI Assistant website on April 18, 2024.

Benj Edwards

Meta trained both models on two custom-built, 24,000-GPU clusters. In a podcast interview with Dwarkesh Patel, Meta CEO Mark Zuckerberg said that the company trained the 70B model with around 15 trillion tokens of data. Throughout the process, the model never reached “saturation” (that is, it never hit a wall in terms of capability increases). Eventually, Meta pulled the plug and moved on to training other models.

“I guess our prediction going in was that it was going to asymptote more, but even by the end it was still leaning. We probably could have fed it more tokens, and it would have gotten somewhat better,” Zuckerberg said on the podcast.

Meta also announced that it is currently training a 400B parameter version of Llama 3, which some experts like Nvidia’s Jim Fan think may perform in the same league as GPT-4 Turbo, Claude 3 Opus, and Gemini Ultra on benchmarks like MMLU, GPQA, HumanEval, and MATH.

Speaking of benchmarks, we have devoted many words in the past to explaining how frustratingly imprecise benchmarks can be when applied to large language models due to issues like training contamination (that is, including benchmark test questions in the training dataset), cherry-picking on the part of vendors, and an inability to capture AI’s general usefulness in an interactive session with chat-tuned models.

But, as expected, Meta provided some benchmarks for Llama 3 that list results from MMLU (undergraduate level knowledge), GSM-8K (grade-school math), HumanEval (coding), GPQA (graduate-level questions), and MATH (math word problems). These show the 8B model performing well compared to open-weights models like Google’s Gemma 7B and Mistral 7B Instruct, and the 70B model also held its own against Gemini Pro 1.5 and Claude 3 Sonnet.

A chart of instruction-tuned Llama 3 8B and 70B benchmarks provided by Meta.

Enlarge / A chart of instruction-tuned Llama 3 8B and 70B benchmarks provided by Meta.

Meta says that the Llama 3 model has been enhanced with capabilities to understand coding (like Llama 2) and, for the first time, has been trained with both images and text—though it currently outputs only text. According to Reuters, Meta Chief Product Officer Chris Cox noted in an interview that more complex processing abilities (like executing multi-step plans) are expected in future updates to Llama 3, which will also support multimodal outputs—that is, both text and images.

Meta plans to host the Llama 3 models on a range of cloud platforms, making them accessible through AWS, Databricks, Google Cloud, and other major providers.

Also on Thursday, Meta announced that Llama 3 will become the new basis of the Meta AI virtual assistant, which the company first announced in September. The assistant will appear prominently in search features for Facebook, Instagram, WhatsApp, Messenger, and the aforementioned dedicated website that features a design similar to ChatGPT, including the ability to generate images in the same interface. The company also announced a partnership with Google to integrate real-time search results into the Meta AI assistant, adding to an existing partnership with Microsoft’s Bing.

LLMs keep leaping with Llama 3, Meta’s newest open-weights AI model Read More »

report:-sam-altman-seeking-trillions-for-ai-chip-fabrication-from-uae,-others

Report: Sam Altman seeking trillions for AI chip fabrication from UAE, others

AI, AI chips, Biz & IT, Google Gemini, large language models, machine learning, microsoft, Microsoft Copilot, openai, sam altman, semiconductor manufacturing, text synthesis, UAE / /

chips ahoy —

WSJ: Audacious $5-$7 trillion investment would aim to expand global AI chip supply.

–

WASHINGTON, DC - JANUARY 11: OpenAI Chief Executive Officer Sam Altman walks on the House side of the U.S. Capitol on January 11, 2024 in Washington, DC. Meanwhile, House Freedom Caucus members who left a meeting in the Speakers office say that they were talking to the Speaker about abandoning the spending agreement that Johnson announced earlier in the week. (Photo by Kent Nishimura/Getty Images)

Enlarge / OpenAI Chief Executive Officer Sam Altman walks on the House side of the US Capitol on January 11, 2024, in Washington, DC. (Photo by Kent Nishimura/Getty Images)

Getty Images

On Thursday, The Wall Street Journal reported that OpenAI CEO Sam Altman is in talks with investors to raise as much as $5 trillion to $7 trillion for AI chip manufacturing, according to people familiar with the matter. The funding seeks to address the scarcity of graphics processing units (GPUs) crucial for training and running large language models like those that power ChatGPT, Microsoft Copilot, and Google Gemini.

The high dollar amount reflects the huge amount of capital necessary to spin up new semiconductor manufacturing capability. “As part of the talks, Altman is pitching a partnership between OpenAI, various investors, chip makers and power providers, which together would put up money to build chip foundries that would then be run by existing chip makers,” writes the Wall Street Journal in its report. “OpenAI would agree to be a significant customer of the new factories.”

To hit these ambitious targets—which are larger than the entire semiconductor industry’s current $527 billion global sales combined—Altman has reportedly met with a range of potential investors worldwide, including sovereign wealth funds and government entities, notably the United Arab Emirates, SoftBank CEO Masayoshi Son, and representatives from Taiwan Semiconductor Manufacturing Co. (TSMC).

TSMC is the world’s largest dedicated independent semiconductor foundry. It’s a critical linchpin that companies such as Nvidia, Apple, Intel, and AMD rely on to fabricate SoCs, CPUs, and GPUs for various applications.

Altman reportedly seeks to expand the global capacity for semiconductor manufacturing significantly, funding the infrastructure necessary to support the growing demand for GPUs and other AI-specific chips. GPUs are excellent at parallel computation, which makes them ideal for running AI models that heavily rely on matrix multiplication to work. However, the technology sector currently faces a significant shortage of these important components, constraining the potential for AI advancements and applications.

In particular, the UAE’s involvement, led by Sheikh Tahnoun bin Zayed al Nahyan, a key security official and chair of numerous Abu Dhabi sovereign wealth vehicles, reflects global interest in AI’s potential and the strategic importance of semiconductor manufacturing. However, the prospect of substantial UAE investment in a key tech industry raises potential geopolitical concerns, particularly regarding the US government’s strategic priorities in semiconductor production and AI development.

The US has been cautious about allowing foreign control over the supply of microchips, given their importance to the digital economy and national security. Reflecting this, the Biden administration has undertaken efforts to bolster domestic chip manufacturing through subsidies and regulatory scrutiny of foreign investments in important technologies.

To put the $5 trillion to $7 trillion estimate in perspective, the White House just today announced a $5 billion investment in R&D to advance US-made semiconductor technologies. TSMC has already sunk $40 billion—one of the largest foreign investments in US history—into a US chip plant in Arizona. As of now, it’s unclear whether Altman has secured any commitments toward his fundraising goal.

Updated on February 9, 2024 at 8: 45 PM Eastern with a quote from the WSJ that clarifies the proposed relationship between OpenAI and partners in the talks.

Report: Sam Altman seeking trillions for AI chip fabrication from UAE, others Read More »