Lumine: An Open Recipe for Building Generalist Agents in 3D Open Worlds (Nov 2025)
Title: Lumine: An Open Recipe for Building Generalist Agents in 3D Open Worlds (Nov 2025) Link: http://arxiv.org/abs/2511.08892v1 Date: November 2025 Summary: Lumine introduces the first open recipe for developing generalist AI agents that can complete hours-long complex missions in real-time within challenging 3D open-world environments. It uses a human-like interaction paradigm, unifying perception, reasoning, and action via a vision-language model (Qwen2-VL-7B-Base). Processing raw pixels at 5 Hz, Lumine generates precise 30 Hz keyboard-mouse actions and adaptively employs hybrid reasoning. Trained in Genshin Impact, it achieves human-level efficiency on the five-hour Mondstadt main storyline and follows natural language instructions for diverse tasks. Lumine also demonstrates strong zero-shot cross-game generalization, completing missions in Wuthering Waves and Honkai: Star Rail without fine-tuning, representing a significant step towards generalist agents in open-ended environments. Key Topics: - Generalist AI Agents - 3D Open Worlds - Vision-Language Models (VLMs) - Real-time Inference - Hybrid Reasoning - Human-like Interaction - Cross-Game Generalization - Long-Horizon Tasks - Imitation Learning Chapters: 00:00 - Introducing Lumen: A New AI Agent 00:34 - Lumen's Architecture & Core Idea 02:08 - Why Genshin Impact is Challenging 03:36 - The Need for a 7B Model 04:22 - Hybrid Thinking & Training 06:04 - Achieving Real-time Inference 07:05 - Latency Optimization & Memory 08:48 - Genshin Performance & Errors 09:51 - Zero-Shot Cross-Game Transfer 11:09 - Current Bottlenecks & Future Goals 12:25 - The Future of Generalist Agents Stock video credits: - Colin Jones - https://www.pexels.com/@larchmedia - Colors Motion Graphics - https://www.pexels.com/@colors-motion-graphics-183847699 - Pachon in Motion - https://www.pexels.com/@pachon-in-motion-426015731 - Pixabay - https://www.pexels.com/@pixabay - Kindel Media - https://www.pexels.com/@kindelmedia - crazy motions - https://www.pexels.com/@crazy-motions-80195021 - Danil Shostak - https://www.pexels.com/@danil-shostak-1324124 - Mikhail Nilov - https://www.pexels.com/@mikhail-nilov - Silviu Din - https://www.pexels.com/@silviu-din-1620549 - Bedrijfsfilmspecialist.nl - https://www.pexels.com/@bedrijfsfilmspecialist-nl-1284006 - StefWithAnF - https://www.pexels.com/@stefwithanf-1955763 - Engin Akyurt - https://www.pexels.com/@enginakyurt - Oleg Gamulinskii - https://www.pexels.com/@oleg-gamulinskii-755060 - @svetjekolem - https://www.pexels.com/@svetjekolem - Trippy Lagoon - https://www.pexels.com/@trippy-lagoon-511515544 - Dan Cristian Pădureț - https://www.pexels.com/@paduret - Yaroslav Shuraev - https://www.pexels.com/@yaroslav-shuraev - tunnel motions - https://www.pexels.com/@tunnelmotions - Kelly - https://www.pexels.com/@kelly - Charlie Mounsey - https://www.pexels.com/@charlie-mounsey-1653902 - Anthony 🙂 - https://www.pexels.com/@inspiredimages - Stas Knop - https://www.pexels.com/@stasknop - José Alfredo Munguía Lira - https://www.pexels.com/@rectorretro - Pressmaster - https://www.pexels.com/@pressmaster - Soumya - https://www.pexels.com/@soumya-1446957 - KATRIN BOLOVTSOVA - https://www.pexels.com/@ekaterina-bolovtsova - Pavel Danilyuk - https://www.pexels.com/@pavel-danilyuk - cottonbro studio - https://www.pexels.com/@cottonbro Буферизовано с YouTube (https://www.youtube.com/watch?v=ohF4t1RHFl4) на время плохой работы YouTube.