The makers of ChatGPT have announced the company will be dedicating 20% of its compute processing power over the next four years to stop superintelligent AI from “going rogue." OpenAI is opening a new alignment research division, focused on developing training techniques to stop superintelligent AI — artificial intelligence that could outthink humans and become misaligned with humans ethics — from causing serious harm. “Currently, we don’t have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue,” Jan Leike and Ilya Sutskever wrote in a blog post for OpenAI, the company behind the most well-known generative AI large language model, ChatGPT. They added that although superintelligence might seem far off, some experts believe it could arrive this decade. Current techniques for aligning AI include reinforcement learning from human feedback, but Leike and Sutskever said that as AI systems become smarter than humans, humans can no longer be relied upon to supervise the technology. “Current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs,” they wrote. Sutskever is a co-founder and chief scientist at OpenAI, and Leike is a machine learning researcher. They will co-lead OpenAI’s new superalignment team. In order to undertake its mission, over the next four years the division will have access to 20% of the company’s processing capacity to build a “human-level automated alignment researcher” that can be scaled up to supervise superintelligence. In order to align the automated researcher with human ethics, Leike and Sutskever said a three-step approach will need to be taken: develop a scalable training method; validate the resulting model; and stress test the entire alignment pipeline. “We expect our research priorities will evolve substantially as we learn more about the problem and we’ll likely add entirely new research areas,” they wrote, adding there were plans to share more of the division’s roadmap in the future. OpenAI acknowledges the need to mitigate potential AI harm This isn’t the first time OpenAI has publicly acknowledged the need to mitigate the risks posed by unregulated AI. In May, company CEO Sam Altman signed an open letter stating that controlling the tech should be a top global priority as the evolution of AI could lead to an extinction event. “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war,” the letter read. OpenAI also has a dedicated section on its website, where the public can access materials related to what the company calls the development of “safe and responsible AI,” alongside a charter outlining the principles it adheres to in order to execute its mission. However, these largely relate to the concept of artificial general intelligence (AGI) — highly autonomous systems that outperform humans at most economically valuable work. “We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome,” the charter, which was published in 2018, reads. Related content news analysis Within two years, 90% of organizations will suffer a critical tech skills shortage Along with AI skills, skills in IT ops and cloud development are severely lacking, IDC reports. By Lucas Mearian May 31, 2024 7 mins Generative AI IT Jobs IT Skills news analysis Google partners with Magic Leap on mixed reality development The Magic Leap collaboration follows Google’s partnership to develop mixed reality devices with Samsung last year. By Matthew Finnegan May 31, 2024 3 mins Augmented Reality Virtual Reality Emerging Technology how-to Microsoft OneDrive cheat sheet: Using OneDrive in Windows If you have Windows 10 or 11, you have OneDrive. With this quick guide, you can learn how to sync, back up, and share files in OneDrive with built-in Windows tools. By Howard Wen May 31, 2024 15 mins OneDrive Cloud Storage Windows analysis Apple's mutating supply chain sees winners and losers Shifting manufacturing centers are generating fresh challenges. By Jonny Evans May 31, 2024 5 mins iPhone Manufacturing Industry Apple Podcasts Videos Resources Events SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe