Japan will hand over to the United States more than 315 kilograms (700 pounds) of weapons-grade plutonium and a supply of highly enriched uranium, officials said on Monday.
American and Japanese officials announced the deal at a two-day nuclear security summit in The Hague - the meeting's first major breakthrough.
US Energy Secretary Ernest Moniz told reporters that the material will be transferred to the United States for transformation into "proliferation-resistant forms".
Yosuke Isozaki, a senior national security adviser to Japanese Prime Minister Shinzo Abe, said handing over the highly enriched uranium and plutonium is part of Japan's efforts to prevent proliferation and possible abuse of nuclear material by militant groups - the main aim of the Hague summit.
As part of the deal, the US will continue to receive spent reactor fuel from Japan for an additional 10 years.
Japan originally received the material from the US and Britain in the 1960s for use in research.
In addition to the weapons-grade material Japan is giving back, it maintains a stockpile of 9.3 tons of lesser-grade plutonium that could be easily weaponised by a country of Japan's sophistication in a matter of months.
Additionally Japan's new Rokkasho nuclear plant, due to come online this year, is capable of producing many more tons of plutonium per year when operational.
Yet, with most of Japan's nuclear plants still shut down in the wake of the 2011 Fukushima nuclear plant disaster, there is no apparent use for that material.