Taro Aso's comments came after chief US central banker Janet Yellen said the Federal Reserve had no powers to control the unit, which she said would be very difficult to regulate.
Aso said he had foreseen difficulties for the crypto-currency, which is generated by complex chains of interaction among a huge network of computers around the planet.
"I was thinking that this sort of thing won't last long," said Aso, 73. "I was thinking it would collapse sometime."
"Japan is overwhelmingly advanced in this field. In this sense, I was thinking since before that we might face a situation where Japan has to act, but I'd say it came earlier than I thought."
The global bitcoin community has been shaken this week by claims that hundreds of millions of dollars' worth of the unit have been syphoned from the digital vaults of the Mt. Gox exchange.
Withdrawals were frozen earlier this month because of what the firm said was a bug in the software underpinning bitcoin that allowed hackers to pilfer them.
Supporters rallied round, insisting that the bitcoin itself is sound and the problems lay with Mt. Gox, which they said was badly-managed and unable to cope with the burgeoning popularity of the young currency.
Aso said Friday that police and government ministries would work together to address the issues raised by the Mt. Gox episode.
"We will hurriedly work it out," he said, adding: "It is not a currency that all the people recognize."
His comments come after Yellen told a Senate hearing that banks under the Fed's regulatory purview were not involved with bitcoin, and so it had no powers in the matter.
"This is a payment innovation that is taking place entirely outside of the banking industry," she said.
"The Fed doesn't have authority with respect to bitcoin." Yellen pointed to other US regulators, including the Treasury, watching the bitcoin sector for potential money laundering and other criminal uses of the digital currency.
However, she stressed, "it's not so easy to regulate bitcoin" because it is used outside the banking system, and has no centralised authority behind, as do regular fiat currencies.
In Tokyo on Thursday night, investors were calling for a legal probe and possible criminal action against the chief executive of the embattled Mt. Gox, Mark Karpeles.
American IT engineer Aaron Gotman said he had 463 bitcoins at Mt. Gox, worth well over $200,000.
"What happened at Mt. Gox was first incompetence, but then it could have gone to fraud at the end if they were knowing they didn't have the bitcoins they were selling," he said.
"If that's true, Karpeles should face justice and go to jail." Bitcoin's rollercoaster week showed signs of settling down on Friday, trading around $567 at 0545 GMT, down approximately 1.3% for 24 hours, according to the Coindesk index.
While that kind of swing is large by the standards of many established currencies and generally indicates a sizeable news event, it is small bear for a unit that began life worth a few cents and frothed its way well above the $1,000 mark last year.
The Mt. Gox website which dramatically went blank on Tuesday, was still carrying a short message from Karpeles on Friday, his only public statement in recent days as the crisis has swirled around him.
"As there is a lot of speculation regarding Mt. Gox and its future, I would like to use this opportunity to reassure everyone that I am still in Japan, and working very hard with the support of different parties to find a solution to our recent issues," it said.
"Furthermore I would like to kindly ask that people refrain from asking questions to our staff: they have been instructed not to give any response or information. Please visit this page for further announcements and updates."