I agree with Paul Krugman on one point -- The Heritage Foundation is an advocacy group, so their numbers should always be met with some skepticism. This is true of any partisan think tank.
It's also true of partisan op-ed columnists for the New York Times.
Update: Greg Mankiw argues that the issue of administrative costs is really a red herring:
Low administrative costs are not to be confused with high administrative efficiency. In other words, administrators are not necessarily a deadweight loss to the system.
Medicare is consistently cited by the GAO as a program that is at "high risk" for fraud. In fact, Medicare administrators have repeatedly pleaded with Congress to increase funding for anti-fraud measures, which would ultimately raise Medicare's administrative costs:
Congress keeps a tight cap on Medicare's administrative overhead, leaving the perennial ''high-risk'' entitlement program with an infinitesimal fraction of its current $456 billion budget to combat fraud.
Indeed, Congress has a history of running Medicare -- whose healthcare programs are funded by U.S. taxpayers -- on the cheap. That policy has contributed in part to a generation of scammers bilking billions out of Medicare, which critics describe as a flawed honor system that's more intent on paying claims quickly than on verifying them first.
Update II: Megan McArdle weighs in on the Medicare debate . . . and on the larger debate over national health care.