This is just post-real Calvinball used as a shield from criticism. Imagine someone saying to a mathematician who finds an error in a theorem that is false, “you can’t criticize the proof until you come up with valid proof.”(As an aside, I should note that agree that DSGE macro stinks, and I have used the Calvinball metaphor myself.)
My academic experience was in Control Systems Theory, a branch of applied mathematics. I ran into a case where I was an anonymous referee for a submitted paper. The new paper contained Theorem B, and Theorem B cited Theorem A within its proof. Meanwhile, Theorem A was contained in another paper that was published in the #2 journal in the field.
The result looked dubious, and I managed to disprove Theorem A within 10 seconds of picking up the volume that held it. (Luckily, the volume opened close to the page with the alleged "proof.") That was a personal speed record, and it was the most entertaining moment of my academic career.
I was able to write a short response rejecting the paper, explaining that the cited theorem was in fact incorrect; hence there was no need to pursue the new paper without a total re-write. Since it was someone else who made the original error, I was able to be quite gracious to the authors. (Although I doubt that made them any happier.)
And that was effectively the end of it. I did not write an anguished public letter to the editors of the #2 journal, telling them that they published a clunker of a paper. I just ignored its existence when I did literature surveys. (I discovered that there was a small sub-literature built around Theorem A.) If anyone objected to my ignoring that paper, I would have quietly sent them the explanation.
I would not say that I was told to ignore the existence of dubious papers, but that was my impression that was how the serious mathematicians within control systems dealt with the problem. (The quality of mathematics within control systems was uneven, as the academics ranged from engineers to people with doctorates in pure mathematics.) From the outside, it may have looked like it was a cabal that was playing favourites, but the reality is that there is no clean way to retract papers from a journal without making a lot of people look foolish.
Pure mathematicians probably have a more rigorous attitude within their journals, but at the same time, there are presumably not a lot of incorrect proofs found there.
An Application To Economics?If you are a researcher, and you think a result is wrong -- and you can prove this -- you should ignore its existence. If someone complains, you just send them the proof that the result is incorrect. That should be the end of the story.
Unfortunately, macro is not in this position; there are no standards of correctness that correspond to that of mathematics. There are problems in the mathematical modelling, and model predictions are routinely contradicted by data. If very strict standards of rigour were applied, the carnage in the post-war literature would be spectacular. (And yes, the post-Keynesian literature would likely be hammered as well.)
Since a wholesale deletion of the existing literature is not going to happen as a result of the institutional structure, the only way forward is to hope that a new methodology can appear, allowing researchers to ignore the existence of the dodgy current literature. This is going to be much easier for heterodox economists to achieve, as their reputations are not based upon that literature. That said, they are up against the institutional factors that favour the status quo.
(c) Brian Romanchuk 2016