Mr Krishna had no "moral right" to continue after he brought "shame" to India, senior Bharatiya Janata Party (BJP) leader Venkaiah Naidu said.
Mr Krishna read out the wrong speech on Saturday for about three minutes before being corrected by an official.
Later, he said it was "a mistake" and that there was "nothing wrong in it".
Explaining the blunder, Mr Krishna said these things happen because many of these speeches make the same initial points.