Agreed. GHG don't add energy. They retain/slow the transfer of energy, right?When comes to the atmosphere and earth, the hotter earth radiates energy out at around 400 watts/m^2. And the radiation from the cooler atmosphere radiates back to earth around 330 watts/m^2. So the net radiation is from the earth outward. The atmosphere does not warm the earth at all. What it does is keeps the earth from loosing as much heat as it would if the atmosphere wasn't there.
Another important concept to keep in mind is that both the Earth and atmosphere are being warmed by the Sun. It is not the much simpler case of both cooling as fast as they can.