They just don't make wood like they used to.
Yesterday's siding was made from trees which were a hundred years old. Made from hard hard wood. Today's siding is made from trees which grow like weeds and the wood is so soft you can dent it with your fingernail. You can paint, you can prime, you can caulk, but you are dealing with glorified cardboard.
If your house has "lap siding"
you are at an advantage. If my siding was beginning to rot, I would replace the boards with HardiPlank Lap Siding. Historic Planning Department won't like it very much, but it is the only way you can keep the siding from rotting out again.
While you have the siding off, pull out that nasty insulation and replace it with rigid panels which will provide insulating qualities and allow for airflow.
If you have other types of siding, Dolly Varden, novelty siding, or a variation of these two, it will not be so easy to fix your siding problem. You will have to replace wood with wood.