Yes, you would need annulments.
A declaration of nullity ("an annulment") is a recognition that a marriage was invalid from its inception.
Consequently, it does not depend at all on anything that happened after the marriage (although events after the marriage may be evidence indicative of what was present before the marriage).
Thus, adultery is not a justification for an annulment. An annulment is not a "Catholic divorce"; marriages cannot be dissolved. If that were true, if you wanted to get divorced and remarried, all you'd have to do is commit adultery, and thus the marriage would be dissolved!
Speak to your local priest; he can shepherd you through the process.