Sounds to me like the kid is also having some feelings surrounding the breakup and subsequent remarriage that aren’t being addressed. Why does he want you to come too? Is he hoping it might bring you and his mom back together? Does he feel uncomfortable around the new stepdad, and wants you around because he’s more comfortable when you’re around? I think if you have a conversation with him as to why he’s asking for you to come too, it might influence how you approach the situation, or at least give you a better way to explain why you can’t come due to your own reasons.
I know this is a difficult decision on your part for how it affects you, but your son is also in a very vulnerable position right now, and needs both of his parents paying attention to him and the feelings he’s having, even if he doesn’t know how to express them directly. My parents broke up suddenly due to cheating when I was around the same age, and it was a traumatic time in my life because my parents both assumed I was old enough to “get it.” I wasn’t. Family is one of the main sources of stability in a young person’s life, and to have it fall out from beneath you isn’t something you get over on your own very easily.
This is honestly the reason why I don’t think we can achieve a successful uprising anymore. Probably not a nuke, but drones definitely could and would be used to tear through even the largest of mobs if they formed today. Marie Antoinette would be happily eating her cake watching her people get mowed down by autonomous turrets if the French revolution happened today.