Is it possible through the UI while leveling, or in char sheet, to replace existing domain card with another of same level or lower (per rules)?
When you go back into the Character Builder you can navigate to the choices you made on previous level ups and deselect the card you chose at that time in order to replace it with another for that level.
It won’t let you choose a higher level though. I want to get ride of all my level 2 spells and upgrade to level 4 when I reach level 4. How do I do that?
I just double checked the rules text and that’s by design. When you gain a level you have the option of taking a domain card of your new level or lower and then you also have the option of swapping 1 domain card for an equal level or lower.
So, RAW, you cannot do what you want to do. You could pick new cards at level up - if you used you two level up options + the domain card we get for free at level up - you could gain three new domain cards at level 4, but if that made you go over the limit of 5, you would have to place those level 2 cards in your vault. The positive news is that when you use your higher-level abilities for the session, you can 1) swap cards out of the vault at a rest or 2) pay the recall cost to switch a card from your loadout to the vault.