Preheat the oven to 350, and line a cupcake pan with parchment paper liners.
In a medium bowl, whisk together eggs, oil, water, vanilla extract, lemon extract, and lemon zest. Add cake mix and whisk until well combined.
Add about 3 tbsp of batter to each cupcake liner, and bake for about 17 minutes until cooked through and some browning starts to occur around the edges of the cupcakes. Remove from oven and let cupcakes completely cool.
While the cupcakes are baking, make the frosting. Using a mixer, cream together butter and cream cheese for about 2 minutes. Add milk, vanilla and lemon extract, mix until combined, then add in the Swerve. Starting on low speed, mix until Swerve is combined, then increase speed to high and mix until light and fluffy, about 8 minutes.