Preheat the oven to 400°F (200°F). Spray the madeleine pan with non-stick cooking spray. Melt the butter and stir in the loose earl grey tea. Steep while you mix the batter.
In a large mixing bowl with the whisk attachment, beat the eggs and sugar on medium high until nice and thick, about 5 minutes. This mixture should fall into a thick ribbon when you lift the whisk.
Mix in the vanilla extract.
Sift flour, baking powder, and salt into the egg mixture and gently fold the batter with a spatula until no dry flour is visible.
Strain the butter through a fine mesh sieve into the batter and discard the tea leaves.
Gently fold the batter to combine.
Divide the batter into the madeleine pan, about one heaping tablespoon of batter into each well.
Bake the madeleines for 3 minutes, then reduce the oven temperature to 350°F for 7 minutes.
Let the madeleines cool in the pan for 3 minutes, then transfer them onto wire rack to cool completely.
Put the lemon curd in a piping bag, cut a small hole in the end, and pipe the curd into the center of each cooled madeleine. Serve right away.