Preheat the oven to 350F. Grease and line with parchment paper a 9x5 inch loaf pan.
In a large mixing bowl or the bowl of a stand mixer, whisk together the flour, sugar, brown sugar, pumpkin pie spice, cardamom, and baking powder; mix well.
Add the softened butter and using an electric mixer, mix at medium speed, until pea size chunks form.
Turn the mixer up to high for 1 minute, until the mixture starts to look more like a thick dough.
Add in the eggs, beating until mixed. Make sure to scrape the sides and bottom of the bowl every so often.
Add the pumpkin puree and beat just until it's combined.
Spread the batter into the prepared pan bake for about 55-60 minutes, until the bread is done and not soft in the middle. You can use a cake tester or toothpick.
Allow the cake to cool in the pan for 15-30 minutes and then remove from the pan and allow it cool to room temperature.
Cream Cheese Frosting
Make sure the cream cheese is nice and soft.
Add the powdered sugar, vanilla, and only 1 tablespoon of the milk. Mix well until it's completely smooth. If it's too thick, add more milk.
Spread the icing on top of the cooled pumpkin bread, and let it drizzle off the sides a bit.