Preheat your oven to 325 degrees F. Prepare your bread loaf pan by greasing it well so the bread doesn't stick.
In a large bowl cream together the sugar and the butter. When the sugar and butter is fluffy, add your eggs one at a time.
Add the mashed banana to this mixture. Mix until combined and then add the milk.
In a medium sized bowl combine the flour, baking powder, baking soda, salt, cinnamon and nutmeg.
Add the dry ingredients to the wet ingredients a little at a time to make sure the ingredients are mixed well. Don't overmix! This is also your opportunity to stir in some chopped walnuts if you prefer them.
Pour your batter into the prepared loaf pan. Bake for 60 minutes or until a toothpick inserted in the center comes out clean.
Let the bread cool on a wire rack for 10-20 minutes before taking it out of the bread pan. Allow the bread to cool completely before serving.