In a large saucepan, whisk together the 4 cups milk, the cocoa powder, sugar, and espresso powder until combined. Heat over medium heat just until simmering, whisking occasionally. Remove from heat. If desired, stir in liqueur.
Pour into mugs. If desired, for frothy topping:
In a small saucepan, heat the 1/2 cup milk over low heat until warm. Transfer warm milk to a blender; cover and blend until frothy. Spoon frothy topping onto individual servings.