In a large bowl combine plums, tomatoes, vinegar, lemon juice, fish sauce, 1 Tbsp. of the oil, and the 2 tsp. rosemary. Let stand at room temperature for 30 minutes.
In a large skillet heat butter, remaining 1 Tbsp. of olive oil, and rosemary sprig over medium-high heat. Sprinkle the fish with salt and pepper. Add fish to skillet. Cook about 6 minutes or until fish flakes with a fork, turning once halfway through cooking. Remove and discard rosemary sprig. Transfer fish to a platter and spoon plum-tomato sauce over to serve.