Brown color on dianthus

I bought perennial dianthus and it blooms fine at first but as flowers die the whole stem starts turning brown and thus doesn't flower more. I tried cutting off flower part once dead. Not sure what I am doing wrong. My soil here tends to have clay. they are planted in nice sunny location. FYI cant grow roses or azalea here.