The scenario described is an example of disruptive selection. In disruptive selection, individuals at both extremes of a trait distribution have a higher fitness than those with intermediate traits. In this case, brightly colored males and very dull males are more fit compared to males with intermediate coloring.
As a result of this selection pressure, it is most likely true that the proportion of males with intermediate coloring will decrease over time. This is because individuals with intermediate coloring do not have the same survival advantage as those at the extremes, leading to a potential decline in that phenotype within the population.