Counterspell has two requirements and neither of them are related to sound.
Counterspell has a casting time of 1 Reaction, which:
you take when you see a creature within 60 feet of you casting a spell
The two requirements here are:
- You see a creature within 60 feet of you
- That creature is casting a spell
Neither of these have anything to do with sound or hearing, so you being deafened or the target being silent are entirely irrelevant. So you can be deafened and they can be silent; as long as you can see them and they are casting a spell, you may use counterspell.
Further, it is important to be careful not to add requirements that are not written here. That is, the spell does not require that you have ascertained with certainty that the target is casting a spell. If I can see while you are casting a spell, I have seen you casting a spell whether or not I have understood that you are casting a spell.
That said, the way this plays out in practice requires some conversation between the player and DM. When character knowledge of casting is dubious, I just ask the player to quickly talk through the moment from the character’s perspective to justify the casting of counterspell. When there’s really no reason the character would even be suspicious, players have almost always agreed they would not cast it.
One time I had a player who insisted that they were suspicious anyway that the enemy was casting a spell despite not having any visual (or audible) evidence. So I let them get that cast in, and then we had a conversation to the effect of ��if you’re always suspicious, you need to be okay with being wrong and sometimes wasting spell slots.” They agreed, and on occasion I would bait them into slinging a counterspell at nothing.