The call comes in as "the server is down". By the time a client says that sentence out loud, the maintenance relationship has already been damaged, whatever you find when you look.
Ping is the network-layer answer to that: is the machine reachable at all? It covers the hosts that have no homepage to request, which is most of the infrastructure sitting behind a client site.
It is deliberately a small signal. A host that answers ICMP can still be running a broken application, a full disk, or a mail queue nobody is processing. What it gives you is the earliest possible confirmation that the box itself has gone.