After you have acknowledged an incident, no other team members get alerted anymore.
You can manually acknowledge an incident by going to Incidents, and then clicking Acknowledge in the upper-right corner of an individual incident's detail page.
To acknowledge an incident, click the Acknowledge incident button in the e-mail you receive when a new incident is started.
It takes less than a minutes to setup your first monitoring.