On-Call

Online services run around the clock, and problems do not wait for business hours. Someone needs to be ready to respond when an alert fires at three in the morning or a customer reports an outage on a public holiday. On-call is how organizations provide that coverage. Engineers take turns being responsible for responding to alerts and incidents for a set period, often a week, with a backup and an escalation path if they need help. Done well, on-call keeps services reliable without burning people out. Done badly, it becomes a source of exhaustion, frustration and turnover. The difference usually depends on how many alerts fire, how well responders are supported and how fairly the load is shared.

On-call has long been part of operations and support work. The DevOps and site reliability engineering movements changed who carries it, encouraging the developers who build a service to share responsibility for running it. Google's SRE guidance, for example, recommends keeping the number of incidents per on-call shift low so engineers can respond thoughtfully and follow up properly. For engineers, engineering managers, SRE teams and HR and finance leaders who set on-call policies, getting on-call right is essential for both reliability and team health.

Definition of On-Call

On-call is an operational arrangement in which designated engineers are responsible, during scheduled periods, for responding to automated alerts and reported incidents for specific services within agreed response times, supported by escalation policies, runbooks and tools for paging and coordination.

It is the first line of incident management, is shaped by alerts based on service level objectives (SLOs) and is a core responsibility in site reliability engineering (SRE).

Common Rotation Models

Weekly rotations assign one engineer as primary for a week, with another as secondary backup. This is the most common model for small and medium teams.

Follow-the-sun rotations pass responsibility between teams in different time zones, so each covers its own daytime and nobody needs to be woken at night.

Split rotations separate business hours and out-of-hours coverage, or separate different services, to reduce the load on any one person.

Shadow rotations pair new engineers with experienced ones before they take primary duty, building confidence and knowledge.

Key Elements

Alerting and paging tools notify the on-call engineer and escalate to the next person if there is no response.

Escalation policies define who to involve for different problems, such as senior engineers, other teams or incident commanders.

Runbooks describe how to diagnose and fix common problems, so responders do not need to rely on memory.

Handoffs at the end of each shift pass on open issues, recent changes and anything the next person should watch. Access to production systems, dashboards and documentation must be in place before a shift starts, not requested during an incident.

Importance of On-Call in Delivery and Engineering Operations

On-call protects customers. Fast response to incidents outside business hours limits downtime and impact.

It improves ownership and quality. Developers who support their own services in production have a strong incentive to build them to be reliable and easy to operate.

It also surfaces problems. On-call experience reveals noisy alerts, missing runbooks and fragile systems that need investment. It spreads knowledge. Engineers who take on-call shifts learn how systems behave in production, which improves design decisions.

On-Call and Financial Management

Many organizations pay on-call allowances or provide time off for out-of-hours work, which is a direct cost to plan for.

Poorly managed on-call contributes to burnout and turnover, and replacing experienced engineers is expensive.

Faster out-of-hours response reduces the cost of incidents for revenue-generating services.

Investing in fewer, better alerts and in automation reduces both incident costs and the human cost of on-call.

Common Challenges

Alert fatigue, caused by many alerts that need no action, leads responders to ignore or miss important ones.

Small teams can end up with rotations so frequent that engineers are on call too often to rest properly.

On-call that is not recognized or compensated feels unfair and damages morale. Knowledge concentrated in a few experienced engineers means they get called even when off duty, which is unsustainable.