Break-glass access should feel deliberately inconvenient

A production engineer needs to make a privileged change.

The ordinary deployment workflow requires:

• a reviewed change
• an approved pipeline
• a successful plan
• automated policy checks

The emergency role requires:

• one command
• one easily available password
• no approval
• broad administrator permissions

Which path will eventually become popular?

Probably the one labelled emergency.

NORMAL PATH

request
review
pipeline
validation
deployment


BREAK-GLASS PATH

shared password
administrator access

The emergency route was created for rare situations in which normal access or delivery systems could not be used.

Instead, it becomes the express lane around them.

A troublesome deployment is called an emergency.

A blocked permission is called an emergency.

An engineer who does not want to wait for the pipeline is suddenly surrounded by emergency-shaped weather.

This is why break-glass access should feel deliberately inconvenient.

Not unusable.

Not improvised.

Not so complicated that nobody can operate it under pressure.

It should introduce enough friction that using it requires a conscious declaration:

Normal controls are unavailable
or insufficient.

The service consequence justifies
exceptional human authority.

We accept that this access will
be visible, reviewed and temporary.

AWS recommends defining and testing emergency-access processes, pre-creating the necessary resources, limiting initiation to authorised users, requiring appropriate approvals, generating detailed logs and alerting whenever the emergency path is used. AWS also warns that the emergency route should not become a convenient substitute for normal operational processes.

Break-glass access should be inconvenient before activation and clear after activation.

The inconvenience protects it from casual use.

The clarity protects us when the emergency is real.

Emergency access is not ordinary elevated access

A platform may occasionally need to grant a person greater authority than they normally hold.

That does not automatically make the situation a break-glass event.

Temporary elevated access can support planned work such as:

• production maintenance
• a database migration
• investigation requiring sensitive read access
• a controlled network change
• a scheduled recovery test

The normal identity system still works.

The approval workflow still works.

The person can receive time-limited access through the standard workforce path.

Break-glass access exists for a different class of problem.

Examples include:

• the workforce identity provider is unavailable
• IAM Identity Center access is disrupted
• federation configuration has been damaged
• the ordinary administrator assignments have been removed
• a critical pipeline or platform control plane cannot operate
• an incident requires immediate action that the standard permission model cannot perform

AWS recommends designing emergency processes around specific failure modes rather than creating one universal route for anything urgent. This reduces the temptation to use emergency access for situations better handled through ordinary elevation or operational procedures.

PLANNED PRIVILEGED WORK

normal identity available
normal approval available
temporary elevated access


EMERGENCY ACCESS

normal access path failed
or cannot perform essential recovery
break-glass procedure

Urgency alone is not the test.

The test is whether the normal control system is unavailable or incapable of supporting the essential recovery action.

The friction should be at the entrance

An emergency operator may be working during:

• an outage
• a security incident
• identity-system failure
• severe time pressure

Once access has been authorised, the instructions should be concise and rehearsed.

AWS recommends detailed playbooks that can be followed quickly during stressful events. It also recommends defining preconditions, assumptions and the circumstances in which the process should not be used.

The process should therefore combine two properties:

BEFORE ACTIVATION

deliberate
approved
visible
difficult to invoke casually


AFTER ACTIVATION

documented
tested
predictable
simple to execute correctly

Making the entire procedure tortuous does not improve security.

A twenty-seven-step login ritual nobody has practised will fail when the identity platform is already unavailable.

The useful inconvenience comes from:

• requiring a declared incident
• involving another authorised person
• retrieving protected credentials or devices
• activating a prepared emergency path
• generating immediate alerts
• accepting mandatory post-use review

The emergency identity should be dormant in ordinary use, not necessarily technically disabled.

Any activation step must depend only on systems expected to remain available in the failure mode it protects against.

An approval workflow that depends on the failed identity provider is not an independent control.

The operator should not be solving riddles.

They should be crossing clearly marked control points.

The recovery path must not share every normal dependency

Suppose bfstore normally uses:

external identity provider
        │
        ▼
IAM Identity Center
        │
        ▼
AWS account roles

A break-glass design that still depends on the same external identity provider cannot recover from an outage of that provider.

It may still be useful for an IAM Identity Center disruption while the external provider remains available.

AWS documents an emergency-access design that uses direct SAML federation from an external identity provider into a dedicated emergency operations account. Operators can then assume pre-created cross-account roles in essential workload accounts. This pattern works when IAM Identity Center is unavailable but the external IdP and IAM data plane remain available.

It does not solve an external-IdP outage.

That failure needs an identity path that does not depend on the external IdP.

This means bfstore should design for named failure modes.

The direct-SAML and local-identity routes are separate recovery mechanisms.

A successful test of one does not prove that the other is usable.

IDENTITY CENTER FAILURE
EXTERNAL IdP HEALTHY

external IdP
      │
      ▼
disabled-by-default
direct SAML application
      │
      ▼
emergency account role


EXTERNAL IdP FAILURE

protected local
emergency identity
      │
      ▼
emergency account role

There is no magical emergency identity independent of everything.

The design must decide which dependencies it is meant to survive.

A dedicated emergency account keeps the path separate

AWS recommends pre-creating a dedicated emergency access account and the cross-account roles required in essential workload accounts. During normal operation, the emergency account should not be used, and any activity there should be monitored as exceptional.

For bfstore, I would introduce:

bfstore-emergency-access

Its account contract would be narrow:

account:
  name: bfstore-emergency-access
  purpose: emergency human access to essential AWS accounts

permitted:
  - emergency authentication resources
  - emergency operator roles
  - cross-account access roles
  - monitoring and audit configuration

prohibited:
  - application workloads
  - CI runners
  - platform tooling
  - general administration
  - ordinary engineering sessions
  - production data storage

The account does not need to host interesting services.

Its usefulness comes from remaining independent, quiet and prepared.

It may contain roles that can assume corresponding emergency roles in:

• bfstore-identity
• bfstore-security-tooling
• bfstore-network
• bfstore-shared-services
• bfstore-prod

Development and sandbox accounts may not need emergency access at all.

The management account is different.

It should not participate in a generic emergency-role rollout.

If organisation recovery genuinely requires management-account access, bfstore should use a separately designed EmergencyOrganisationRecovery role with:

• an exact trust relationship
• narrowly listed AWS Organizations or account-management actions
• stronger approval requirements
• independent alerts
• no ordinary workload-recovery permissions

The same broad EmergencyOperations role should not be deployed to both workload accounts and the management account.

AWS recommends first identifying the accounts where emergency operator access is genuinely essential and then defining consistent critical roles for those accounts. Its Identity Center guidance suggests beginning with distinct read-only and operations roles rather than one universal administrator role.

Break-glass should not mean unlimited administrator access

An emergency does not automatically require every AWS permission.

A failed identity-provider integration may require authority to repair:

• IAM Identity Center configuration
• SAML provider configuration
• selected IAM roles

It probably does not require permission to delete production databases.

A production-capacity incident may require:

• reading current infrastructure state
• modifying selected scaling settings
• inspecting application health

It probably does not require organisation-policy administration.

A useful role model is:

EMERGENCY OPERATOR
       │
       ├── ReadOnly
       ├── Operations
       ├── IdentityRecovery
       └── SecurityResponse

The first role entered should usually be read-only.

The operator can establish what has failed before activating greater authority.

Broad administrator access may still be necessary for a small set of failure modes.

It should be treated as a specifically justified capability, not the default meaning of break glass.

The target emergency roles should be managed centrally and protected from modification or deletion by the administrators whose normal access they are intended to recover.

A workload administrator should not be able to weaken the role trust, expand its permissions or remove the role before an incident.

Emergency roles still operate inside organisation guardrails

Naming a role EmergencyOperations does not place it outside AWS Organizations policy.

The source identity in the emergency account must be allowed to call sts:AssumeRole.

The resulting role in a workload member account must also survive every applicable Service Control Policy on that account’s path.

EMERGENCY-ACCOUNT PRINCIPAL

IAM permissions
       │
       ▼
emergency-account SCP path
       │
       ▼
permission to assume target role


TARGET EMERGENCY ROLE

role permissions
       │
       ▼
target-account SCP path
       │
       ▼
effective recovery authority

An IAM policy granting broad permissions cannot restore an action denied by an ancestor SCP.

When an inherited deny would prevent a required recovery action, the exception must be designed deliberately into the organisation policy, commonly through a narrowly scoped principal, account or tag condition.

It cannot be repaired merely by attaching AdministratorAccess to the emergency role during the incident.

The management account is the important exception because SCPs do not restrict its users or roles.

That makes a management-account emergency role more dangerous, not more convenient.

Emergency-role protection and emergency-role exemptions must be designed together.

A policy may need to prevent ordinary administrators from changing the role while still allowing that same role to perform its narrow recovery action.

Both behaviours should be tested before the incident.

Multi-person control makes activation deliberate

AWS recommends multi-person approval for highly privileged root access where practical. It gives the example of separating access to the root password and MFA so that one person cannot sign in alone. AWS also recommends protecting account-recovery channels and avoiding one person controlling both the email and telephone mechanisms used for recovery.

The same principle can shape emergency access more generally.

One person declares the incident.

Another approves emergency activation.

A security or identity custodian activates the access group or releases the protected authentication factor.

The operator then uses the access.

INCIDENT COMMANDER

declares qualifying emergency
        │
        ▼
APPROVER

authorises break-glass activation
        │
        ▼
IDENTITY CUSTODIAN

activates or releases access
        │
        ▼
EMERGENCY OPERATOR

performs recovery action

A small personal project cannot always provide four independent people.

The principle remains useful even when roles are combined.

For bfstore, deliberate inconvenience could include:

• credentials stored outside the everyday workstation
• a separate hardware MFA device
• written activation criteria
• notifying a trusted second person
• opening an incident record before use
• immediate alerts sent to an independent channel

Emergency credentials should be stored in a controlled enterprise vault whose availability does not depend on the identity provider or AWS account being recovered.

Access should require strong authentication, be individually attributable and generate audit evidence.

The goal is not corporate theatre.

It is to prevent one ordinary moment of impatience from silently becoming exceptional authority.

Local emergency identities are an exception, not a new normal

AWS recommends workforce federation and temporary credentials for normal human access.

It also recognises emergency access as one of the limited cases in which an IAM user may be appropriate when the normal identity provider cannot be reached. Such users carry long-term credentials, so their permissions and credentials require unusually careful protection.

A dedicated emergency account might therefore contain a very small number of individually attributable IAM emergency users.

Each would need:

• a strong unique password
• MFA
• no access keys unless a specific emergency requires programmatic access
• permission only to enter the emergency role path
• immediate login alerts
• periodic credential testing
• documented custodianship

AWS’s current Well-Architected emergency-access guidance lists both individual IAM emergency users and a protected emergency-account root identity as possible approaches when the normal IdP is unavailable.

Current IAM role documentation, however, says that a principal signed in as an AWS account root user cannot switch to or assume an IAM role.

Because those official sources are inconsistent, bfstore should not depend on a root-to-role chain without testing it and obtaining clarification from AWS.

The safer primary design is individually attributable emergency IAM users that have permission only to enter the pre-created emergency-role path.

Root access should remain a separately protected last-resort route for tasks that genuinely require root credentials.

The architecture should select its local identity model consciously.

It should not create one shared breakglass-admin user with a password known by half the engineering team.

Shared emergency identity may reduce credential count.

It also weakens individual attribution.

The emergency path should announce itself loudly

Normal account activity creates noise.

The emergency account should be quiet enough that any sign-in is remarkable.

AWS recommends detailed audit logging for successful and failed emergency-access attempts, immediate monitoring and alerts for activity outside an active emergency, and central retention of CloudTrail evidence.

Useful detections include:

Emergency account console login

Failed emergency login

Emergency role assumed

Cross-account emergency role assumed

Emergency IAM user changed

MFA device changed

Emergency trust policy changed

Emergency role permissions changed

Emergency account used without active incident

The alert should reach people outside the session being monitored.

Sending the alert only into a dashboard that the emergency operator is also repairing is rather too discreet.

Evidence should record:

• incident identifier
• requester
• approver
• operator
• source emergency identity
• source identity where supported
• role-session name
• target role session ARN
• activation time
• role assumed
• accounts entered
• actions performed
• deactivation time
• credential rotation
• review outcome

The role-session name should identify the operator and incident rather than use a generic label.

Break-glass access is an exception.

Its audit trail should look exceptional too.

The roles must exist before the emergency

An incident is a poor time to discover that:

• the emergency account was never created
• the role trust is wrong
• the MFA battery is dead
• the credentials are inaccessible
• the production SCP blocks the recovery role
• the runbook refers to an old account ID

AWS recommends pre-creating the emergency account, IAM identities, federation configuration, target roles and trust relationships before disruption occurs.

This avoids depending on IAM and Organizations control-plane changes, newly created trust relationships or eventual-consistency delays after the incident has already begun.

The bfstore path should be tested periodically:

DECLARE TEST INCIDENT
        │
        ▼
OBTAIN APPROVAL
        │
        ▼
ACTIVATE EMERGENCY IDENTITY
        │
        ▼
ENTER EMERGENCY ACCOUNT
        │
        ▼
ASSUME READ-ONLY ROLE
        │
        ▼
ASSUME TEST OPERATIONS ROLE
        │
        ▼
PERFORM HARMLESS RECOVERY ACTION
        │
        ▼
EXIT AND DEACTIVATE
        │
        ▼
ROTATE, REVIEW AND RECORD

The test should verify more than login.

It should prove:

• authentication works
• MFA works
• the correct role can be assumed
• the role can perform its intended task
• unrelated actions remain denied
• SCP exemptions and protections behave as intended
• alerts arrive
• CloudTrail records reach the archive
• the path can be disabled again

Each identity failure mode needs its own exercise.

DRILL A

IAM Identity Center unavailable
external IdP available
direct SAML path


DRILL B

external IdP unavailable
local emergency identity path

A successful direct-SAML drill provides no evidence that local credentials are retrievable or that their MFA devices work.

The drill should also verify session duration.

A federated emergency-account role that assumes a target-account role uses role chaining. The target role session is limited to one hour, regardless of the target role’s configured maximum session duration.

The runbook should explain how an operator reauthenticates safely when recovery lasts longer, without leaving uncontrolled overlapping sessions.

A glass case containing a key is not a recovery process.

It is interior decoration until somebody proves the key fits.

Returning to normal is part of the procedure

Emergency access should end deliberately.

Once ordinary access is restored, the organisation should:

• remove users from emergency groups
• terminate or allow emergency sessions to expire
• disable temporary federation applications where applicable
• rotate emergency credentials when required
• restore any controls altered during recovery
• review all actions
• link the evidence to the incident
• identify improvements to the ordinary platform

AWS’s emergency-access guidance recommends rotating credentials and notifying relevant teams after returning to normal operations.

The post-incident review should ask:

Why was break-glass necessary?

Did the emergency role contain
the right permissions?

Which permissions were unused?

Were any actions performed
outside the declared purpose?

Did monitoring work?

Can the normal platform be improved
so this failure is less likely?

The emergency path should not leave behind permanently elevated access.

Nor should it become the new operational workflow because it worked once.

The glass is repaired after use.

The bfstore model

For bfstore, the complete model would be:

NORMAL ACCESS

external IdP
    │
    ▼
IAM Identity Center
    │
    ▼
permission set
    │
    ▼
account role


IDENTITY CENTER FAILURE
EXTERNAL IdP HEALTHY

declared incident
    │
    ▼
multi-person activation
    │
    ▼
external IdP
    │
    ▼
direct SAML application
    │
    ▼
bfstore-emergency-access role


EXTERNAL IdP FAILURE

declared incident
    │
    ▼
multi-person activation
    │
    ▼
protected local identity
    │
    ▼
bfstore-emergency-access role


BOTH EMERGENCY PATHS

bfstore-emergency-access role
    │
    ▼
narrow target-account role
    │
    ▼
essential recovery action
    │
    ▼
logging and independent alert
    │
    ▼
deactivation and review

The account, identities, federation configuration and target roles are pre-created.

No one uses them during ordinary operation.

The access is difficult to activate accidentally.

The procedure is easy to follow once authorised.

Every use creates immediate evidence.

The authority is divided by recovery purpose.

That is deliberately inconvenient.

Questions I now ask

Failure mode

Which normal access dependency has failed?

Necessity

Why can ordinary temporary elevation not solve this?

Scope

Which essential accounts require emergency access?

Role

Is read-only access sufficient?

Independence

Does the recovery path depend on the failed system?

Activation

Who declares and approves the emergency?

Authentication

How are emergency credentials and MFA protected?

Detection

Who is alerted when the path is used?

Evidence

Can every action be linked to the incident and operator?

Recovery

Has the path been exercised before the emergency?

Closure

How is access removed and credential state restored?

Learning

What normal control should improve after the incident?

These questions prevent break glass from becoming a theatrical name for administrator shortcut.

The mental model I am keeping

My earlier model was:

BREAK-GLASS ACCESS

powerful credentials
stored somewhere safe
for a bad day

The stronger model is:

                        QUALIFYING FAILURE
                                │
                                ▼
                         INCIDENT DECLARED
                                │
                                ▼
                     DELIBERATE AUTHORISATION
                                │
                                ▼
                 FAILURE-SPECIFIC IDENTITY PATH
                                │
                                ▼
                    NARROW RECOVERY ROLE
                                │
                                ▼
                      ESSENTIAL ACTION ONLY
                                │
                                ▼
                 IMMEDIATE LOGGING AND ALERTING
                                │
                                ▼
                    RETURN TO NORMAL ACCESS
                                │
                                ▼
                     ROTATION AND REVIEW

Normal access supports routine work.

Temporary elevation supports planned sensitive work.

Break-glass access restores essential authority when the normal path fails.

Rehearsal proves that the alternate path is independent and usable.

Break-glass access is sometimes described as a back door.

That description is uncomfortable because back doors tend to receive less attention than front doors.

A proper emergency route should be designed more carefully than ordinary access, not less.

It should require a conscious decision to enter, provide only the authority required for recovery and leave enough noise behind that nobody can pretend the door opened by itself.

Emergency credentials should be stored in a controlled vault whose availability does not depend on the failed identity system or AWS account.

The role should not appear in the ordinary access portal.

The account should not host convenient tooling.

The procedure should not begin with “Mariam normally remembers how this works.”

It should feel unusual because its use is unusual.

It should feel serious because its authority is serious.

And once the emergency has been declared, the runbook should be calm, tested and almost disappointingly clear.

References and further reading

1. AWS Well-Architected: Establish an emergency access process
2. IAM Identity Center: Set up emergency access to the AWS Management Console
3. IAM Identity Center: Plan your emergency access model
4. IAM Identity Center: Emergency access configuration
5. IAM Identity Center: Emergency preparation tasks
6. IAM Identity Center: Return to normal operations
7. AWS Organizations: Service Control Policies
8. AWS Organizations: Best practices for the management account
9. AWS IAM: Switch to or assume an IAM role
10. AWS IAM: Root user best practices
11. AWS IAM: Create an IAM user