What can you learn from an AWS environment audit? Checklist and how to proceed with improvements.

If you haven't reviewed your configuration and settings since implementing AWS, you may have unnecessary costs, inadequate permission settings, or configurations that are difficult to recover from in the event of a failure. On the other hand, it can be unclear what "AWS environment assessment" covers and how it differs from vulnerability assessments and security configuration assessments.

This article outlines the items to check during an AWS environment assessment, the implementation process, and the criteria for deciding whether to outsource the assessment. It also explains how to prioritize the issues found during the assessment and use that information to drive configuration changes and operational improvements.

What is an AWS environment audit?

AWS environment assessment is a general term for services and initiatives that inspect the configuration, security, cost, availability, and operational status of an AWS environment and identify areas for improvement. The scope of the assessment varies depending on the provider and purpose.

A configuration that was appropriate at the time of implementation may no longer suit the current situation due to changes in usage scale, system requirements, personnel changes, or organizational structure. In some cases, unused resources, unnecessarily broad permissions, and outdated recovery procedures may remain.

AWS environment diagnostics go beyond individual settings to identify issues across the entire system. Based on the diagnostic results, we guide you through configuration changes, system review, and the development of operational rules.

Differences between vulnerability assessment and security configuration assessment

The scope of checks differs between AWS environment assessments, vulnerability assessments, and security configuration assessments.

Security configuration assessments and vulnerability assessments are methods for evaluating the security of an AWS environment. On the other hand, AWS environment assessments may also include aspects such as availability, cost, monitoring, backup, and operational structure.

Since the scope of the diagnosis varies depending on the service, when requesting services from an external provider, be sure to check the specific diagnostic items and deliverables, rather than just the name of the service.

AWS employs a shared responsibility model, where AWS is responsible for the security of the cloud infrastructure. Users manage their data, access permissions, operating systems, applications, network settings, etc., depending on the services they use. AWS environment diagnostics primarily examine this user-side management scope.

Relationship with AWS Well-Architected Framework

The AWS Well-Architected Framework is used as a standard when evaluating AWS environments.

The AWS Well-Architected Framework is a set of principles and best practices for designing and operating systems on AWS, and is comprised of the following six pillars:

• Operational excellence

• Security

• Reliability

• Performance efficiency

• Cost Optimization

• sustainability

This framework allows you to evaluate your current AWS environment from multiple perspectives without being biased towards any particular setting. In the actual diagnostic, you select the necessary improvements based on the system's importance, budget, usage scale, and internal operational structure. 

When to consider an AWS environment audit

AWS environment audits aren't just for after major outages or security incidents. Reviewing them whenever usage patterns or operational structures change allows you to identify configuration errors and cost increases early on.

The following situations are indicators that may warrant considering a diagnosis:

• Since adopting AWS, the configuration and settings have not been reviewed for a long time.

• We don't know the reason why our AWS usage costs are increasing.

• The employees responsible for design and operation have been transferred or have left the company.

• I have concerns about how to recover from a failure and about backups.

• I would like to reconfirm the permission settings and the scope of external access.

• We are planning to expand, integrate, and migrate the system.

• It is necessary to comply with audits and internal security standards.

In particular, we want to review cases where the importance of the system has increased since its initial implementation. An environment that started as a small-scale business system may expand into a foundation supporting the business after a few years. In accordance with the change in role, the levels of availability, backup, monitoring, and access control also need to be adjusted.

If your AWS usage is increasing, before reducing resources, check the impact on performance and availability. Then, revise your configuration to match your current usage.

Items to check during AWS environment diagnostics

The scope of the diagnostics will vary depending on the size and challenges of your AWS environment. Here, we will cover four aspects to consider when reviewing your entire environment.

Configuration/availability

If the resources supporting a system depend on a single Availability Zone or a single resource, the entire service may come to a halt in the event of a failure.

The configuration and availability assessment checks the redundancy status, failover methods in the event of a failure, handling of increased load, and recovery procedures.

Specifically, the following points are covered:

• Dependence on a single Availability Zone or resource

• Switching method in case of failure

• A configuration that allows resources to be scaled according to the load.

• Choosing AWS services based on your intended use

• Recovery Time Objective (RTO) and Recovery Point Objective (RPO) 

The configuration and availability assessment involves analyzing the impact of system downtime on business operations and confirming the balance between necessary availability and cost.

The required configuration differs between a temporary internal testing environment and a production system used continuously by customers. The necessary level of availability is determined according to the criticality of the system. 

Security and Permissions

In an AWS environment, as permissions are granted in response to the addition of personnel or system changes, access permissions that do not match the current role may remain.

The diagnostic checks IAM users and roles, network exposure, encryption, and logging. This includes checking accounts of former employees, unused access keys, excessive administrator privileges, and unintentionally exposed Amazon S3, databases, and management ports.

We also check whether the system can track the history of configuration changes and whether there are established contact points and response procedures in place for when security events are detected.

We will establish a system that allows for regular review of permissions and access settings in response to changes in personnel and roles.

Costs and Resources

AWS usage fees increase not only due to business growth and increased usage, but also due to unnecessary resources and excessive configuration.

For example, this could include situations where storage or snapshots remain on a stopped system, instances are being used that are larger than the actual load, or a test environment is running continuously.

A cost and resource assessment compares billing amounts with actual usage to determine if the current configuration is appropriate for current needs. This review covers unused resources, instance and storage sizes, uptime, purchasing options, and departmental/system-specific cost management.

If your usage of computing services like Amazon EC2 is stable over the long term, then pricing models such as Savings Plans are a viable option. In environments where usage and configuration fluctuate, the decision should be based on the contract period and future change plans.

Cost optimization involves reducing unnecessary spending while maintaining business requirements, and while monitoring the impact on performance and availability.

Monitoring, backup, and operation

It is only after a failure occurs that it becomes apparent that there are insufficient monitoring items, that the system cannot be restored from backups, or that there is no one in charge who knows how to handle the situation.

The diagnostic process involves not only preventing failures but also verifying whether a system is in place to detect anomalies early and restore them within the necessary timeframe. It checks whether monitoring targets and notification recipients have been updated to reflect changes in the system configuration, whether monitoring settings have been applied to new resources, and whether the person receiving the notification can determine the appropriate next steps.

The backup process involves checking the target data, frequency of backups, retention period, and storage location. Regularly perform restore tests to ensure that necessary data can be recovered within the target recovery time.

If operations depend on the experience of a specific person, update the configuration diagrams and procedures to create a system that can be handled by multiple people.

How to proceed with an AWS environment assessment

Simply checking settings without defining objectives during an AWS environment assessment will not lead to actionable improvements. First, define the scope, evaluate the issues found, and improve those with the highest priority.

Organize the scope of diagnosis and the necessary information.

First, clarify the purpose of the assessment. The scope of the check will vary depending on the purpose, such as reviewing costs, strengthening security, or addressing system failures.

Next, you determine the target AWS account, region, system, and resources. If you are operating multiple systems, you can also proceed in stages, starting with the environment that has the greatest impact on your business.

Before making a diagnosis, please organize the following information.

• AWS account and organizational structure

• System diagram

• AWS services currently in use

• AWS usage fees and their breakdown

• Monitoring, backup, and incident response status

• Access permissions and administrators

• System availability and recovery time

• Current identified cost, security, and operational challenges

If documentation is not available, we will begin the diagnosis by visualizing the current situation. Sometimes, the fact that the configuration diagrams or operating procedures have not been updated can itself be identified as an area for improvement.

The time required for the diagnosis will vary depending on the number of accounts and systems involved, the scope of the investigation, and the availability of documentation. When outsourcing the work, it is helpful to confirm the schedule, including preliminary interviews, analysis, and reporting sessions, in addition to the investigation period, to facilitate internal coordination.

Analyze the issues and prioritize them.

The issues identified during the diagnosis will be prioritized based on their impact and urgency.

Settings that could lead to information leaks or unauthorized access, configurations that would have a significant impact on business operations in the event of a failure, and items with defined deadlines for resolution will be given high priority. The effort required for improvement and dependencies on other measures will also be considered.

For example, we will promptly address issues such as unintended external exposure and excessive administrator privileges. We will gradually review underutilized instances and update operational procedures while assessing the impact on business operations.

The diagnostic results will include the following information necessary for the improvement plan:

• Reasons why the problem is occurring

• Impact on business and systems

• Recommended actions

• Priority of response

• Required man-hours and costs

• Impact of changing settings

By categorizing not only the technical severity but also the impact on the business and feasibility, it becomes easier to begin implementing improvements.

Implement configuration changes and operational improvements.

Once priorities are determined, we move on to reviewing the settings and configuration.

Depending on the issue, we will determine the appropriate course of action, such as clearing out unnecessary resources, optimizing instances, changing IAM policies, reviewing network exposure scope, and redesigning backup configurations.

Before making changes to the production environment, confirm the scope of impact and test in a test environment as needed. Additionally, having change procedures and rollback methods prepared will make it easier to handle unexpected impacts.

To prevent the same problem from recurring, we will review not only the settings but also the operational rules. We will establish procedures for granting and revoking permissions, verification items when creating resources, and communication channels in the event of a failure.

After the improvements are made, we verify that the changes have been reflected as intended. By determining the date and person responsible for the next inspection, we can use the diagnostic results to drive continuous improvement.

Criteria for deciding whether to outsource an AWS environment assessment.

By utilizing the various services and check functions provided by AWS, you can also verify some items yourself.

On the other hand, if you need to evaluate across multiple accounts or systems, or if changes after the diagnosis require expert judgment, you can utilize external support companies.

Cases where diagnosis is difficult to perform solely within the company.

In the following situations, you should consider outsourcing to an external specialist company:

• There is no one in charge who has an overview of the entire AWS environment.

• We operate multiple AWS accounts and systems.

• I want to evaluate security, cost, availability, and operations across all aspects.

• It is difficult to determine whether the current configuration meets the business requirements.

• We want to make improvements while minimizing the impact on the production environment.

• Third-party evaluation is necessary for audits and compliance with internal standards.

• I want to translate the diagnostic results into an improvement plan.

When managing AWS in-house on a daily basis, decisions are often based on existing designs, and opportunities for improvement through alternative configurations may be overlooked. Involving external experts can help identify issues and suggest improvements that might be difficult to notice internally.

On the other hand, if the scope of the diagnosis is limited and the items to be checked and the countermeasures are clear, it is possible to proceed in-house. The decision should be made based on the scale of the environment, the company's knowledge, the time that the person in charge can allocate, and the difficulty of the change work.

The cost of external diagnostics varies depending on the number of accounts, target systems, diagnostic items, report content, and scope of improvement support. When comparing quotes, be sure to check not only the price, but also the scope of the investigation, deliverables, and post-diagnosis support.

Choose a company that can provide support from diagnosis to improvement.

When choosing a support company, make sure they can translate the diagnostic results into a concrete improvement plan and provide support all the way through to implementation.

The following points will be used as criteria for making a decision:

• Can we evaluate across the entire AWS environment?

• Are the diagnostic criteria and confirmation methods clear?

• Can you explain the cause of the problem and its impact on the business?

• Can you provide a priority list for the response?

• Can they provide support for setting changes and configuration changes?

• Can we consider the impact on the production environment and the migration procedure?

• Can I consult with you about monitoring and operation after the improvements are made?

Simply applying a general recommended configuration may increase costs and operational burden. This approach is best suited for companies that can select the necessary measures based on their business requirements, internal structure, and budget.

Serverworks and IIJ Group support the diagnosis and improvement of AWS environments.

Serverworks and the IIJ Group provide support for AWS environments, from diagnosis and improvement to operation.

Serverworks will prioritize the challenges based on the AWS Well-Architected Framework and be responsible for reviewing the configuration and permissions, optimizing costs, and improving monitoring and backup. The IIJ Group will support post-implementation operations, taking into account the IT environment, including the network and overseas locations.

By collaborating, our two companies can comprehensively review your AWS environment, network, and operational structure. Please contact us if you would like to identify challenges in your AWS environment or if you would like to proceed with improvements after the diagnosis.