Recent studies show that 62% of organizations have experienced DevOps pipeline breaches due to inadequate security measures.

Securing DevOps pipelines is essential to protect your software delivery process from threats and vulnerabilities. Pipelines act as the backbone of continuous integration and delivery (CI/CD), handling sensitive information like source code, credentials, and access tokens. Without robust security measures, these pipelines become attractive targets for attackers seeking to inject malicious code, steal intellectual property, or disrupt services.

By embedding security into every stage—from code commits to cloud deployment—you mitigate risks like data breaches, unauthorized access, and compliance violations. A secure pipeline ensures your software is not only delivered quickly but also remains reliable, confidential, and trustworthy for end users.

Understanding consequences of unsecured DevOps

Unsecured DevOps pipelines pose significant risks that can compromise the integrity, confidentiality, and availability of your systems. Some key risks include Data Leaks, Unauthorized Access to Pipeline and Artifacts, Supply Chain Attacks etc.

Data Leaks

• Sensitive Information Exposure: Pipelines often handle credentials, API keys, and private configurations. Without proper encryption or restricted access, these can be leaked, leading to unauthorized access.
• Source Code Theft: Exposed repositories can allow attackers to access proprietary code, intellectual property, or customer data.
• E.g. If you are using Default agent machines provided by different DevOps tools like GitHub, Azure then they are more likely to result in Data leaks and other issues. Because these agents are globally distributed and shared across multiple customers/applications, using these agents is Highest Risk.

Unauthorized Access

• Compromised Build Systems: Attackers may exploit weak authentication or unpatched vulnerabilities to gain access to CI/CD systems.
• Privilege Escalation: Misconfigured roles or permissions may allow unauthorized users to execute actions beyond their intended scope.

E.g. Most of the time we do not treat our self hosted agents as our data storage which results in weak access mechanisms and open surface for cyber attacks. While Designing the DevOps pipelines we should make sure that we consider the Pipelines as part of our application, i.e. its resources should be accessible within the same restricted area as your application and not outside.

Supply Chain Attacks

• Injection of Malicious Code: Attackers can compromise build artifacts, inject malicious dependencies, or tamper with code during deployment.
• Downstream Impact: Infected pipelines can deploy compromised applications to production, affecting users and linked systems.

By addressing these risks through robust security practices, such as securing credentials, implementing least privilege, and using trusted tools, you can ensure that your DevOps pipelines remain resilient and trustworthy.

What should we do ?

Setup Self Hosted Agent

To avoid these issues with Default Agent Machines, you can use self hosted agents. Most of the DevOps services provide the configuration to add Self Hosted Runners/Agents to the Pipelines. 

E.g. Setup a Self Hosted Agent in your environment and use these self hosted agents instead of Public Agents. Here are some references that you can use;

• Github : https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/adding-self-hosted-runners
• Azure DevOps: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=yaml%2Cbrowser#self-hosted-agents

This will help you to run the build & deployment activities from the secure VM’s in your environment instead of Public Agents. This way your data/application/secrets never leave your environment and keep safe on these self hosted agents.

Setup Private Virtual Networks for your Self Hosted Agents

Let me Help you with the Example here;

Figure 2.1 Sample Private Virtual Network For Application with DevOps Agents

In the diagram figure 2.1 above you can see that self hosted agents are deployed in the same Vnet as other resources to avoid any communication over the Public internet. Basically

There are options where you can choose to set up a separate Private Vnet for self hosted agents and peer the network with your application network for access based on the Organizational size and policies. Where only limited traffic is allowed between the peered network. 

Again, the purpose here is to keep self hosted agents inside the private networks and use these agents in your pipelines instead of global agents.

Setup Jumpbox to access private networks

When we set up the private networks, there are ways to access these networks like through jumpbox (refer figure 2.2). Jumpbox is the machine through which you can expose an endpoint to connect to these networks for troubleshooting purposes. In cloud there are also services like in Azure we have Bastion Host service for the same purpose. 

Setup Network Firewall Rules or Network Security Groups

You can connect to this machine with your user identities with limited access and then you have access to the resources inside the network. We can configure the Firewalls and Network Security groups for securing these jumpbox and hardoning of the network.

Figure 2.3 Sample Network Security Rules for Self Hosted Agent

In the example above figure 2.3, we have a network security group created to allow only SSH connection to the jumpbox, that too specifies inbound IP’s which are organizational IP’s. That means, you can connect to these machines only through your organizational networks and no public access is granted to these machines. 

Setup Private Repositories or Private Container Registries to store pipeline artifacts

Supply chain attacks pose a significant risk to DevOps pipelines. Protecting pipeline artifacts, such as OS images and application containers, is critical to ensuring the security of your deployment process.

Steps to Set Up Private Repositories or Container Registries

1. Choose a Cloud Service:
   Select a private repository or registry service. Examples:
   • Azure Container Registry (ACR)
   • Amazon Elastic Container Registry (ECR)
   • Google Container Registry (GCR)
2. Store Artifacts Securely:
   Store your OS images, Docker container images, and other build artifacts in these registries.
3. Enable Private Endpoints:
   Configure the service to use private endpoints to ensure communication occurs over your virtual network (VNet) and not the public internet.
4. Integrate with CI/CD Pipeline:
   Update your CI/CD pipeline to authenticate and interact with the private registry.

Setup credentials and sensitive information as Secrets

Setup environment variables or critical credentials as Secrets and allow access to them only through pipelines with user defined identities or system managed identities. These identities should have limited access to read the repo code or other artifacts that we access in the pipelines.

E.g. Nowadays most of the solutions like Github, Azure, AWS  provide options to set up environment variables and secrets. 

• Github Secrets: https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions
• Azure Devops Secrets: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/set-secret-variables?view=azure-devops&tabs=yaml%2Cbash

The major difference between environment variables and secrets is that variables are not masked and printable in pipelines while the secrets are masked and not printable in the pipelines. 

Though there are ways to print secrets, which I will say are bad coding practices.

We hope this blog provides a clear understanding of the essential secure practices that can be adopted in your projects. Security is a shared responsibility, and we invite you to share your experiences, insights, or suggestions on this topic. Together, we can learn, innovate, and strengthen our ability to deliver a secure and trustworthy environment for our customers and stakeholders.

In today’s competitive software landscape, SaaS providers often face the challenge of delivering personalized data visualizations to multiple customers. This need is particularly pressing for multi-tenant SaaS products where different customers, or tenants, require isolated access to dashboards tailored to their data.  Amazon QuickSight, a fully managed service from AWS, offers a robust solution for building and embedding dashboards that align perfectly with the multitenancy architecture of modern SaaS products.

Multitenancy in QuickSight enables organizations to efficiently manage and segregate dashboards, data sources, datasets, and user permissions for different tenants, all within a single QuickSight account. This approach simplifies operations while ensuring compliance with data privacy and security standards.

This blog will provide insights into implementing multitenancy in QuickSight, leveraging namespaces for isolating users and resources, configuring access control, and best practices for tenant isolation. The blog also discusses strategies for sharing dashboards across namespaces, dynamically embedding dashboards in client applications, and handling complex setups where namespace-specific administrators manage their own users and resources.

Leveraging Namespaces for Multi-Tenancy

QuickSight provides namespaces as a powerful feature to implement multi-tenancy, allowing organizations to segregate users and resources across different tenants. A namespace is essentially a container within QuickSight that isolates users, groups, and resources, ensuring that each tenant operates independently within the same QuickSight account.

For organizations setting up tenants, the ‘default’ namespace can be reserved for internal use, such as managing resources for subsidiaries or internal teams. Additional namespaces can then be created for each tenant, enabling

• Data Isolation: Ensuring that data and resources for one tenant are not accessible to others.
• Custom User Management: Defining specific users and groups for each namespace, aligning with tenant-specific roles and permissions.

Creating all users across tenants in the same default namespace in Amazon QuickSight can lead to several challenges. 

1. Lack of Tenant Isolation : Without namespaces, users across tenants share the same resource space. Misconfigurations in access control (e.g., incorrect permissions on shared folders or row-level security rules) could lead to users inadvertently accessing dashboards, datasets, or analyses that belong to other tenants.
2. Complex Access Control Configuration : One would rely heavily on RLS to ensure users see only their data in shared dashboards or datasets. This increases the complexity of dataset configurations and requires constant maintenance to prevent errors.
3. Scalability Issues : As the number of tenants grows, managing shared dashboards, datasets, and analyses for all users in a single namespace becomes unmanageable
4. Performance Bottlenecks : A single namespace means all users query shared datasets. Heavy usage by one tenant could degrade performance for others, leading to contention and slower query times.
5. High Risk of Admin Errors : Misconfiguring a single shared resource or folder can impact multiple tenants, resulting in widespread access issues or data exposure.

Provisioning Users within the Namespaces

Provisioning users in namespaces requires using the AWS CLI or SDK to programmatically create and manage users. Administrators can create users via the console for only the default namespace. For any other custom created namespaces, the users can only be created and provisioned access via the API. This process ensures that users added to a specific namespace cannot view or access resources outside their assigned namespace and hence provides complete isolation and control over tenant-specific configurations.

For a straightforward multitenancy setup, where users are only required to view QuickSight dashboards, the administrator of the default namespace can directly provision these users. However, in a more complex scenario where control needs to be delegated to a superuser within a specific namespace, the administrator can provision a user with an ‘admin’ role for that namespace. This superuser can then manage the namespace independently, provisioning additional users with roles such as ‘author’ or ‘reader.’ This approach empowers tenants with greater autonomy while reducing reliance on the default namespace administrator.

Resource Management

Creating Data Sources for Each Tenant and Sharing with the Namespace

Creating a dedicated data source for each tenant provides clear separation of data and preventing accidental data leaks or unauthorized access. This has 2 major benefits 

1. Custom Configuration : This allows for Data sources to be customized for tenant-specific requirements, such as applying connection settings, query optimizations, or filters. 
2. Operational Independence: Updates or maintenance on one tenant’s data source do not impact others, allowing for better scalability.

To share the data source with a namespace, you can use QuickSight APIs or CLI commands to assign the data source permissions to users or groups within the namespace.

Publishing a Dashboard Across Namespaces

Frequently, an admin may create a standard dashboard in the default namespace and need to share it with users in a custom namespace. This scenario is particularly useful when you want to provide a unified experience for tenants while maintaining data isolation. Steps involved in this process are 

Replicate assets : Replicate the datasets, analysis across the namespace

• Duplicate an existing dataset using the API & modify the data source ARN in the dataset configuration to point to the new tenant’s data source.
• Duplicate the analysis & modify the analysis definition to reference the new dataset created for the tenant.
• Publish the dashboard copy to the namespace updating the dashboard definition to use the new tenant-specific analysis or dataset.

Grant Access: Assign permissions to groups or users in the custom namespace to view or edit the dashboard.

Embedding QuickSight Dashboards in Client Applications

Finally, let’s talk about embedding Amazon QuickSight dashboards into client applications while maintaining strict access control. The process involves dynamically generating embed URLs based on user authentication and authorization within the application. This ensures that only authorized users can access the appropriate dashboards.

When a user requests access to a dashboard, the application sends a request to an AWS Lambda function, providing the namespace, user ID, and dashboard ID as inputs. The Lambda function verifies the user’s existence within the specified namespace and, upon successful verification, generates an embedded URL tailored to the namespace-specific configuration. This URL is then returned to the application, allowing the user to securely access the dashboard. 

The Lambda function operates under an IAM policy designed to enforce namespace-level security. 

{
  "Effect": "Allow",
  "Action": "quicksight:GenerateEmbedUrlForRegisteredUser",
  "Resource": [
    "arn:aws:quicksight:region:account-id:namespace/tenant_namespace_1"
  ]
}

It ensures that only users belonging to the specified namespace are permitted to generate embedded URLs.

For every new tenant, a corresponding custom namespace is created, and a new IAM policy is associated with the Lambda function, granting access to users from the new namespace. This setup eliminates the need to replicate the Lambda function for each tenant while maintaining a scalable and secure embedding solution for multitenant applications.

SPICE Management 

In a multi-tenant environment with AWS QuickSight, SPICE memory management primarily operates within the context of each individual tenant’s namespace. Each tenant’s namespace essentially has its own dedicated pool of SPICE capacity. This means that the memory consumed by one tenant’s datasets does not directly impact other tenants. Additionally, the amount of SPICE capacity available to each tenant can be managed and controlled by the administrator.

Administrators can monitor SPICE capacity usage for each tenant within their account. This helps identify potential issues and proactively manage resource allocation. If a tenant requires more SPICE capacity, administrators can increase the allocated capacity. Conversely, unused capacity can be released to optimize costs.

While each tenant’s data is logically isolated within its namespace, the underlying hardware infrastructure for SPICE is shared across all tenants within the same AWS region.

Advanced Multitenancy setup 

In a more advanced multitenancy setup, the primary admin of a QuickSight account can provision tenant-specific admins within custom namespaces. These tenant admins, in turn, are empowered to manage users (authors and readers) within their respective namespaces and create custom dashboards tailored to their tenant’s needs. This decentralized administration model fosters scalability, autonomy, and flexibility, especially in SaaS environments or enterprises managing multiple tenants. Here the Role Hierarchy and Responsibilities are as follows –

Primary Admin: The primary admin operates within the default namespace and holds overarching responsibilities for managing all namespaces. This includes provisioning tenant namespaces, assigning tenant admins, and setting up initial shared resources such as data source templates and dashboard layouts

Tenant Admins: Each tenant namespace is managed by a tenant-specific admin. These admins have the autonomy to create and manage users—both authors and readers—within their namespace. They are responsible for ensuring that tenant-specific data sources and permissions are configured correctly and that the dashboards created within their namespace meet the tenant’s requirements.

Authors: Within their namespace, the authors collaborate with tenant admins to build and refine dashboards that provide actionable insights tailored to the tenant’s business context.

For example, when a new tenant is onboarded, the primary admin creates the namespace and assigns an admin to it. The tenant admin then provisions users, organizes them into groups for streamlined access management, and ensures that the necessary data sources and templates are available. Authors within the namespace can then create custom dashboards, while readers access these dashboards as required.

This model allows tenant admins to create and distribute dashboards independently of the primary admin, reducing bottlenecks and enabling faster response times to tenant-specific analytics needs

Choosing the right Pricing Model

AWS QuickSight offers two primary pricing models: User-Based and Capacity-Based. User based pricing model is simple where one pays a fixed fee per user per month and is suitable for organisations with a stable and predictable number of users. On the other hand, Capacity based pricing is Ideal for organizations with fluctuating user numbers or varying usage patterns.

Pricing models are applied at the namespace level. This gives you the flexibility to tailor your pricing strategy to the specific needs and usage patterns of each tenant or department within your organization.

For eg. You can have one namespace operate on a User-Based model, where you pay a fixed fee per user. This is suitable for departments with a stable user base and consistent usage.

Simultaneously, another namespace can utilize the Capacity-Based model, paying only for the resources consumed (e.g., reader sessions). This is ideal for departments with fluctuating user numbers or high usage variability.

Understanding QuickSight Costs by Namespace

AWS QuickSight allows you to track and allocate costs at the namespace level. This provides valuable insights into the cost consumption of each tenant or department within your organization.

Here we use AWS Cost Explorer for understanding costs by namespace. By applying cost allocation tags such as “Namespace: Sales” or “Namespace: Marketing” to your QuickSight resources, you can effectively filter and group costs within Cost Explorer. This allows you to visualize and analyze the cost consumption of each namespace, identify cost drivers, and make informed decisions about resource allocation and pricing strategies.

For example, you can easily track the total cost associated with the “Sales” namespace by filtering your cost data based on the “Namespace: Sales” tag. This provides a clear picture of the costs incurred for all users and resources within that specific namespace.

Summary

This blog aimed to provide a comprehensive understanding of the journey when onboarding Amazon QuickSight as an analytics solution, especially for multitenant environments. QuickSight offers robust capabilities to streamline tenant onboarding, manage resources, and ensure secure, scalable analytics. By replacing custom-built solutions with QuickSight’s offerings, organizations can significantly reduce operational complexity while delivering powerful, tailored BI experiences. From leveraging namespaces to implementing dynamic embedding and access control, QuickSight equips businesses with the tools needed to meet diverse tenant requirements. As you transition to QuickSight, this guide highlights the critical steps and considerations to ensure a seamless and efficient integration into your analytics ecosystem.

In today’s fast-paced digital landscape, seamless workflows are essential for maintaining operational efficiency. One critical aspect of many workflows is the approval process, where decisions need to be made quickly and securely. Email-based approvals have become a preferred solution due to their accessibility and ease of use.

But how do you ensure that this method is not only user-friendly but also secure? Enter the concept of email-based approval workflows using one-time links. This approach bridges the gap between convenience and security, empowering users to approve or reject tasks directly from their inbox without compromising sensitive data.

In this post, we’ll explore how email approval workflows work, the role of one-time links in ensuring security, and best practices for implementing this feature in your systems.

Need for a Token in One-Time Link Generation

A key element of implementing one-time approval links is the use of a unique token. This token serves as a secure identifier that ensures the authenticity of the approval action and ties it to a specific leave request. Without a token, the link would be a simple URL, which could be easily guessed or reused, undermining its security and purpose.

Here’s some key aspects essential for one-time link generation –

• Uniqueness : Each token is randomly generated and unique to the leave request, ensuring that no two links are the same. This uniqueness prevents unauthorized users from tampering with the link or attempting to generate a valid approval link on their own. 
• Security : The token serves as a form of authentication, guaranteeing that only the correct user (the manager or approver) can interact with the link.
• Non-Reusability : A well-designed token is single-use, meaning once the link has been clicked and the approval action is completed, the token expires. This prevents any possibility of reusing the same link, providing an added layer of security. Even if someone gains access to the link after it has been used, it will no longer be valid, ensuring that approvals or rejections cannot be mistakenly repeated.

Generating a Secure Token

A UUID (Universally Unique Identifier) is a simple yet powerful way to achieve this. A UUID is a 128-bit identifier designed to be globally unique. It ensures that no two tokens are ever the same, making it ideal for one-time links in approval workflows. UUIDs can be generated programmatically using most modern programming languages and libraries.

The UUID, along with relevant metadata, is stored in the database. The associated metadata includes:

• Request ID: The unique identifier for the leave request.
• Token Creation Timestamp: The time when the token was generated.
• Token Usage Status: Indicates whether the token is valid, has been used, or has expired.

The token is designed to be time-sensitive, with a validity period (e.g., 2 days). This gives the approver enough time to review and approve the leave request while ensuring that the token expires after the defined period to prevent misuse.

When the approver clicks the one-time link, the system performs a series of checks 

1. Validate the Token’s Existence: The system first checks if the token exists in the database.
2. Check Token Association: The system ensures that the token is linked to the specific request.
3. Verify Token Validity Period: The token’s creation timestamp is compared to the current time to confirm that it is still within the validity period.
4. Ensure the Token Has Not Been Used: Finally, the system checks if the token has already been used. If it has, the link is considered expired and no further action is allowed.

Only when all of these checks are successfully passed, the approval action is executed.

Adding Login-Based Verification for Secure Email Approvals

Token-based links alone may be insufficient in systems where strict access control is necessary. If the email link is accessed by an unauthorized user, login-based verification ensures that approvals are granted only after confirming the identity of the approver.

How It Works

1. Token Verification: The user clicks the one-time link, which routes them to the system (e.g., https://example.com/approve?token=<UUID>).
2. Authentication Prompt: Before processing the approval, the user is redirected to the system’s login page.
3. Login Validation: The system authenticates the user’s credentials (username, password, and potentially additional security checks like 2FA).
4. Approval Grant: Upon successful login, the system validates the token, associates it with the logged-in user, and processes the approval request.

The system leverages the existing login mechanism that users already know and trust, eliminating the need for new MFA tools. Additionally, it maintains detailed records of who approved the request and when, even through this token-based approach, ensuring accountability and traceability for every action.

This balance of security, usability, and traceability makes login-based verification an ideal choice for workflows where approval integrity is critical.

Notifications for Approval Actions

In any approval workflow, notifications serve as a direct acknowledgment of the user’s action while also acting as a safeguard in case of unauthorized activity. They –

• Keeps the approver informed that their action has been successfully recorded.
• Provides a record of actions taken, helping users and administrators track approvals.
• Alerts users immediately if an approval they didn’t authorize occurs, allowing them to report suspicious activity.

When sending notifications for approval actions, it’s essential to include key details that provide clarity and context. Each notification should specify the request ID, a brief description of the action, the date and time of the approval, and the identity of the approver. Additionally, include a link to review the request or take further action if needed. 

For enhanced security, the notification should also prompt users to report unauthorized activity, such as: “If this approval was not made by you, please contact support immediately.” These elements ensure the notification is informative, actionable, and serves as both a confirmation and a security safeguard.

Conclusion: Technical Design Decisions and Impact

The implementation of one-time approval links using UUID tokens in the approval system reflects a well-considered balance of simplicity, security, and efficiency, while also addressing potential challenges. 

UUIDs for Simplicity and Security.
The decision to use UUIDs is driven by the need for a secure yet simple solution. This approach strikes a balance between maintaining security and keeping the system simple, without the complexity of additional systems for token generation or storage. 

Notification for Unauthorized Approvals
By incorporating notifications for approval actions, workflows become more transparent, users stay informed, and the system achieves a higher standard of accountability and security.

Auditability and Transparency
Finally, the design emphasizes transparency and accountability. Every action taken via the one-time approval link is logged, ensuring that the system remains auditable and that stakeholders can track approval history, providing full visibility into the decision-making process.

In today’s cloud-first era, AWS has become the backbone of countless applications and enterprises, offering unparalleled scalability, reliability, and flexibility. Its pay-as-you-go pricing model is revolutionary, allowing businesses to only pay for what they use. However, this flexibility can also lead to unexpected surprises on the billing dashboard if resources are not carefully managed.

As the cloud infrastructure grows and becomes more complex, here are some of the common hurdles that businesses encounter

• Lack of Visibility and Control  – With AWS’s vast range of services, tracking usage and spending across regions and resources can be daunting, making it challenging to identify cost drivers.
• Unpredictable and Spiky Costs – AWS services like EC2 and Lambda can face sudden usage spikes from traffic surges, misconfigurations, or accidental provisioning, leading to costly overruns if not properly monitored.
• Difficulty in Forecasting Costs – The complexity of AWS pricing models combined with the dynamic nature of cloud environments, makes it difficult for organizations to accurately forecast costs.

To help organizations address these challenges, AWS offers a range of powerful tools and services that include –

• AWS Pricing Calculator
• AWS Budgets
• AWS Cost Anomaly Detection
• AWS Cost Explorer

In this blog, we’ll explore how these AWS services can help businesses optimise their cloud spend, offering practical guidance on how to use them to manage, control, and forecast costs effectively.

Evaluating Service-Specific Costs

One of the most effective tools for managing costs before diving into AWS implementation is the AWS Pricing Calculator. This tool allows businesses to estimate the costs of AWS services in advance, helping to align cloud expenses with budgetary constraints and project requirements. The AWS Pricing Calculator is an interactive web tool designed to provide detailed cost estimates for AWS services. It’s particularly useful for planning new implementations, comparing service costs, or assessing the financial impact of scaling existing infrastructure.

Key Features include 

1. Allows users to create cost estimates based on specific workloads, configurations, and usage patterns.
2. Provides a breakdown of costs for compute, storage, networking, and managed services, including associated charges like data transfers or snapshots.
3. Accounts for variations in pricing across AWS regions, ensuring estimates reflect local cost structures.

For ex. consider a serverless application is being developed where AWS Lambda functions process user-submitted data. The Lambda function has a 500-millisecond execution time and is allocated 256 MB of memory, with 1 million invocations per month. Additionally, the application is estimated to transfer 10 GB of data out of AWS each month. The AWS Pricing Calculator is used to estimate the associated costs, factoring in free-tier usage for requests and compute time, with additional charges for data transfer beyond the free 1 GB allowance. In this example it would cost approximately $1.01 per month. 

Knowing this cost upfront has 2 key benefits 

Cost Awareness and Preparedness
It allows you to plan development scenarios while considering the impact of each factor on the overall cost. This preparation ensures you’re aware of potential costs, enabling better budgeting and adjustments if any parameters change during the execution phase.

Improved Service Understanding and Risk Mitigation
While some cost factors for this cost calculation might be unknown at the start, understanding that these factors contribute to the total cost helps you gain a deeper insight into the service. This awareness allows you to be better prepared for potential cost fluctuations or the worst-case scenario as your usage scales or requirements evolve.

To know more about how to create an estimate see this. AWS enables users to create cost estimates across services, which can be saved and shared for collaborative planning.

AWS Budgets for cost tracking

AWS Budgets helps you proactively manage costs by providing early warnings if your spending exceeds your expectations, allowing you to address issues before they become larger problems. You can create budgets based on cost, usage, or specific service usage. Whether you’re tracking monthly costs or monitoring the usage of a particular service, AWS Budgets allows you to set thresholds and receive notifications when your actual costs or usage exceed your predefined limits. This feature ensures that you are immediately aware of potential overages and can take corrective action.

The 2 most frequently used templates include – ‘Zero budget spend’ and ‘Monthly Cost budget’. As the name suggests ‘zero budget spend’ notifies you once your spending exceeds $0.01 which is above the AWS Free Tier limits and is relevant when you are in your free tier account usage. 

The ‘Monthly Cost budget’ is more relevant for businesses beyond the free tier usage. For eg. if you estimate your monthly cost for AWS account to be around 100$, its safe to setup budget exceeding 20% of this cost which is 120$. This means you will be notified when 1) your actual spend reaches 85% 2) your actual spend reaches 100% 3) if your forecasted spend is expected to reach 100%.

Cost Anomaly Detection

AWS Cost Anomaly Detection is an advanced tool designed to help you identify unusual spending patterns in your AWS environment. By leveraging machine learning algorithms, it provides automated insights into unexpected cost variations, enabling you to take timely action to control costs. While AWS Budgets tracks overall cost trends or specific usage against planned limits for a fixed period, AWS Cost Anomaly Detection, on the other hand, focuses on reactive monitoring by identifying unusual spending patterns, even within budgeted thresholds.

For ex. If your usual daily Lambda costs are $2 but suddenly spike to $8 on a single day due to unexpected increases in invocations or execution time, AWS Cost Anomaly Detection flags this as an anomaly, even if your total monthly cost is still under the budgeted cost.

By identifying anomalies early, you can investigate and address potential cost drivers before they escalate into significant expenses. When starting out with the Cost anomaly or a new AWS account, It’s advisable to set up monitoring at a Service level. This grants you the basic required monitoring. 

As you get familiar with this service or depending on your account usage, you can opt for ‘Cost category’ or ‘Cost Allocation Tag’ that helps you track and identify unusual spending patterns based on specific tags applied to your resources. This enables granular cost monitoring and provides insights into anomalies within tagged projects, environments, or business units.

You can attach ‘Alert Subscription’ to these monitoring alerts that notifies you when a cost monitor detects an anomaly. Depending on the alert frequency, you can notify designated individuals by email.

Visualizing and Analyzing Your Cloud Spend

AWS Cost Explorer is a visualization tool that provides insights into your AWS spending and usage patterns over time. It enables you to analyze historical data, identify trends, and predict future costs, offering an intuitive way to manage and optimize your cloud expenses. A typical cost explorer dashboard would look like this

Some of the Key Benefits include

1. Cost Explorer provides a breakdown of your AWS usage and costs across services, accounts, and tags, helping you pinpoint cost drivers.
2. With cost predictions based on historical data, it helps in estimating future expenditures and planning budgets effectively.

When viewing the current month cost, its most effective to use the Date range of ‘Month-to-date’ , granularity of ‘Daily’ and a groping dimension of ‘Service’. This will allow you to view per service charges on a daily basis as shown below

By enabling focused analysis, filters in AWS Cost Explorer make it easier to identify the root cause of cost spikes. It allows you to customize your cost and usage reports by focusing on specific attributes such as services, accounts, regions, or tags. This precision helps in identifying and analyzing cost spikes effectively.

For ex. By filtering costs by AWS regions, you can detect if a spike is related to region-specific activity, such as higher usage in a particular geographic area.

Addressing Unexpected Costs

The above described methods help one proactively highlight the cost spikes. However, it doesn’t help to avoid them. So what do you do if you encounter unexpected charges?

Try to find the root cause of the issue
The Cost explorer helps in understanding the service associated with the cost spike. You can make use of the filters to appropriately drill down to the Region to try to understand the resource that is possibly causing the spike.

Reach out to AWS Support promptly
The AWS support team can help investigate and resolve the issue, providing clarity on the root cause. In cases of genuine mistakes or misconfigurations, AWS Support may also offer billing adjustments, ensuring that your costs align with actual usage. Don’t hesitate to leverage this valuable resource for maintaining cost efficiency and transparency.

To Summarise

Effective cost management in AWS is not just about reacting to spikes but proactively planning and monitoring your spending. Tools like AWS Cost Anomaly Detection, AWS Cost Explorer, and AWS Cost Calculator empower businesses to visualize, predict, and control their cloud costs while optimizing resource usage. Taking preventive steps, such as setting up cost alerts and anomaly detection models, helps highlight potential issues early. 

However, In case of unexpected cost spikes, leveraging AWS customer support can provide quick resolutions. AWS’s support team is highly responsive and prioritizes customer satisfaction, especially for long-term users, ensuring that issues are addressed efficiently.

By combining these tools and strategies, you can maintain better financial oversight, avoid surprises, and ensure your cloud infrastructure remains both scalable and cost-effective.