The ECS task retrieves the notification from the SQS queue and performs the required processing for the job. The output of the ECS task is specific to the engine. For eg. Communication engine will send out notifications.
Technology Stack
Architecture
The overall system architecture comprises of 2 subsystems
- one that supports the end user application and is based on Java microservices,
- the other which is a job service - an engine which serves as a distributed processing system for the biz ops team to perform bulk operations for various tasks.
Microservices Backend
Some of the core components of this architecture includes –
Job Service (Application Engine Subsystem)
The ‘Engine’ serves as a distributed processing system for the biz ops team to perform bulk operations for various tasks. Some of these engine processing include –
- Outbound communication - responsible for email / sms end users, will contain savings cost for a month.
- Extract generator engine - responsible for generating reports for biz users for a particular client.
The following is the architecture diagram
The front-end application’s REST API publishes a notification to the SNS topic. The SNS topic broadcasts the notification to all subscribed resources. Ie. SQS and Lambda. The AWS Lambda function receives the notification and triggers an ECS task.
Monitoring
For monitoring the microservices deployed in the Kubernetes-based architecture, Datadog is used. The setup involves the following steps:
This configuration allows comprehensive monitoring and analysis of microservice events and logs within the Kubernetes cluster.
The integration allows for real-time notifications of important events, such as service outages, high error rates, or high traffic volume. This allows teams to quickly respond to issues and minimize downtime.
Integration with Slack allows for contextual information to be sent ensuring prompt awareness and response to issues.