Atmecs-Casestudy

Cloudops Automation Project ATMECS – Content Team The client is the world’s leading independent, end-to-end IT services company, serving nearly 6,000 private and public sector clients from a diverse array of industries across 70 countries. The company’s technology independence, global talent and extensive partner network help clients harness the power of innovation to thrive on change and guide their digital transformation journeys. Challenges: Development of single pane of glass platform to monitor, automate, orchestrate multiple public and private cloud offerings.  To implement single design to support all cross offerings in the public and private cloud family. Cost optimization by transiting operation from DevOps to DevSecOps. Solution Approach: The client platform is a digital-generation delivery platform that underpins: Intelligence, which provides the client digital platform ecosystem with the ability to ingest and process a wide spectrum of data types  Orchestration, which focuses on the ability of the platform to organize, execute, and track the processes and workflows  Automation, which encompasses the technologies that enable the mechanization of tasks within the delivery environment services to managed service offerings for public and private cloud. ATMECS Contribution: CloudOps Dashboard Development Operational Automation for AWS and Azure  Azure Powershell automation  AWS Boto automation script  Vault Implementation for Securing customer credentials Setting up CI/CD process Cloudops Automation Project

Building IoT platform for an Ad network company ATMECS – Content Team Client is the next-generation technology for digital, out-of-home market. It creates interactive experience of digital marketing anywhere, on any space. Client’s solutions allow its customers to elevate their physical locations, turning the into digital, social environments that add value and personality. Client’s open and extensible, Experience as a Service (EaaS) platform has five core capabilities: Automation Support, Control Network, Content Lifecycle Management, Ad LifecycleManagement, and Experience Lifecycle Management. Challenges:  The application is exposed on the Internet, outside trusted on premise boundaries. It prevents malicious or accidental actions which can compromise security. The changes made could degrade the performance to which users are accustomed. High business complexity to design and maintain all the services Advertisement schedule setup and running associated with Hardware setup, So need more resources Achievements: Maintaining test scenarios, test reports, and use cases for every new release of firmware.  Analyzed embedded software modules to bring negative scenarios. Found at least one corner case in all modules.  Documenting scenarios and test cases in confluence. Found an issue while testing the module which doesn’t handle sound and video together. The client appreciated fordoing R&D level testing for a future-focused feature. By analyzing the functionalities, found some interesting issues which had not been fixed from so long as thosescenarios are not straight away. Ability to deliver on time. Completing tasks within the sprint with increased productivity. Ability to work on all hardware modules. Working on high-end features correlating with trending market values. Planning to implement automation for hardware. Building IoT platform for an Ad network company

Infrastructure Support – Application Servers Administration ATMECS – Content Team Challenges: Multiple versions of Application servers Application dependency Non-availability of documentation Limited alert setup for Application servers Solution Approach: Application patching on jboss , tomcat and mule for more than 5 applications Building new jboss servers for application- setting up jboss,connection & http pools, data sources, properties, MQ connections, memory sizes — for all the above configuring server groups, profiles for management purposes. Troubleshooting application issues like application unreachable, slow application and front end error responses. Troubleshooting Linux issues which effect Jboss applications like wrong permissions , mount points, file not found. Configuring monitoring on Appdynamics for new requests. Dev, QA team requests on logs, config files and deployment issues. ATMECS Contribution: Undertook server analysis for more than 50 issues/servers , which intended to reduce repeated issues by fixing them permanently. Jboss script upgradation. Inventory preparation for jboss, which was earlier undocumented Infrastructure Support – Application Servers Administration

Infrastructure Support – Cloud Administration – AWS ATMECS – Content Team Challenges: Lack of Governance in the new environment Lack of proper cloud formation templates Requirements for a hybrid cloud model for different applications Lack of proper thresholds for alerting Solution Approach: Make sure the operations run smoothly and resolve any production issues at the earliest without downtime In case of an outage, follow it up with an RCA, at the earliest Make sure that we can continuously monitor our application’s behavior and optimize alert thresholds based on historical data Make sure that every change to the infra is recorded and documented in CloudFormation Templates (Infrastructure as a code) Come up with new suggestions/recommendations as per the industry’s best practices Try to use fully managed AWS service to have minimal operations overhead ATMECS Contribution: Implemented the best practices of DevOps as well as of Agile methodology across multiple environments in AWS Implemented CI/CD pipelines involving Packer, Salt Stack, Jenkins and AWS for web app environments so that the client is confident about the changes in code and its deployment across server Created several Jenkins jobs which run across multiple AWS environments to find AWS resources (IPs | ELBs | NameTags | ENICs etc.) Created custom dashboards and configured SNS notifications for proactive alerting of any potential outages Infrastructure Support – Cloud Administration – AWS

Database Administration SQL Server ATMECS – Content Team Challenges: Limited documentation Competing priorities Limited alert setup for database functions Application dependency on lower functions Solution Approach: SQL Server Provisioning SQL Server Upgrade SQL Server Monitoring to keep alive(24*7) Disk Spaces, CPU and Memory verification Database Backups and Recovery Database refresh from higher to lower environment Database Patching Database User Management Database Indexes Debugging Application related DB issues Achieve SLAs ATMECS Contribution: Implemented a robust monitoring system to monitor SQL Services status and CPU, Memory and Disk level thresholds Placed a PowerShell script to monitor and report all the error events in production error logs Automated the process to perform DB refresh from higher to lower environments Automated the process to rebuild index offline/online on periodic basis Placed scripts to generate various types of application reports Developed jobs for Index Optimization Developed scripts to auto failover the SQL Instances Database Administration – SQL Server

Infrastructure Support – Linux Administration ATMECS – Content Team Challenges: Limited documentation Varied flavors and versions across the enterprise Space issues due to log files Elevated access for developers leading to infrastructure issues Solution Approach: Implementing the RFCs Managing the user global access through Third party AD environment Solarwinds, Linux Configuration Veritas Volume Manager Administration Server Patching and year end reboots Managing Servers through VMWare VSphere Environment ATMECS Contribution: Implementation of log rotation on multiple servers to save Disk space and which in turn helped in reducing Incident Volumes Automated Daily routine tasks to reduce 40% of the tickets Automated JBoss services Implementing ELK-stack infra in the client environment for Centralized log and Log analysis Infrastructure Support – Linux Administration

Infrastructure Support – MQ Administration ATMECS – Content Team Challenges: Limited documentation Limited alert setup for MQ issues Complicated setup for DB replication Solution Approach: Monitoring queues, channels and queue manager statistics using MQ explorer, IR360 and manually Providing the levels of access to users/applications for MQ objects such as queues, channels and Queue managers etc Creating the MQ point to point setup Creating client server architecture by creating server connection channels, client connection channels and tables ATMECS Contribution: Created alerts in all environments in case of any MQ queue manager goes down. Setup a completely new MQ queue Capture/Replication architecture on Production DB servers. Setup queue depth alerts based on thresholds Setup re-processing of dead letter queue messages by using dead letter handler Infrastructure Support – MQ Administration