Backendless is a US-based company that offers a platform for developing mobile applications without the need for coding skills. The platform’s functionality includes tools for creating both the frontend and backend sides, allowing Backendless’s customers to bring their ideas to life without coding skills.

PROBLEM DESCRIPTION

The Backendless platform is a popular solution used by hundreds of customers, ranging from small startups to large corporations. Before partnering with SHALB, Backendless allowed customers to deploy the platform in their environments while hosting the installation on a server. However, as more enterprise customers began using public cloud services, Backendless needed to provide the ability to deploy its SaaS within their cloud accounts.

Moreover, Backendless aimed to make their software easily installable. Ideally, they wanted it to be a single command that could enable fast and hassle-free installation on the customer’s side.

Additionally, Backendless had another requirement: to be closer to their customers around the globe, they needed to quickly launch new AWS regions in specific areas. However, this was technically challenging since the company’s infrastructure wasn’t codified. The task required manual reproduction of stack configuration; a lengthy and complex process. Moreover, Backendless was time-constrained as they planned to launch marketing campaigns in new geographic locations soon.

WHY THEY CHOSE US

After evaluating different contractors for the tasks, Backendless decided to choose SHALB. Having been a customer of ours before, they were already highly satisfied with our services. So, when they needed help again, they didn’t hesitate to contact us.

The tasks assigned to us by Backendless included:

• Developing a solution that would enable quick and easy deployment of their enterprise SaaS on the customer’s side.
• Integrating the solution within the complex Backendless system.
• Providing the Backendless team with the ability to swiftly launch AWS regions within new target areas.

PROJECT DELIVERY

During the course of the project, the SHALB team developed a codified solution that enables the quick deployment of a fully-fledged AWS infrastructure with an EKS cluster. The cluster is launched with a pre-installed set of the customer’s software, including databases and workloads, and a monitoring and logging stack is set up.

For the infrastructure deployment and provisioning, we used Cluster.dev – SHALB’s very own solution for managing cloud-native infrastructures. The advantages of using Cluster.dev met all the requirements for this particular project, including:

1. Fast and easy deployment of SaaS. We could have employed Terraform to achieve the same results. However, Cluster.dev has a simpler configuration and, unlike Terraform, allows for a one-command deployment.
2. Saving time on resource validation. Cluster.dev has its own state, which eliminates lengthy validation of AWS resources performed by Terraform every time before applying changes. In large infrastructures with frequent deployments, like that of Backendless, it can save up to two hours of working time per day.
3. Managing complex systems. Backendless has a complex infrastructure with many interconnected resources. Cluster.dev is ideal for managing such systems as it keeps track of interrelations and dependencies among components.

Challenges

According to Roman Progonnyj, the senior DevOps engineer engaged in the project, the hardest part was integrating the developed solution with the customer’s existing stack while complying with its logic. This required developing numerous connecting points that our engineers devised in collaboration with the customer’s team.

To summarize; the SHALB team had to integrate several components together and teach them how to work with one another seamlessly. For example, teach the Backendless workload to work with Cluster.dev and perform additional functions such as creating domains and Ingress controllers in EKS using Kubernetes API, and enabling the Cert manager to access the domain zone in another account. All of this required a high level of technical expertise and collaboration between SHALB and Backendless teams.

PROJECT OUTCOME

After the project’s completion, Backendless gained a fully codified solution for deploying their enterprise SaaS on customers’ cloud accounts. The solution was seamlessly integrated with Backendless’ existing system and complied with its inner logic. This enabled the company to quickly launch new AWS regions in required areas, making it easier to be closer to their customers around the world.

The solution also allowed for fast and easy installation of Backendless’s SaaS on the customer’s side, which now takes less than an hour. A one-command launch eliminates the need for lengthy and complex manual reproducing of stack configurations, while a high level of automation minimizes chances of human-induced errors.

Building infrastructure code with Cluster.dev simplifies its further management and has certain advantages compared to Terraform.

CONCLUSION

The SHALB team has the knowledge and skills for building infrastructures of any complexity. So, if you have any problems with your existing infrastructure, don’t hesitate to contact us. Book an online meeting or contact sales@shalb.com for more information. We’ll be happy to hear from you!

The post Revolutionizing Enterprise SaaS Deployment: How Backendless Achieved Swift and Seamless Installation with SHALB appeared first on SHALB.

CUSTOMER BACKGROUND

DRINKS.CH is the largest online spirits retailer in Switzerland. The company’s online platform hosts over 5,000 products and serves customers all over the country, including restaurants and specialist retailers.

In 2014, when DRINKS.CH was founded, it was perhaps the only online retailer in the mostly offline and traditional Swiss spirits market. Its founder, Alexander Curiger, relied on digitization from the start. The company had an AWS-based infrastructure that enabled the serving of customers and processing of orders in real time.

PROBLEM DESCRIPTION

Over time, DRINKS.CH’s existing stack grew outdated and was no longer capable of meeting business demands. Their infrastructure wasn’t defined in code or scalable, and the apps weren’t containerized. The company needed a new system that would allow dynamic provision of environments and a reliable delivery system.

A mutual contact recommended SHALB as a contractor and the tasks we were subsequently assigned with included:

• Designing an architecture for the new system
• Implementing dev/stage/prod environments
• Setting up a code delivery system
• Containerizing existing apps
• Migrating the workloads, databases, and message queueing services to the new infrastructure
• Setting up monitoring
• Providing 24/7 support.

PROJECT IMPLEMENTATION

Using a self-made reference code, SHALB engineers launched an EKS infrastructure in AWS with a predefined VPC, private subnets, and access permissions. The Terraform code deploys a fully-provisioned Kubernetes cluster with add-ons, cluster resources with dependencies, and preset monitoring stack.

Code delivery and feature environments

Code delivery to target environments is implemented by means of GitOps. GitLab CI Runner runs inside the dev-stage cluster and leverages its unoccupied resources to run builds.

Setup of the CD system allows for dynamic provisioning of the feature environment each time developers commit new changes to the feature branch. The temporary environment is automatically launched in a dedicated namespace within the cluster. This is a fully-fledged, totally isolated setting with all services running, preconfigured ingresses, SSL certificates, and domains specified for an app. The environment allows testing of the functionality of either an individual app or several apps working together.

Successful changes are automatically delivered to a stage environment for system testing. Here, the app is being tested in suite with other apps in an environment that replicates the production. Deployment to production is done manually after approval.

Containerization and Helm charts

Within the project workflow the customer’s apps were containerized and made ready for deployment to the cluster. Writing Helm charts was one of the most time-consuming tasks. The customer’s apps significantly differ from each other, so it was decided to create an individual Helm chart for each one. This was also more convenient for the developers who try to maintain the charts for their apps to have the chart within the application repository.

Monitoring

Monitoring is based on a cloud-native stack that includes Grafana, Prometheus, and Alertmanager configuration.

PROJECT OUTCOME

Upon completion of the project, DRINKS.CH had an up-to-date and scalable infrastructure, perfectly aligned with their business needs. Now they are able to process as many customer queries as needed without degrading their systems’ performance. Automated CI/CD pipelines enable quick delivery of changes and the launching of temporary environments on demand. In addition, the advanced monitoring system provides insights into the state of deployed services and infrastructure components.

Deploying a fully-provisioned EKS cluster can be challenging and time-intensive. With a reference code at hand we reduced this time fourfold, from a month up to a week. Working on DRINKS.CH’s case, we further refined the code to be able to deliver large projects in the shortest terms.

SHALB provides all kinds of services for infrastructure development and maintenance, including infrastructure design and scaling, Kubernetes deployment and infrastructure migration. Our team is flexible, accommodating, and knowledgeable. Book an online meeting or contact sales@shalb.com to see it for yourself. We’d be delighted to hear from you!

The post Scalable infrastructure for a Swiss spirits shop appeared first on SHALB.

Concert.ua is one of Ukraine’s leading event ticketing agencies. The company handles almost half of the country’s online ticket sales. For its business to thrive and remain competitive, Concert.ua needs to provide fast and consistent services that will keep its customers happy.

Problem description

Before going serverless with AWS, Concert.ua’s systems were already in the cloud, but they were not optimized for it. It was a legacy infrastructure running on virtual machines that was inefficient and unable to handle sudden traffic spikes.

Announcements of upcoming events attract a huge number of visitors and increase the website’s traffic up to 1,000 percent within minutes. Although Concert.ua’s technical staff planned for such activity spikes in advance, it was hard to estimate the precise number of resources. “Even when a traffic spike was expected, it was always a guess as to how many servers we’d need,” says Yevgen Lysenko, founder and Chief Technology Officer (CTO) at Concert.ua. “But there was no other option with the resources and technologies we had at the time.”

With databases flooded with queries and services not responding, customers were faced with delays waiting for their transactions to proceed. Needless to say, this affected the company’s reputation and revenue.

Managing infrastructure was becoming a troublesome and time-consuming task. As a result, Concert.ua was looking for a way to resolve these issues and free up its developers to focus on application enhancement rather than infrastructure troubleshooting.

SHALB, a long-term technical partner of Concert.ua and a certified AWS associate, suggested the company’s transition to AWS out-of-the-box services and offered its help in migration.

Project implementation

SHALB engineers automated the customer’s cloud setup by implementing an Infrastructure as Code approach using Terraform. With infrastructure resources defined in code, the Concert.ua’s systems became easily reproducible, scalable, and manageable from configuration files.

The customer’s applications were containerized and prepared for running serverless on a combined platform. The solution brought together Amazon Elastic Container Service (ECS), an automated tool for container orchestration, and AWS Fargate, a serverless pay-as-you-go compute engine.

The combination with Fargate made it possible to achieve the fault tolerance of ECS-running applications while also enabling the possibility of their infinite scaling to accommodate sudden traffic spikes.

The customer’s MySQL databases were migrated to Amazon Aurora, a relational database service designed for working in the cloud. The transition to Aurora DB significantly improved the performance of Concert.ua’s databases, making them scalable and capable of efficiently handling customer requests, even when traffic is high.

Employing AWS Lambda, a serverless, event-driven compute service allows Concert.ua to concurrently process multiple transactions in real time; customers now receive their tickets straight away upon completion of a transaction with no additional waiting time.

Also, using Amazon API Gateway and AWS Lambda dramatically reduced the time it takes for the Concert.ua’s developers to implement APIs. “We couldn’t launch APIs as quickly as we can now,” says Lysenko. “Previously, we had to do a lot of coding but now it’s 300–500 percent faster. Using AWS, our software development cycle takes less time and effort, by fewer people. And it costs less than our previous setup.”

Business results

Since migration to a scalable architecture built on AWS, Concert.ua has increased the reliability and availability of its systems. The company’s website is now secured against sudden traffic spikes and reports 99.9 percent uptime.

Transition to a serverless platform allowed Concert.ua to cut its infrastructure costs and improve customers’ ticket-purchasing experience.

SHALB is proud to be a part of the Concert.ua success story. After all, our teammates are keen event-goers themselves and are the first to appreciate the fast and reliable services of Concert.ua!

SHALB is a certified AWS partner eligible to provide consultancy services and build infrastructures in the Amazon cloud platform. Our expert staff can advise on any kind of cloud-related questions, including migration and optimization of cloud cost and services, to building large scale distributed systems.

The post The highly efficient serverless platform for Ukraine’s largest ticketing agency appeared first on SHALB.

Corefy is a unified payment orchestration platform for online businesses and payment institutions. Integration with Corefy enables its customers to build into their websites and apps the feature-rich functionality of making and receiving online payments. The company has partnered with 200+ PSPs and offers support for any currency, payment method, and flow.

Corefy is an international company that has several global locations, including the HQ in London, the APAC office in the Philippines, and local offices in the Netherlands and Israel. The company’s R&D office is located in Kyiv.

WHY THEY CHOSE US

Around eight years ago, before the emergence of the Corefy platform, its co-founders were engaged in developing another payment solution, Interkassa. Back then, SHALB specialists helped them to adjust their existing infrastructure and design a new one on AWS, which was more suitable for their product needs. Working together had been a positive experience, so when this new project arose, they didn’t hesitate to contact us again.

THE TASK

Before our involvement, Corefy ran their processes on an on-prem, VM-based infrastructure. It was outdated, not scalable, and had many problems. Corefy had been thinking about cloud infrastructure, however, they lacked the expertise in the cloud native field to design a new one with their own resources. Dmytro Dzubenko, CTO at Corefy, says:

SHALB provided us with consulting on how to build our new infrastructure using new technologies to cover our needs. Initially, they took our vision and gave us excellent recommendations for databases, and we decided to go with Kubernetes.

So our task lay in providing consulting services to help Corefy migrate their infrastructure to a new platform, and ultimately guide them on how to rebuild it on Kubernetes.

PROJECT DELIVERY

Infrastructure as Code (IaC)

Our main efforts revolved around the dev environment, which was set up as a reference for Corefy’s own developing endeavors.

Following the IaC approach, our specialists described with Terraform the creation and provisioning of infrastructure resources – a managed Kubernetes cluster on AWS to run Corefy’s services. Subsequently, this code has been used by the Corefy team to launch staging and production environments.

GitOps approach

In order to enable deploying to the Kubernetes cluster from GitLab, we set up GitLab pipelines and employed Argo CD to automate the deployments. Kubernetes manifests were specified as Helm charts. Our team advised Corefy’s engineers on how to write the charts and deploy applications to Kubernetes.

Release changes are committed to the infrastructure repository, where they are then taken by Argo CD controller and applied to destination environments. The changes are automatically rolled up to dev and staging, except for the production rollouts that require manual approval.

Autoscaling

Autoscaling of Corefy’s applications is enabled by KEDA – a Kubernetes-based event driven autoscaler with Amazon SQS as an event source. The customer’s processing operations are defined as events. If the number of events is rising, this triggers upscaling and an increase in the number of worker nodes.

Service mesh

Сustomer’s services in the cluster communicate over a network secured by Linkerd, which is integrated with Argo Rollouts to enhance its traffic shaping abilities. The configuration made it possible to gradually shift traffic during a release, enabling shifting of customer queries to the new version in percent.

Special requirements

In order to meet SLA commitments, Corefy needed to divide traffic on the platform between its customers: multi-tenant that share a payment application, and dedicated or white label, which have a personal payment system under their brand. The latter have better SLA offering, and their data is being stored and processed separately.

The solution has been realized on an infrastructure layer by implementing separated load balancers for multi-tenant and dedicated customers.

PROJECT OUTCOME

Thanks to our collaboration, Corefy acquired a cutting-edge infrastructure that is scalable, cloud native, and based on open-source technologies. According to Dmytro Dzubenko, our involvement resulted in a significant reduction of bugs in their system.

Corefy has improved its release process and made it more reliable through the implementation of GitOps, together with Argo CD. Employing Argo Rollouts expanded the deployment functionality in terms of traffic flow and introduced new update strategies (blue-green and canary deployments). An advanced observability system, based on Prometheus and Grafana, provides for the visualization of infrastructure issues, effective debugging, and log aggregation.

Currently, Corefy’s team is implementing their new infrastructure using the skills and knowledge that SHALB provided them with. The customer praised our vast experience in software consulting, with Dmytro Dzubenko stating that: “SHALB is an expert in building software architecture; they did an excellent job.”

Any questions concerning infrastructure development, maintenance, or support? Get in touch because SHALB engineers love such tasks! Our team is communicative, organized, and works remotely as part of your working community. Book an online meeting or contact sales@shalb.com for more information.

The post High level IT consulting services for a payment solutions company appeared first on SHALB.

CUSTOMER BACKGROUND

Spryker Systems GmbH, (Spryker), is one of the key players in the market of digital commerce. The company provides a high-functional platform-as-a-service (PaaS) solution that enables its customers to run their online commerce businesses in the most advantageous and secure way. The platform enables B2B, B2C, D2C, and Marketplace capabilities out of one stack. It is one of the most advanced commerce platforms in the market with over 900 API-based modules, cloud-enabled, and enterprise-ready.

PROJECT DESCRIPTION

The specifics of Spryker business model consists of two parts: PaaS, a platform where Spryker’s customers register as tenants to run their online shopping infrastructures; and SaaS, a software that enables customer services on the platform. The applications that enable certain functionalities are being developed both by Spryker’s in-house teams and 3rd party providers. In order to make these services easily available to customers, Spryker wanted to implement a middleware solution – an event platform that would link customers (PaaS) and applications (SaaS).

However, building such a product required strong skills in the field of cloud engineering – the resources that Spryker lacked in-house and had to hire out. Contacting SHALB was an internal suggestion that resulted in two months of fruitful collaboration. Piotr Nowinski, Head of Product Engineering at Spryker Systems GmbH, says: “We were looking for certified AWS experts, and they checked all the boxes.”

Requirements

According to Volodymyr Tsap, CTO at SHALB, the task involved building a core piece of functionality – a basic component that underlies the communication between customers and applications on the Spryker platform. As a core technology, it needed to work on a steady basis, be fault-tolerant and scale on demand. As the platform also had to process all system queries, it was essential to implement enhanced security measures to avoid security threats.

Implementation of subscription model

The platform allows for two types of communication between customers and applications: synchronous for REST API requests and asynchronous for independent tasks, in particular processing working queues.

By using Amazon SQS/SNS service we realized a queue subscription – the model that underlies communication between users (PaaS tenants) and applications on the Spryker platform. The process involves creating working queues that both users and applications subscribe to in order to exchange messages. The user puts a task on the queue, and a message broker notifies the subscribed application. Once the task is executed, the user receives a notification. Queue subscription allows for executing multiple tasks in parallel without blocking each other, making all the processes run much faster.

Access control

To ensure secure access to the platform, we used AWS API Gateway Lambda authorizer, integrated with Auth0.

Once a client calls the platform API, the API Gateway calls Lambda authorizer, which in turn invokes Lambda function to verify the identity of a user or service. The caller identity is received in the Auth0 access token. Then, based on the response from Lambda function, the API Gateway determines whether the client can access the platform API.

Scaling

In order to eliminate bottlenecks in performance, we provided horizontal scaling for the platform. Considering the amplitude of Spryker’s business, we opted for the same technologies as Amazon web developers use to enable limitless scaling of AWS platform – Simple Queue Service (SQS), DynamoDB, Lambda, Simple Notification Service (SNS), etc. As a result, we achieved almost infinite scaling for Spryker’s systems.

Challenges

Among the main project challenges, Volodymyr Tsap mentions a large number of components involved and the work required to assemble them. “On the whole, the project required a lot of custom programming from our side, – he explains. – We needed to programme Lambda functions and enable integration with external services such as Auth0. As there were no ready components, we developed some of them from scratch and tested how they work together. Apart from that, we captured all our working in Terraform code following the Infrastructure as code approach.”

BUSINESS RESULT

In two months, SHALB successfully delivered the work, meeting the client’s requirements. Working together with Spryker’s internal team, we developed a well-tested, configurably managed solution that, after further elaboration and integration, formed into the Spryker App Composition Platform.

Introducing the subscription model made life easier both for Spryker’s cloud customers and developers. First, it allowed for simple integration with 3rd party services for Spryker’s PaaS tenants: now they can subscribe to applications and obtain certain functionality in a seamless, no-code way. Secondly, it facilitated the development of the marketplace platform for Spryker’s tech partners. With the application subscription model in place, developers are spared the necessity of deploying similar functionality to multiple PaaS environments.

CUSTOMER FEEDBACK

Commenting on the collaboration with SHALB, Piotr Nowinski says:

Piotr Nowinski, Head of Product Engineering at Spryker Systems GmbH

Overall, they did a good job. They understood the timelines and worked towards them, delivering on time and within the budget. We didn’t have to push them. SHALB understood the goal we wanted to achieve and worked towards it.

Working on the Spryker project added to our expertise in cloud computing and custom software development.

Have some challenging tasks? Don’t hesitate to contact us! The SHALB team is capable of delivering even the most complex project. List your business and technical goals, and we show you the way to achieve them. Book an online meeting or contact sales@shalb.com for more information.

The post Delivering an event platform for e-commerce giant appeared first on SHALB.

LotusFlare is a high-tech US company that provides a cloud digital commerce and monetization service for communications and media service providers. Having started as a mobile internet provider for areas with limited internet access, LotusFlare has grown to become one of the leading players on the US telecom market.

The company develops its own SaaS and operates the in-house built stack to provide cloud digital services to their customers. LotusFlare is dedicated to cloud native, which underlies the company’s development cycle and their core products.

WHY THEY CHOSE US

As a growing telecom business, LotusFlare needed the ability to quickly launch new environments for connecting more customers to their cloud. The company had been working on creating a platform that would allow them to launch and provide typical infrastructures for each customer.

At the stage of the platform design, LotusFlare realized the need for a technical consultancy. SHALB was chosen for the job because of our experience and comprehensive knowledge in the field of cloud native products. “Cloud native has an impressive abundance of products and services, – explained Volodymyr Tsap, CTO at SHALB. – Working with different customers makes us constantly grow our expertise on the variability of available tools. So it was a wise decision to reach out to SHALB for expert advice.”

REQUIREMENTS

Together with the LotusFlare team we started working towards our common goal: designing a resilient platform for deploying cloud environments. Among key customer requirements were high availability, automated deployments, and simple cluster upgrades.

TECHNICAL IMPLEMENTATION

As a company actively engaged in software development, LotusFlare already had a well-running system of software delivery and deployment. They used Kubernetes managed services to run their workloads on AWS, and a GitOps CI for cluster deployments.

However, the problem with Kubernetes managed solutions is that they are usually a few versions behind the official release cycle. This means that the newly released functions are unavailable until they are adopted by the provider. Also, having no access to master nodes restricted the LotusFlare team from using some services that they needed. Taking all this into consideration, LotusFlare concluded that they needed self-managed clusters.

We suggested K3s as a cluster management utility. Based on our research, it had more advanced characteristics in terms of resilience and security than other tools.

Evaluation of Kubernetes control plane utilities as of 2020:

In order to make the process of cluster upgrades easier, we introduced the technology of rolling upgrades, where each of the cluster nodes is being upgraded in turn without service interruption. Moreover, each cluster deployment/upgrade is then followed by conformity tests that validate the cluster compliance to production demands.

We enhanced the K3s internal security features by using the Cilium CNI network plugin and integration with OpenID Connect. This way we restricted access from workloads to the K8s cluster and added a user-identity layer on top of the authorization system.

The clusters are failover-aware and run in several availability zones at the same time. This ensures the system remains available even if one of the DCs inside the availability zone fails.

PROJECT OUTCOME

Together with the LotusFlare team we created a core orchestration platform that is quick to deploy and easy to upgrade. The platform provides automated deployment of self-managed Kubernetes clusters to the cloud. You can think of it as a customized installer that allows for easy reproduction of infrastructures. LotusFlare are now able to roll out environments and set them up for new customers much faster than before.

Full control over system configuration and components allows LotusFlare engineers to use the cutting-edge Kubernetes version and try out the latest features available. The state-of-the-art technology stack ensures that the company stays technologically advanced and ahead of its competition.

The company has been using the platform in production for over a year and is extremely satisfied with the results. What’s more, the LotusFlare team has been constantly fine-tuning the initial solution so that it completely aligns with the company’s needs. Commenting on the project, Volodymyr Tsap said: “LotusFlare has a top-notch professional in-house team. We did mostly research and prototyping code, while they elaborated the rest of it.”

Our experience with LotusFlare proves that sometimes even the most tech-savvy team needs friendly advice. Any questions concerning infrastructure development, maintenance, or support? Get in touch because SHALB engineers love such tasks! Book an online meeting or contact sales@shalb.com for more information.

The post High-tech orchestration platform for 5G monetization company appeared first on SHALB.

PartsTech is a US-based start-up that helps auto repair shops find the right parts. The company provides an online platform that links repair shops with distributors and manufacturers. Working with a large network of suppliers, both locally & nationally, PartsTech enables one-stop shopping for their customers.

PROBLEM DESCRIPTION

PartsTech has a cloud-based infrastructure that enables their customers to look for products’ pricing and availability in real time. Needless to say that for their business to be successful everything should run like clockwork. Any infrastructure issue such as search not working or databases down could affect their customers and as a result, their sales.

Moreover, PartsTech planned to further grow their business, which meant the increasing number of customers and as such, more queries to process. That made the question of reliable infrastructure of paramount importance. Still, the company’s existing infrastructure could have hardly been called so: PartsTech struggled with major downtime issues every couple of months.

To make things worse, they couldn’t find a reliable system administrator who would oversee their infrastructure. At some point, as Olexy Sadovy, CTO at PartsTech recalls, they grew tired of this. So, when somebody from the PartsTech team recommended us, they were happy to get in touch and start working with SHALB.

WHY THEY CHOSE US

Having talked with Alexander Iatsuk, CEO at SHALB, Olexy Sadovy was pleased with our approach and the ways we suggested solving their problems:

Over 10 years of DevOps experience and extensive cloud expertise make SHALB engineers fully understand the customers’ needs and the space of their projects. We were happy to join hands with the PartsTech team to design the most optimal solution for them.

PROJECT DELIVERY

By the time PartsTech started working with SHALB, they already had a Kubernetes-based infrastructure on AWS, with clusters managed by kOps. However, as long as the clusters were self-hosted, it required manual management of Kubernetes masters (etcd, API) and backups, which significantly complicated cluster updates and often resulted in system downtimes. On top of that, the PartsTech infrastructure was not fully described in code. The codified pieces and Terraform documentation didn’t match the actual configuration, which made the infrastructure difficult to manage and reproduce.

As part of the project, SHALB engineers covered the whole PartsTech infrastructure with Terraform code and had it transitioned to managed EKS clusters. With AWS being responsible for cluster upgrades, the whole process became easier and more predictable. The EKS clusters were launched and provisioned by Cluster.dev – SHALB’s very own solution for infrastructure automation and management. Cluster.dev manages all necessary infrastructure used by EKS, such as IAM Roles, subnet tagging, and EC2-workers configuration.

To deploy the PartsTech infrastructure, we used one of the Cluster.dev stack templates and customized it according to the customer’s needs. Following the IaC approach, all working is done through the Cluster.dev infrastructure repository, ensuring the exact correspondence between the code and actual infrastructure.

PROJECT OUTCOME

Working with SHALB, PartsTech achieved their main goal: infrastructure reliability. According to Olexy Sadovy, the company had no major downtimes since the project implementation. Now, as PartsTech has a scalable and well-performing infrastructure, they are much more confident about their plans to grow the business. Olexy Sadovy says:

By transitioning the PartsTech infrastructure under Cluster.dev management we automated most of the infrastructure processes, this way minimizing human intervention and the chances of human errors. What is more, Cluster.dev templating enabled the PartsTech team to easily launch new environments for testing and development needs: with the cdev template it takes only a few minutes to spin up an infrastructure replica. Now they can have a production-ready EKS cluster with defined roles and provisioned worker nodes in less than an hour.

CUSTOMER FEEDBACK

Commenting on the works done, Olexy Sadovy admits that they are pretty happy with the achieved results: PartsTech has done with infrastructure issues and got a reliable partner whose expertise they can trust. With SHALB teammates constantly sharing their knowledge and skills, PartsTech feels like they have access to the knowledge base of not just people they work with, but of all SHALB engineers as well. On top of that, they really appreciate the synergy of PartsTech and SHALB teams working together on solving issues.

Currently PartsTech keeps on collaborating with SHALB to maintain their new infrastructure. As for the project, it was not that much technically challenging rather than complex in terms of making things right:

Olexy Sadovy, CTO at PartsTech

The problem with our project was we had so many people working on it that it was very convoluted, – comments Olexy Sadovy. – So it was really hard to untangle all this mess. And I think SHALB did a great job of working with that. We had a couple of bumps but we got through that and I think we ended up on the good side of it.

Working on the PartsTech project proved how smoothly the transition from almost having no system administration to having some could be. Our engineers seamlessly integrate into your workflow and work as part of your in-house team to achieve better results.

So, if you have any problems with your existing infrastructure, don’t hesitate to contact us. Book an online meeting or contact sales@shalb.com for more information. We’ll be happy to hear from you!

The post Reliable and secure infrastructure for auto parts marketplace appeared first on SHALB.

Bedrock Analytics, a US company, is one of the key players in the global market of retail analytics, in particular CPG (Consumer Packaged Goods) data analytics. The company’s customers are manufacturers of beverages, food products, detergents and cosmetics that compete for a share of sales in supermarkets.

Bedrock Analytics uses AI algorithms to gather, process, and analyze data that comes from a variety of sources, such as points of sales data, Syndicated Data Aggregators, Retail Store portals, e-commerce databases, consumer panel data sets, etc. In the input Big Data are gigantic arrays of information. In the output – a content-rich analytical extract that would take teams much longer to do in any other way. It is more storytelling than statistics. It is a set of tools that helps product manufacturers find an optimal way to customers.

Thanks to the digital products and services of Bedrock Analytics, both unique craft products can succeed in retail and the largest brands can maintain their market shares. Big Data based and well-designed solutions can give a boost to products that would otherwise struggle to compete in a highly competitive marketplace.

The company is listed in the top twenty global players of CPG analytics and occupies an expert position along with IBM, Microsoft, Manthan Software Services, Oracle, etc. So, when in August 2020 from Oakland, CA came a working inquiry from Navdip Bhachech, Senior Vice President Engineering at Bedrock Analytics, we were excited to work with his team. After all, we had heard a lot about their products! From the very beginning it looked incredible, and then turned into an inspiring DevOps experience that is worth sharing.

THE TASK

The market of CPG data analytics has been continuously growing. It is estimated that during 2021-2028 it will increase by 20%. The systems of Bedrock Analytics will have to process ever larger volumes of data. Their IT infrastructure should be able to handle that scope of operations, as the information provided by Bedrock Analytics is crucial for their customers to make their strategic business decisions on.

The initial scope of work included two main points: containerization and migration to Kubernetes in order to reduce Amazon bills. However, the requirements were complicated by their need for a multi-cloud solution: it was necessary to deploy the technology stack of Bedrock Analytics on two public clouds, AWS and Microsoft Azure, at the same time with the same code base. Their goal was to expand beyond the US and, with larger partners, required Microsoft’s Azure cloud infrastructure.

Among the factors that simplified the task implementation was clarity from the customer’s side – Navdip Bhachech understood the scope of work and deep requirements of Bedrock Analytics, so the communication was smooth. The challenge lay in the uniqueness of the solution.

What Bedrock Analytics wanted was the ability to deploy multi-cloud for the same application, which was quite uncommon. “Doing things multi-cloud means that you have to consider a lot of subtle differences between how AWS and Azure do things, – commented Navdip Bhachech. – If you want it to work the same way you have to tune it differently. And I think that was probably the most challenging part, keeping all of those details straight and organized in the work that was done.’’

The SHALB team has strong Kubernetes expertise and the experience of implementing unconventional solutions. We were chosen for this job because of our technical skills and the ability to understand the space of the project. Commenting on the process of choosing a contractor, Navdip Bhachech admits:

Develop a multi-cloud platform with Kubernetes, Terraform and support of GitOps and Argo CD? Let us handle the task! SHALB engineers love such challenges!

PROJECT IMPLEMENTATION

Infrastructure audit

As of 2019, the system of data orchestration at Bedrock Analytics wasn’t optimal to process a large quantity of operations. Since it was based on AWS OpsWorks technology, the orchestration system was vendor-bound and could not be used in other clouds. This also caused inefficiencies and additional expenses. The product complexity and the outcomes of the infrastructure audit revealed the necessity for a new orchestration system that would be based on Kubernetes and cloud-native technologies: Argo CD, Prometheus, etc.

Working with repositories

Having decided on optimal workflow, SHALB came to an agreement with the development team in terms of working with repositories: GitFlow and Feature Branching. As a result, we created CI/CD pipelines and chose a monorepo concept for infrastructure development.

Workload estimation

Bedrock Analytics is a complex platform that solves hundreds of tasks concurrently. This means that there are approximately as many batch workloads for both analytics and orchestration. In this case it was decided to use an open-source engine Airflow atop of Kubernetes. Serhii Matiushenko, leading DevOps engineer at SHALB, was assigned to implement automation, prepare an infrastructure code, and integrate it with the existing system. Along with other tasks, the process took nearly 3 months to accomplish.

Development of Kubernetes manifests and Helm charts

Kubernetes primitives are necessary for the faultless operation and scaling of each workload. Describing the primitives with Helm charts and Kustomize appeared to be quite a time-consuming process.

Data orchestration

Since the specifics of Bedrock Analytics imply processing of batch workloads, they needed an orchestrator to run data science jobs. They chose Airflow having considered the experience of the development team and its suitability for python-based Machine Learning environments.

Describing cloud resources with Terraform code

Since there were no public Terraform modules available for Microsoft Azure, we had to develop them from scratch. Unlike Terraform modules for Amazon that are plentiful and have been constantly improved, modules for Microsoft are fewer and of poorer quality.

CI/CD development

The team used to have Jenkins, which required considerable effort from the developers to maintain. We chose Bitbucket CI/CD pipelines as an alternative to Jenkins, saving the developers time. With Bitbucket you have your CI/CD pipelines running at the same place where the code resides.

Documentation

At SHALB, we thoroughly document all stages of infrastructure development to make its further maintenance easier. The case of Bedrock Analytics was no exception.

Delivering the solution

After the implementation of the project tasks, we delivered the prepared solution to the customer. The delivery process took several stages:

• POC
• DevInfras
• Staging
• Production
• Synchronized migration to AWS and Azure

PROJECT OUTCOME

As a result of the project, Bedrock Analytics received a more modern infrastructure, driven and managed by config. As Navdip Bhachech points out, they can now set up and tear down new environments much more easily and through source code. The flexibility of the new platform also provides for configurable scaling and tracking infrastructure changes, which enhances overall observability of the system. Their ability to manage cloud costs has also improved thanks to automation and better configuration of services.

The infrastructure solution designed by SHALB covered the customer’s business processes and had most of them automated. As a result, we managed to optimize the costs of their infrastructure maintenance and accelerate their product TTM due to the faster and simultaneous deployment to different clouds.

CUSTOMER FEEDBACK

Commenting on the project, Navdip Bhachech said:

Navdip Bhachech, Senior Vice President Engineering Bedrock Analytics

What I liked about Volodymyr and Sergii is that they would figure things out. They were very proactive in finding the resolution to every little issue and figuring it out and doing the research. That was nice to see. We were leaning on them for their expertise as the process went on. I think they did a great job on that and figured out a lot of little details.

The case of Bedrock Analytics added another significant and challenging project to the SHALB portfolio. Our team is friendly, proactive, and true masters of their craft. But what is more important, we understand the space of your project, no matter how complex it might be. Modernize your stack and stand out from the crowd with competitive advantages! Book an online meeting or contact sales@shalb.com for more information.

The post Multi-cloud solution for retail data analytics company appeared first on SHALB.

CUSTOMER BACKGROUND

Bank of Georgia (BoG) is one of the leading companies in the banking and financial services sector. It has a wide network of service centers and ATMs throughout Georgia, with representative offices in London, Budapest, and Tel-Aviv.

BoG is one of the largest employers in Georgia that actively supports healthcare, education, the environment, and other important social issues.

DESCRIBING A PROBLEM

Banks can be very traditional in some ways, but even they cannot ignore modern trends. In order to stay competitive, you need to keep pace with the latest technological developments and best practices.

Vazha Pirtskhalaishvili, Head Of DevOps Engineering unit at BoG, commented:

BoG was determined to become a technologically advanced bank. Driven by this realization, they decided to future-proof their systems and make them cloud-ready. To do so, their search began for a contractor that would help them achieve their goals.

WHY THEY CHOSE US

BoG initiated the project with a clear vision of what they wanted to achieve. Their criteria for choosing a contractor was equally clear: it had to be a trustworthy high-tech company with local representation in Tbilisi. SHALB was introduced as a potential contractor by HT Solutions, a Georgian IT consulting company and longtime partner of BoG.

That is how SHALB received a task to design a new infrastructure for the largest bank in Georgia. It was a unique and technologically challenging project that immediately ignited our professional interest: we were eager to start.

PROJECT DESCRIPTION

Requirements

BoG wanted a technology solution that would allow them the flexibility to manage applications on infrastructure that best suited their needs. According to their values and goals, they opted for microservices architecture, containerization, and Kubernetes.

We were asked to design a solid Kubernetes-orchestrated platform with the possibility to create and manage infrastructures which BoG could migrate their microservices to. One of the key requirements was the system’s fault-tolerance as it was to run business critical applications and have high availability of all architecture components. The solution also had to meet strict security requirements, be integrated with existing security and authorization systems, and reside in BoG’s inhouse data centers.

The first step to take was refactoring the monolith to microservices. BoG teams were responsible for containerizing applications, creating Docker images and running builds. SHALB was assigned with the task of having the services migrated and integrated into Kubernetes and provide their smooth operation. Based on the prepared Docker images, our engineers created Helm charts, pods, and deployments that were deployed to the Kubernetes cluster.

Technical implementation

The Kubernetes-driven platform design is based on VMware and Rancher technologies and has been implemented by means of Terraform.

vSphere, a cloud computing virtualization platform from VMware, provides the basis for low architecture level and unites all servers into a single system. On top of that we applied Terraform to create and provide Kubernetes clusters, and Rancher Kubernetes Engine (RKE) to manage them.

The clusters’ Control Plane is shared between two data centers, enabling automatic switching and traffic redirection in case the active DC fails. This scenario has been thoroughly tested as one of the critical customer requirements.

The network connectivity between container workloads has been implemented with Cilium CNI. By using the Cilium CNI network plugin we created a connectivity model with the fewest privilege’s access, and includes awareness of Layer 7 communications, thereby further enhancing the network security.

The robust monitoring system is designed to comply with strict security regulations. All the communication and behavior between microservices and the components inside the Kubernetes cluster is tracked. Any anomalous or potentially harmful behavior is detected and immediately reported to the security department. Aggregation of security reports is based on Falco rules; the reports are further streamed into the existing SIEM system.

Challenges

It was essential to meet BoG’s high standards, in particular for network security. The bank operates several data centers that work under complex rules. The rules also define how the microservices communicate and connect with each other. As a result, our system had to be designed in accordance with these rules in order to provide secure connection, which was architecturally challenging. In addition, the project was required to be completed within a short timeframe. This made things even more complicated as almost everything had to be designed from scratch.

As a bank, BoG also has strict regulations in terms of working with other contractors and third-party teams. SHALB specialists had no access to the production site and had to prepare some solutions on their side before BoG engineers could implement them on theirs. This sometimes required double the work to perform and inevitably slowed the whole process down.

With one of the largest IT departments in the region, BoG has strong engineering teams that are very good at what they do. Considering the project complexity and scale, we were happy to join forces with their qualified staff and work together on some technical issues.

In particular, our team was stuck on the problem of how to properly configure Rancher to create servers on the VMware side in order to pass data to other systems that run on these servers under Kubernetes management. Finally, thanks to technical advice of BoG VMware-certified engineers, we managed to solve the problem and move forward.

What makes it different

The project is trailblazing both in terms of the application field (banking sector) and technical implementation (architecture and configuration of services). Despite the technologies in use being actively developed, there is still room for new features and components, both in Kubernetes and Rancher. Our custom solution covers the missing functionality and makes it work regardless of how this stack implements the features that the customer needs.

Also, the solution is set up on the customer’s own data centers and is based on the VMware virtualization platform, although normally such systems are designed for public clouds or their on-prem analogues like OpenShift.

PROJECT OUTCOME

On completion of the project, the customer received a flexible and up-to-date platform, and all the tooling needed to launch, scale, deploy and destroy clusters on it. BoG specialists duly appreciated the advantages of Kubernetes in terms of scaling and quickness of deployments: first, it allows for automatic scaling in times of higher demand and downscaling when demand is reduced, and second, it significantly accelerates deployments making them a lot faster than before. According to Vazha Pirtskhalaishvili, they noticed the difference almost immediately after the project implementation.

CUSTOMER FEEDBACK

Commenting on the project, Vazha Pirtskhalaishvili confirmed:

Vazha Pirtskhalaishvili, Head Of DevOps Engineering unit at BoGh

The collaboration process went on smoothly and I think the guys did a pretty good job to achieve our goals. Personally I have to mention that there were some aspects that required additional work because with a project that big it is very difficult to plan everything ahead and you always have some pieces and bits that emerge in the process. And the guys didn’t hesitate to help even though it was not covered in our contract, which was a pleasant part.

Working on the BoG project gave us an inspiring DevOps experience and invaluable know-how of how to deploy cloud native in the fintech sector. Now we are ready to share this knowledge with you! Invest in modernizing your systems today to future-proof them for tomorrow’s challenges. Drop by for a friendly talk by booking an online meeting or contact sales@shalb.com for more information.

The post Modernizing infrastructure stack for the largest bank in Georgia appeared first on SHALB.