Tag: cloud

Scaling Up Your SaaS Product Infrastructure: High-Availability

Scaling Up Your SaaS Product Infrastructure: High-Availability

Many SaaS (software-as-a-service) products store massive amounts of data while required to provide seamless user experience when the data is being accessed. When serving low volumes of traffic, this mission is considered relatively simple, by operating one main server for HTTP responses, static assets, databases and background tasks. Things become more complex when higher traffic volumes get into the picture.

We’ve recently reached this point of evolution in ClickFrauds, which made us restructuring our servers’ infrastructure to become robust as well as highly-scalable. In ClickFrauds, we help Google AdWords advertisers monitor their campaigns against fraudulent clicks, and automatically prevent from fraudsters to interact with the ads.

A super sensitive application component which is exposed to the world-wide-web is the tracker which get hit thousand of times per minute and responsible for:

  1. Storing data from each request (click)
  2. Redirecting the request to its final destination (the advertiser’s website)

By the nature of the tracker, it goes without saying that malfunctioning of this component is not something we can afford our system to have.

When we started, we compared the main cloud providers out there to make a “smart enough” temporary decision for the next few generations of the product. Choosing DigitalOcean to be our cloud provider was an extremely wise decision. DigitalOcean has a seamless easy-to-use dashboard with a convenient pricing model. Droplets (server units) are highly-scalable, robust and contain most of the features used by an early-stage startup. The support team is very helpful, and the greatest thing about DigitalOcean is the endless collection of tutorials that are also ranked at the top of many Google search results.

Infrastructure 2.0

Starting with the requirements, we’ve listed three main concepts that must be taken into account when redeveloping the new servers’ architecture of ClickFrauds:

  1. High-availability and redundancy.
  2. Security layers against different potential cyber attacks.
  3. Monitoring mechanisms to ensure healthy infrastructure overtime

Now we will dive into the steps that were taken to fulfill each of the above requirements.

High-Availability

High-Availability illustration

High-availability means avoiding a single point of failure in each and every component of the system. When considering many different moving parts with different objectives as well as critical hardware issues that might happen but cannot be controlled or expected, this is not a trivial task. However, there are great open-source tools and mechanisms that if used properly, can achieve a highly-available infrastructure in an elegant way.

Load Balancing

To eliminate a single point of failure on machines that are exposed to the web and respond to HTTP requests, we use redundancy. Redundancy means duplications of machines that are aimed to fulfill the exact same set of tasks. We control and distribute the load over the redundant machines utilizing a load balancer. Now, the load balancer becomes the only machine that is exposed to external requests, while the duplicated nodes are hidden inside the internal network.

To implement a basic yet reliable load balancer we can use Nginx with a round-robin algorithm for distributing the requests between the nodes. Nginx will take care of everything for you, from periodic health-checks for each node, to managing the nodes and getting them back to the line after a failure recovery. Additional configuration should be implemented in order to ensure proper cookie sticking, consistent and reliable headers (like client’s IP address) and proper caching.

At this point, you may ask yourself, what happens if the load balancer goes offline? This is a great question that has to be answered to achieve a full highly-available system.

Floating IP addresses

Another great feature offered by DigitalOcean (for free!) is the floating IP address. As you may already know, DNS A records might take sometime to update from one server to another. In case of a load balancer failure, we need to change the main A record, for example, of the subdomain tracker.clickfrauds.com to point from the unavailable load balancer to an available backup one. Floating IP address enables us to achieve this goal seamlessly. It is an external IP that can be attached to different machines immediately. This way, in case of a load balancer failure, we leave the DNS A record from tracker.clickfrauds.com pointing to the floating IP as is, and reattaching the floating IP address to the backup machine.

 

Floating IP address illustration

Databases

Like any other component in a multi-layered architecture, also databases might fail from many different reasons, either hardware or software related. Databases are highly sensitive components, that are required at any time during the application life cycle. Therefore, it is extremely important to maintain a sustainable database access using a method called replication. Replication, in its most simple form, means holding your data in several different accessible places, rather than on a single machine. Replication eliminates the scenario of a single point of failure in your data system, and provide your application multiple access points to your data. Multiple access points are helpful for distributing I/O (input/output) operations from different machines within your infrastructure. In other words, different requests from users may reach different nodes (machines) to provide them with the data they need in order to balance the load of requests.

In order to accomplish proper replications of a database, your data is either distributed among different machines without duplications, or duplicated between the machines and therefore identical from one machine to another. In the latter, synchronization is a consideration that has to be taken into account – once something changes on one machine, it has to be updated on all the rest. Fortunately, the popular databases contain robust replication mechanisms, both SQL and NoSQL based databases. SQL (relational databases), will mostly use the duplication based replication method, while NoSQL, which was built for scale and for holding huge amounts of data, will use the distributed replication where on each node you will find different parts of you data.

It is common in mature applications to use both types of databases: SQL and NoSQL-based for different purposes. In ClickFrauds, we made a decision to use PostgreSQL, which work seamlessly with Django web framework, as our relational database for storing user-related data, and Redis, which is an in-memory super-fast storing system, for storing cache data, and holding a queue for all the background processes in our system. Both databases are replicated on several machines, which prevents a single point of failure and extends dramatically the I/O capabilities of each of the databases.

Celery Workers

Analysis of clicks data is a computationally heavy operation. Especially when the analysis that we perform on each click is based on a deep investigation of every aspect of the click, using technologies like supervised machine learning models, as well as graph database for pairing clicks that have a high likelihood to originate from the same source. Therefore, this analysis must not be done at the expense of user experience, rather is has to be performed in a background process. On the other hand, the analysis is the most effective when it is completed as closest as possible to the click operation in order to make a decision whether a specific IP address should be blocked immediately. Therefore, in order to accomplish our task at ClickFrauds, we have a requirement for a component that will manage the queue of thousands of background tasks per minute (a.k.a broker) and endpoints that will consume the tasks of click analysis and execute them successfully (workers). To accomplish that, we’ve decided to work with Celery which has high capabilities for managing and consuming background processes in large scale. Celery cannot operate without a broker to store its task queue, which we decided as a Redis database. To scale up the task consuming capabilities, all we have to do is to duplicate the celery worker machine and then immediately the consumption velocity is increased.

Once you reach the point of scaling up your SAAS infrastructure, the main concept that you need to bear in mind is to be able to run new servers in seconds, that will be integrated into the infrastructure seamlessly for doing their job. You can do it easily by taking snapshots of machines just before you launch your new infrastructure.

The next posts in the series will discuss how to secure your highly-available infrastructure and how to monitor it from potential issues. All of the topics mentioned here can be easily discussed further in details and will be discussed in next posts, please feel free to comment about specific topics that interest you the most.

How to choose a cloud computing technology for your startup

How to choose a cloud computing technology for your startup

Cloud computing technology becomes a standard when talking about developing applications nowadays. A few years ago, companies were enforced to have dedicated teams for configuring, running and maintaining server rooms which made it extremely difficult to scale up easily and offer a sustainable product. For small startups, it was even more difficult due to the lack of human resource as well as funding.

In present days, not only there are cloud computing technologies for almost every architecture you might imagine, but the cloud vendors also compete nonstop about our (the developers) attention. Most of the largest tech companies, like Google, Amazon and IBM launched cloud services in the past few years. They advertise, offer free tiers, present in tech conferences and conduct free-of-charge workshops for experiencing with their cloud solutions. They are aware that once you fall in love with their services, it will most likely be your favorite choice in every project for years to come.

So what is a cloud provider anyway? A cloud provider is an entity that offers cloud services for operating your application. Operating may include running servers, serving your application, hosting static files, providing database solutions, handling networking between servers, managing DNS and much more. Different cloud vendors offer different levels of abstractions in their services, usually defined as IaaS vs. PaaS.

cloud computing technology

IaaS (Infrastructure-as-a-service)

IaaS, or infrastructure-as-a-service, refers to a low-level solution, like providing a Linux Ubuntu server with nothing installed on it. This kind of solutions is suitable for more advanced developers who have experience with designing, configuring and securing servers infrastructure in all aspects. IaaS services provide you with flexibility and scalability down the road, and this will most likely be the way to go when designing application for scale. This approach requires, as already mentioned before, at least one developer in your startup who has this skill-set, otherwise, your product will turn into a big mess sooner than later.

PaaS (Platform-as-a-service)

PaaS, or platform-as-a-service, refers to a fully-maintained and managed environment that is hidden under a layer of abstraction you should not even care of. The cloud vendor takes care of maintaining the servers needed for the operations for you, and you get high-level databases for storing your data, services for user authentication, endpoints for client side applications etc. This approach is much easier and faster to get up and running with, and typically satisfies most of the basic applications. You should take into consideration though, that for more complex architectures it might not be enough.

Generally speaking, both IaaS and PaaS are huge time-savers when dealing with deploying and serving applications. You are able to run a server with a click-of-a-button and usually pay per use. Scaling your servers can be done manually or even automatically using APIs when a peak in traffic suddenly occurs. You can be sure that you’re in a good company (as long as you choose wisely) and whatever you can imagine, you can basically create.

cloud providers list

In early-stage startups, using cloud computing technologies became a standard because of the flexibility, the pricing models and the accessibility. Choosing the best cloud service for your startup is an essential task every technological entrepreneur must perform. As the head of development in your company, you should know the differences between the main alternatives, and choose the one that suits your product best.

Technical debts may stack up in a case of a bad decision. In addition, migrating an entire architecture from one cloud provider to another is not considered to be a trivial task at all. Therefore, you should be able to know the differences, experiment with each of the main alternatives and make a wise decision.

cloud services comparisonAfter examining and experiencing the best cloud providers out there:

and using them in a wide variety of project, I’ll take my top two: AWS and DigitalOcean and compare them using a set of parameters.

 

I’ve chosen these two cloud providers to be my best choice after grading each of them using the most important parameters when building a startup from the ground up:

  1. Features (offering) – how wide is the range of available cloud computing technologies, integrations and possibilities for the next generations of your application. In order to build for scale, you need to be sure that a cloud vendor can support your application for years to come.
  2. Pricing – Available pricing models, free tiers for startups and pricing transparency. Early-stage startups (startups that fund themselves) look for the largest value possible in the lowest price.
  3. Ease of use – How fast an intermediate developer can build a basic cloud architecture and deploy his application, How easy is it to iterate over the existing cloud architecture and what about the learning curve for beginners.
  4. Tutorials and support – Availability of online resources to help you get up and running with different services, as well as human customer support accessibility.

Three, two, one, fight!

Features

How wide is the range of services offered?

Amazon Web Services: AWS has by far the widest range of services and it comes to offerings. If you don’t find a cloud computing technology under AWS manifest, you’ll most likely not find it anywhere else. AWS has many different IaaS and PaaS services dedicated to every task needed to be performed by a server, divided into organized categories. When using AWS you can be sure that your startup scalability is potentially endless. On the other hand, the offering might sometimes be confusing for beginners as it makes the getting started process a little longer. If your application has many custom components that are not trivial, AWS might be the cloud provider you should consider.

Grade: 5/5

 

DigitalOcean: DigitalOcean offers a relatively narrow range of services. As for IaaS, you can find droplets (servers), data storage units, networking and monitoring services. As for PaaS, you can easily deploy apps with zero configuration needed, like Node.js, Redis, Docker etc. Although the offering is very concise, I find it to be exactly what you need for more than 80% of the applications. In addition to the standard droplets, high CPU and high memory droplets are available for custom use, as well as backups and snapshots for each droplet. DigitalOcean team is working nonstop on increasing their offering based on the community requests. As a developer who uses DigitalOcean for quite a lot of time now, I can admit that their desire to satisfy their community is highly appreciated.

Grade: 4/5

Amazon Web Services cloud provider

Pricing

Pricing models available and transparency

Amazon Web Services: AWS is based on a pay per use pricing model. Every cloud computing technology has its own unique pricing and a pricing calculator is available for trying to estimate your costs upfront. You might find this calculator a bit complex if you haven’t used AWS before. In order to estimate your costs up front you need to translate your servers architecture design into AWS terms, and then try and estimate by choosing the appropriate services from the sidebar. I find the wide range of offering sometimes overshadows the costs estimations, so I find it useful sometimes to start firing up services and tracking the costs inside the dashboard using pricing alerts. On the other hand, AWS offers a very useful free tier for 12 months that can help early-stage startups get up and running.

Grade: 3.5/5

 

DigitalOcean: DigitalOcean extremely transparent pricing models exist in two different yet similar approaches: pay per hour and pay per month. When using DigitalOcean you have no surprises. You can calculate the exact amount that will be charged, due to fixed prices for each droplet unit. Starting at $5/month for a 512MB droplet, DigitalOcean is suitable also for tiny side projects. Besides droplets and data storage units that are charged according to the resources allocated to you, networking, monitoring, alerts, DNS management and more, are completely free of charge. Bottom line, you pay only for the allocated resources, and you get a lot of useful extra components as a free of charge service.

Grade: 4.5/5

 

Ease of use

How easy is it to get up and running as well as to iterate

Amazon Web Services: AWS dashboard is quite comfortable once you get used to it. Because of the large amounts of services, you might find it a bit overcrowded in comparison to the other alternatives presented here. You can use the default settings for your services and then to get up and running relatively quickly, but if you’d like to dive deeper into details (also for reducing costs) you might find yourself spending quite a lot of time on configurations using AWS dashboard. On the other hand, in large-scaled applications, you can find the additional features available for each service extremely useful and necessary.

Grade: 4/5

 

DigitalOcean: DigitalOcean is branded for a reason as “Cloud computing, designed for developers”. As developers, we have so many things to take care of, especially when in charge of the end-to-end technological stack of our startup. Therefore, we need our cloud provider to be as simple as possible to setup. DigitalOcean’s user interface is the best I’ve used. It’s intuitive and let you get up and running in minutes even when using it for the first time. You don’t need to explore and scroll over too many features and options, just choose your Linux distribution, plan and geographic location, and you’re up and running in no time.

Grade: 5/5

DigitalOcean droplet creation

Tutorials and support

Available resources and support team

Amazon Web Services: AWS has a very useful tutorials library. There are many tutorials, but ones sometimes seem to be less detailed and user-friendly than others. You need to be experienced with servers infrastructure design before accessing many of the AWS tutorials. So, it might take you some time to explore their library before you’ll be able to actually find what you’re looking for. On the other hand, their customer support team is extraordinary. AWS support agents are super responsive and sensitive and will answer your questions in a professional way.

Grade: 4/5

 

DigitalOcean: The tutorials library of DigitalOcean is endless. In almost every Google search about a topic related to servers or cloud infrastructure, you’ll find results from DigitalOcean tutorials library. The tutorials are well-written and cover important principals alongside with the technicalities of how to achieve your goal. In addition to accomplishing your task, you’re actually learning new things when following DigitalOcean’s tutorials. The support team is very responsive and professional, and free of charge virtual meetings are available with cloud specialists to help you design the architecture of your server.

Grade: 5/5

 

Summary – choosing the best cloud computing technology for your startup

Amazon Web Services: AWS is by far the leading cloud provider when it comes to offering, scalability and features. On the other hand, its learning curve is moderate, so if you haven’t experienced with AWS before, it might take you some time to get up and running with properly.

Final startup grade: 4.5/5

 

DigitalOcean: I like comparing DigitalOcean to a boutique hotel. When using their cloud computing technologies you feel like you’re part of a family and treated like one. DigitalOcean covers everything you need as an early-stage startup, it is easy to use and provides expected convenient pricing models.

Final startup grade: 5/5

best cloud solution

The most important thing about cloud provider is to have one. In our world, it’s much better to have your application deployed in a little smaller cloud provider than keep arguing about which cloud provider is better when you have no idea where your application will be 6 months from now.

If you’re familiar with one of the cloud vendors, use it for your main startup unless you’re sure it will not meet your requirements.

When developing side projects, I highly encourage you to try and play with new cloud providers. Who knows, maybe you’ll fall in love with another.

Try DigitalOcean with $10 credit

Try AWS free tier