Skip to main content

Rethinking the way we build on the cloud: Layering the Cloud


To help, you can split your cloud architecture into three layers:
  • Visible
  • Volatile
  • Persistant

Visible

This is the layer between the cloud and the rest of the world. It is mainly public DNS entries, load balancers, VPN connections etc. etc. These things are fairly static and consistent. If you have a website you will want the same A Name records pointing at your static IP.
Things in the visible layer rarely change or if they do they are for very deliberate reasons.

Volatile

This is where the majority of your servers are. It’s called volatile because it tends to change a lot. The known state (how many servers, what versions of which software they run etc.) are changing frequently, perhaps even several times per hour.
Even in a traditional data centre your servers are being upgraded, having security patches applied, new versions of software deployed, extra servers added etc. On the cloud this is even more volatile when you use patterns such as the Phoenix server and machines are rebuilt with new IP addresses etc.
You should be able to destroy the entire volatile layer and rebuild it from scratch without incurring any data loss.

Persistent

This is where all the important stuff is kept, all the stuff you can’t afford to lose.
Ideally only the minimum infrastructure to guarantee your persisted state is here, so for example, the DB server itself would be in the volatile layer but the actual state of the DB server, its transaction files etc. would be kept on some sort of robust storage that is considered ‘permanent’.
By organising your infrastructure around these three layers you are able to apply different qualities to each layer, for example the persistent layer would require large investment into things like backup and redundancy to protect it whilst this is completely unnecessary for the volatile layer. Instead the volatile layer should be able to accommodate high rates of change while you will want to maintain a considerably more conservative attitude towards the persistent and visible layers.
Stay tuned for the next part where we analzye the issues with the traditional approach to environments on the cloud.

Comments

Post a Comment

Popular posts from this blog

Real-Time Talk: Windows 10 IoT Core Background Tasks and ASP.NET Core Web Apps

Display useful information from your Windows 10 IoT Core application in an ASP.NET Core web app, essential for integrating IoT data into a solution. Windows 10 IoT background task talk with a web application using WebSockets. Problems As my path to this solution has been troublesome, I am listing here the main problems I faced so my dear readers have a better idea of dead-end streets along the way: I was not able to make the ASP.NET Core web application run under a Windows 10 IoT background service. I found no information about when or if it will be supported in the near future. ASP.NET MVC and ASP.NET Core have different SignalR implementations. I was not able to make a SignalR client for .NET Core work with SignalR hosted on a web application. I was able to make things work by directly using a WebSocket. It’s not as nice a solution as I had in my mind, but it works until things get better. Making the Background Task and Web Application Talk I worked out sim...
Post a Comment
Read more

Python and Parquet Performance

In Pandas, PyArrow, fastparquet, AWS Data Wrangler, PySpark and Dask. This post outlines how to use all common Python libraries to read and write Parquet format while taking advantage of  columnar storage ,  columnar compression  and  data partitioning . Used together, these three optimizations can dramatically accelerate I/O for your Python applications compared to CSV, JSON, HDF or other row-based formats. Parquet makes applications possible that are simply impossible using a text format like JSON or CSV. Introduction I have recently gotten more familiar with how to work with  Parquet  datasets across the six major tools used to read and write from Parquet in the Python ecosystem:  Pandas ,  PyArrow ,  fastparquet ,  AWS Data Wrangler ,  PySpark  and  Dask . My work of late in algorithmic trading involves switching between these tools a lot and as I said I often mix up the APIs. I use Pandas and PyArrow for in-RAM comput...
Post a Comment
Read more

DANGER (STILL) LURKS IN THE INTERNET OF THINGS (IOT)

All of the media, both “mainstream” and “tech”, has gushed over all of the new appliances and devices that are now in the category of what we would call the Internet of Things.  Items like home security,  home lighting, and refrigerators, to name a few. There are many advantages to having connected appliances and devices, Threats that can and will be exploited if unsuspecting users don’t secure them.   Two “layers” of security : The first layer of offering  is a security API that will provide [a way] to easily do a virtual patch, to prevent a remote attack, for example . . . the third layer is cloud: IoT cannot do anything without the cloud.  Most data is sent to the cloud and you will need to have proper protection and make sure the cloud is always available. In both situation Users are vulnerable, mostly due to their own apathy.  Users often either don’t know how to patch their own machines (and in this case, devices) or have glanced...
Post a Comment
Read more