Month: November 2010

Storage Terminology

I sat out to write a post about choosing the right storage. I found that so many of the topics that I wanted to discuss required some terminology explanation that I decided to make this post just about that. Stay with me for the next post when I’ll get into choosing actual storage.

Host – The server or application which will access the storage.

File System – This is the part of the storage system responsible for managing storage resources and brokering read and write transactions. It handles such things as block and file locking to ensure that no two systems are trying to write to the same parts of the disk at the same time. It also stores metadata about files such as which disk(s) they are located on, and where; filenames, sizes, etc.

SAN vs. NAS – These are the two [main] different ways to access shared storage systems. It is easiest to understand them in relation to each other. The key difference is where the file system resides.

– In a NAS, the file system is on the storage system itself. This allows multiple hosts to naively access data without worrying about all the technicalities mentioned above. The host can read and write to files blindly, content in the knowledge that the file system will not allow it to make a mistake. The downside of NAS is that it does not offer the application the ability to control the disks directly, and therefore the application cannot tune disks access to it’s own ends. Also, some of the helpful things that file systems do – like file locking – may pose a problem for certain types of applications.

– In a SAN the file systems resides on the hosts or applications that are connecting to it. The hosts themselves must carefully manage disk access between themselves in order to prevent data corruption. The is a fairly high development burden, so only applications that can truly benefit from highly customized disk access usually support it. In other cases it may be an explicit requirement that multiple hosts be able to access the same shared files. Some common ones are databases and various sorts of HA clusters, like with VMware for example.

Controller – This is the device that contains all the brainpower for a storage array. It’s really a suped-up, custom-built server, specifically designed for managing storage. It contains CPUs, RAM, and various storage-related add-on cards. This is where the file system will run, and all the disks will connect to it. They are often paired-up in a single chassis called a head unit to form an HA storage cluster. In large array clusters there may be multiple tiers of head units.

Disk Shelf – The disk shelves are the devices that hold the disks. They have little-to-no brainpower at all, and simply serve to power up the disks and pass-through all the read/write operations.

Three-tier architecture

I’ve been doing some thinking about application architecture and some of the problems that I’ve seen with that during implementation and development.

A three-tier architecture is often used in the context of web site development, for a couple of reasons:

  • It creates a clear separation of roles for each server component
  • It allows abstraction of each piece
  • Each role can be improved, updated, upgraded, or wholesale reworked, without impacting the rest of the system
  • Redundancy is easier to build in on a function by function basis

These are all great reasons, and most successful web properties do this in some fashion. The tiers may vary in number and type, but the goal is functional separation. Despite these worthy goals, often things go awry during implementation and ongoing development.

Take the case of a typical web-app-database structure. The user makes a request of the web front end. The web layer passes the request to the app layer for processing, which usually entails a call to the database. The user cannot interact with anything other than the web layer, and only the app layer can talk to the database.

At some point, another feature is required – and quickly. The only way to get it done right now is to make an exception (just this once) to allow the web layer to make a database query directly. The tier-structure is broken. A few months down the road, a dozen one-time exceptions later, and the whole thing is one big mess. Now you can’t really tell if the redundancy you thought you had is really there. You’re afraid to change any of the pieces. What’s worse is that you’re equally afraid to fix anything in case your “fix” breaks something else.

Another thing that I see happen quite often is to confuse a logical, functional tier-structure with one that is hard-wired into the network. This need not be the case, but it is often done this way. One can certainly develop the application with a multi-tier architecture, but that doesn’t mean this needs to be built into the network.

There are a couple of good reasons to segregate your network as well:

  • You have a very large network and need to deal with bandwidth and addressing issues
  • Security

For most of the small to medium sized networks that I’m talking about here, only the second of those will apply.

Unfortunately, there are also few good reasons not to segregate your network this way that aren’t taken into account:

  • The security you are trying to gain proves to be illusory. In most cases, the security breach will be a result of an application vulnerability rather than a network one.
  • If your web server is compromised, a malicious user is a hop-skip away from everything else. The network can’t provide any safeguards in that case. It must allow traffic from your web server to your app servers in order for your application to function. Again, you have to rely on a properly secured application to thwart the bad guy from digging deeper.
  • All of your internal traffic is now going through a router or firewall that will cost much more per Mb of traffic than if you did not do this. In most cases the vast majority of traffic is internal traffic that never leaves the application. You can buy a switch with a 48Gb backplane for under $2,000 today. A firewall that can pass 1Gb of traffic will cost over $50,000. And don’t forget the standby unit.

Despite all the good reasons not to segregate the network into multiple tiers, I still see this attempted all the time. I think it comes down to application developers not wanting to sacrifice development cycles for the sake of something that’s not a tangible feature. So by segmenting the network, which is conceptually easy, they can appease the security gods. Then when something goes wrong it’s a problem with the network.

My advice: Multi-tier architecture is great when enforced on application design, but not when applied to small-medium local networks. (Yes, that was a rather large qualification, but I want to be clear we’re not talking about WANs or campus-wide, inter-departmental networks here. We’re talking about web applications.)