Cache Flushing Toilet Flushing – An introduction to what controller cache does, in a way everyone can understand.

The other day I wanted to explain to some people what forced cache flushing was and why it was a bad thing.  This happened while working a few hours helping a colleague tune a CLARiiON Array that was performing poorly because of this.  For some reason I decided that the best way to explain a technical concept to someone is to find something that they do in everyday life and make parallels to the technical concept I am talking about.  Fortunately or unfortunately to my colleague at work and later on my wife, I explained this concept using toilets.

All bathroom humor set aside, Cache is a great thing that most modern storage arrays have.  When things are working correctly, data gets written to cache, and acknowledgement sent to the host that the write is complete and then data is flushed out of the cache and written to disk.    In normal operation, this is a very good thing and allows data to be written in an efficient manner.  The problem can happen when you have a rouge host trying to do too many reads or writes to a group of disks that cannot keep up with what the cache is sending it.  This causes many problems to include Dirty Pages and forced flushing.  If this prolongs over a long period of time the whole array ends up writing most if not all data directly to disk to compensate for not being able to quickly write out to cache to send acknowledgement to the host.

Your toilet is a great thing that most modern households have.  When things are working correctly, “data” gets put into the toilet, and acknowledgement is sent to the host that your “data” is correctly in the toilet.  Once you acknowledge that the “write” is complete, you press a button and your “data” is now written to the City Sewer system.  In normal operation this is a very good thing and makes you happy when all goes well.  The problem comes when you have some very dirty pages, “data” that hasn’t gone to disk come checkpoint time and you need to flush again.  If it gets written to the city sewer the second time around, that is good but if it doesn’t you need to take out a plunger and “force flush” the “data”.  Now imagine a house full of 100 hosts that need to complete their “data” writing at the same time…

Now you see why force flushing and dirty pages are bad not only in your house, but on your storage array.    -Sangeek