I have clients that ask me to look at their environments all of the time, citing that “it just feels slow.” In other words, they believe their environment is performing poorly, but they usually don’t have any baselines to compare to or any statistics to go off of.
It’s a commonly reported problem that “everything is too slow” even though there are no specific examples highlighted or pointed out.
Regularly when investigating these types of complaints I find their storage latency averages around 25ms or more. Even though we would most likely agree that 25ms is above acceptable average latency, the client is perfectly happy with their current results and claims storage performance isn’t a problem. I find this fascinating as I think it also relates to the tech industry as a whole.
For example, when I upgraded to the new HTC M8 smartphone, I immediately noticed a performance increase. However after several weeks of use, I wasn’t as aware of it, as I became accustomed to the new normal. It wasn’t until I picked up my son’s older iPhone 4 and started using it that I noticed again the performance and usability improvements my phone really has.
The old saying, “You don’t know what you’re missing until you try it,” can also apply to storage and application latency. This got me thinking about how IT professionals should handle these types of situations.
Application I/O workload characteristics are one of the most difficult challenges in today’s virtual datacenter. The true challenge as a virtual admin is to understand the full workload pattern over the application lifecycle.
Many believe that they understand their workloads…at least until something goes wrong or users start screaming that the report they run every day now takes twice as long. Their understanding of the workloads is then challenged by these issues. (Hopefully this comes in a constructive manner and not from the boss with a why-did-you-let-this-get-so-bad conversation.)
There are a ton of reasons this situation happens. It’s not that the admin doesn’t care about user experience. There are a few very common reasons an IT team may lack this knowledge: lack of monitoring tools, software development and/or stakeholder communication disconnects, unexpected growth, etc. As an application owner, how do you know what the app is capable of delivering? Do you know what your end users are expecting from it, if anything at all?
The admin may not know what his users’ perception of performance is or have a capable tool to help him determine the actual reality. This is important in order to verify if a user’s perception has merit, whether it be regarding performance or function.
How do you address these perceptions and misconceptions?
- Establish good baselines and document them to keep track of how your environment runs. For example, X process takes 25 minutes to run.
- Have monitoring tools that provide the appropriate data to make informed decisions.
- Rely on metrics that don’t hinge on human activities and perceptions. Asking a user how fast they felt their workstation was is not a good metric; everything needs to be quantitative. Refer to your established baselines.
It’s my hope that in 2015 we can test and challenge our perceptions so that we can have a more accurate understanding of our environments.