Join Carahsoft and Riverbed Technology for an informed and engaging discussion led by Donald Kyzar, former CIO, Department of Justice’s Office of Inspector General and Marlin McFate, Riverbed’s Public Sector CTO about how federal agencies can cost effectively ensure the productivity, performance, and experiences of end users, applications, remote workforces, citizen-facing services, and IT investments with Digital Experience Monitoring (DEM) solutions.
Ensure the Success of Fiscal Year Spend by Focusing on End User Experiences
Corey Baumgartner 00:14
On behalf of Riverbed Technology and Carahsoft, we would like to welcome you to today's podcast, focused around The Success of Fiscal Year Spend and End User Experiences, where Marlin McFate Public Sector CTO at Riverbed, and Donald Kaiser, former CIO of the Department of Justice's Office of Inspector General will discuss how federal agencies can make meaningful IT investments in digital experience monitoring and end user experience monitoring to further their mission success.
Marlin McFate 00:41
As mentioned, my name is Marlin McFate, I am the Chief Technology Officer here at Riverbed Technologies Public Sector Group. We have a great topic today, we're gonna keep it conversational today. And without further ado, let me introduce him, Donald, go ahead, introduce yourself.
Donald Kaiser 00:57
Sure. I am Donald Kaiser. And I am a recently retired federal employee, I spent 26 years with the Department of Justice. 23 of those years were with the Federal Bureau of Investigation. And throughout that time, I worked in a number of different information technology positions throughout the FBI. And then my final position was as the CIO for the Department of Justice, Office of the Inspector General, for anyone that's kind of watched the news over the past couple of years, you've probably seen Michael Horowitz speaking at, had a couple of events up on the Hill, as it relates to DOJ activities. The thing that comes to mind, with all these positions that I've held was maintain operational focus the need for us to really have a true operational focus if we were an IT, organization. And that's a difficult challenge for us. In today's world, where the speed of the mission is putting enormous pressure on the information technology teams, and so for me, that is something that I've seen over time and in different positions. And I'm happy to talk to you guys about today.
Marlin McFate 02:30
Well, thank you so much for coming. Let's actually get started. I think that that was a really good segue maybe into one of the very first interesting questions. As you said, you've retired here recently, from Department of Justice. What are some of the biggest pain points? You know, you mentioned operational? Operationally, what are some of the biggest points that you've had to consistently overcome within that role?
Donald Kaiser 02:55
Sure. Well, what comes to mind right away, of course, is the technology challenges that we often encounter for various IT groups. But really, I think that the chief challenge is user expectations. The user expectations is what's driving us to do things like cloud migration, you know, that that really started as an initiative to drive economies of scale inside of the federal government with its purchasing of data center hardware equipment, but really, it transformed into something much more important. And that was operational agility, the ability for, for the government to keep pace with organizational demands and citizen demands at the same time. And so I think that that is the essence of what our challenges is trying to figure out how to how to satisfy those user expectations. In today's world, where I'm in the office one day, I'm at home, the next home on the road the rest of the week. And my expectations as a user from my IT team is that that environment should be same for me, regardless of where I'm located at as long as I have connectivity, my expectations or my services are going to be the same. And that's a difficult challenge for us to meet in today's world while maintaining security inside of the environment. And so, for me, I think that it is around managing that user expectation and really understanding what our customers need from us and then focusing on those, those are key elements of this. You know, other things that have come along recently is moving to Zero Trust. The ability to move to Zero Trust, everybody's talking about rolling, improve security. Well, there was a hidden message in this I want you to get and that is you're going to see improvement in the user experience, when people are using Zero Trust, because the throughput on it is better than you're going to get through a VPN connection through Cisco connect or something like that. Not that I'm an anti Cisco guy. But it's a reality that we've seen through testing is there we're seeing improved performance improvements that we didn't expect. And so that's the nice benefit that we're seeing that I think that those are kind of the main points Marlin.
Marlin McFate 03:11
Yeah, I would imagine actually, being the Department of Justice, you know, taking a look at a lot of our different customers. And it ranges right, there's certain federal customers of mine, that moving to the work from home environment was a complete change a 100% or 180 degree turn for them. I can imagine, though the Department of Justice, you had some experience with that with a, you know, people having to be in the field of probably one of the larger groups of mobile workforce, if you will, within the federal government. And so with that, you know, that spurred enormous amounts of modernization to kind of take place over a one or two year period of imagining you might have been a little bit of ahead of the curve on that one. But amidst those major modernization efforts, can you walk us through your biggest takeaways on what truly changed in agency IT environments with that?
Donald Kaiser 06:23
Sure, I think that what happened is we had an ability to demonstrate to highlight that it's not about a location any longer, it's about a capability, that, that information technology professionals can bring to the table. They can transform organizations in ways that nobody really thought about before, or they thought about it, but he didn't know quite how they're going to get there. I think that the last two years have afforded information technology components to really take big steps, big leaps, that would have taken probably 10 years, it was collapsed into two years organizations that maybe lacked that ability to bring to bring that good user experience to just about anywhere in the United States. And they were limited to their facilities. Well, they had to transform overnight. And so I think that that's one piece of the pie. I think that the other the other part of this is, the change in engagement with the customer base was profound. It has to be profoundly different than it was in the past that more and more information technology has got to be connected to this customer in order to satisfy those emerging requirements to understand them, and integrate them into an overall strategic plan of delivery of capabilities. I think that those are some of the pieces there that I believe are fundamentally important for us moving forward.
Marlin McFate 08:00
Absolutely. We sell it as well. But one of the customers we're at right at the very beginning, one that wasn't necessarily as prepared and had to really kind of go from a 100% on premise workforce to a 90%, obviously, probably can't get everyone there are going to be those positions where somebody actually has to be on site. But what we were finding was that, and I'm sure we'll get more to this later on in the conversation was, you know, the first thought was, especially from a performance perspective, or visibility perspective was let's throw bandwidth at the problem. Because the description we were getting from some organizations where a lot of their on premise applications. Of course, that's what drives towards SAS drives toward the cloud. Very shortly after that were completely unusable. The performance was actually that off compared to what it was when people were in the office. I'm sorry, go ahead.
Donald Kaiser 08:57
No, I, you made me smile, as you talked about that bandwidth issue there. You know, just kind of as a personal aside here, in the experience that I learned was, we had a lot of business applications that were at the FBI, we had some at the OIG where the user experience just wasn't good. And, and people that tried to solve that problem over time. And oftentimes what they were doing is they were they were throwing more bandwidth at trying to solve the issue with improvements in performance, they might see some incremental improvements there. Or they might put their business application or service on our classic server, whether it was one that they did in house, or they did it in the cloud. And, and still it might not have delivered the kind of true performance increases that they were seeking here. And oftentimes, what I learned from looking at this kind of broadly was, we weren't really looking at the user experience, we weren't understanding where the problem was. And because of that, we were returning to the tried and true, which was okay, well, we'll give you more bandwidth and see what happens. And so what I learned is that oftentimes organizations over invested in bandwidth, it's over invested in, in service computing resources. And it's masking the real problems. And that is that oftentimes, we've got business application problems, or we've got database performance problems, or we've got a combination of the two that are occurring. And if we fix those issues there, we can, right size, watching investment looks like for our network, or for our servers. And we're going to see economies and doing that. More importantly, we're gonna fix that user experience that we didn't really, significantly improve or change that we're those changes that we made in the past. Now, the only way you get there is you've got to have a good end user performance monitoring tool, that democratizes, that view of what that experience looks like. And what I mean by that is that it can't be something that just network team sees it, I want my help desk team to see it, I want my server team to see it. I want my business application and database teams to see what that performance looks like. And it gets me out of the cycle of having teams bounce a problem from one to another without really doing problem resolution. So I think that for me, having that kind of transparency in problem solving in a complex environment where maybe some users are at home, some are in the workplace, some are on travel, the ability to solve this issue, by having a good in user performance marketing tool is fundamental in today's world. We didn't talk about that up till now. Because everybody's been trying to Okay, I'm gonna get to the cloud, well, I gotta get Zero Trust implemented. But at the end of the day, your users need to have good performance business applications are not going to be happy with you.
Marlin McFate 12:20
Absolutely. 100%, I sometimes find it you know, it's, I won't say comical. But one thing that I see with a lot of organizations where I have in the past, you know, not as much now to your point, end user experience and monitoring that being able to quantify in some way, shape or form has become even more important, I think that we're all kind of realizing that. But in the past, especially before the pandemic, when we were working within these closed environments, private environments, teams would look at end user performance, trying to ascertain it through a different method, whether that be I'm looking at my network monitoring tools, and I can see things like response time for, you know, the network, I can see what the time is, and I can see what the response is for both the end user and the, the network. But that's in a completely private environment. And if you've implemented visibility correctly, in that environment, it's you can reliably sometimes, you know, say 75% of time reliably use that particular data to kind of try to ascertain what end user experience is. Love comical part was, was the one thing that we weren't looking at, even though we were all kind of, you know, beholden to end user experience as the measuring stick to whether or not you were doing a good job or not, the one place we weren't really looking at was at the actual end user. As the pandemic starts happening. And we move from, you know, a completely private environment. Now, we have applications in the cloud, we have applications that are SAS, these are, you know, not necessarily within our ability to control that environment, you know, to the same degree, we have end users that are places where we don't necessarily have control, and user experience and monitoring that being able to quantify it is just significantly more important. And so that might be the answer to the next question I had for you, which was where have you seen the biggest changes? And what is expected to create that reliable enterprise IT environment? You know, this is more the our agency missions.
Donald Kaiser 14:23
I think that some of the biggest things that we face today is this continuous change that's occurring inside of our environment, the rapid pace of software updates that occur inside of the environment, many which you may not even have direct control and release of those even as a member of the Enterprise Services team. And what that means is that I've gotten an environment now is constantly evolving. It's changing all the time. We see Microsoft zero day releases coming into our environment at a steady pace. Oftentimes they're, they're minimally tested by Microsoft, I'm not picking on Microsoft here, guys. So but they're kind of a classic illustration of what happens in that is that we are pushed properly. So I our cybersecurity arm within the Department of Homeland Security to get these releases out as fast as possible. And the vendors are out there and they're scared to death of a zero day hack. And so what they're doing is they're getting news releases out as fast as they can. The pace at which we're moving today means that we're pushing things into the environment with minimal amount of real test data across a complex environment. And, and the result of this is that we have to have a mechanism by which to have some confidence that when we push something, we know how it affected the environment, the only way that you can know if something affected the environment is if you previously were measuring the environment. And historically, we've not done a very good job of measuring the environment, an enterprise environment in the end. And so I think that, for me, the thing that has changed the most is that we have to think in terms of, it's not a static environment. And if we don't baseline what performance looks like, and then measure against performance over time, we can't logically identify five where a problem may exist, we can't identify if a Microsoft service pack release, has suddenly began to introduce anomalous things that were not occurring inside of our environment that 24 hours ago wasn't happening. And so I think that that's an aspect that is critically important today, is having the awareness of what's happening in your environment has never become more important for organization. Now, my apologies to you, if I if I did kind of like a politician here, and I took your question, and I changed it up a little bit. But I think that for me, I've lived this experience of what it's like when we release things into our environment, and you change that. And then we're trying to sort out complex problems that that exist. And we didn't necessarily have all the test tools that are available today, to help really to help your enterprise team look at and solve a problem a lot faster, and with a lot less anxiety about what the problem really is. And then the other piece of this is being able to communicate clearly what its plan of attack is in trying to solve a problem once it's been introduced into the environment. I think that by having a measurement tool in place, you can do those sorts of things.
Marlin McFate 18:12
Yeah, no, absolutely. What I'll add to that, it's actually kind of something that you said, kind of made me think of another area where this is, you know, very much so related to end user experience. But I've also seen oftentimes, especially in the past, I call it the death by 1000, paper cuts, right? You trying to implement these things, kind of one, as you said, without a really good understanding of what the end users experience is prior to making these changes. But then, I think sometimes groups think of things like modeling and simulation as a difficult thing to do, or it sounds very hard to do, or how hard to accomplish. And so therefore scary, when there really are the tools out there to take the information that we already have, you know, through end user experience monitoring, so we know what that end user experience is like, and then very easily build a model not, you know, piece by piece and having to, you know, program it by hand or something along like that, but take an actual model of my real environment that is built from my real environment. And then all of the other pieces, you know, how much application traffic for every specific user for every specific end user, and apply that to a model that I've changed to say something and again, not picking on Microsoft, but I think that every organization recently has made that change from you know, Exchange to Office 365 For example, right? But how do I model how do I know that by move from one to the other when it's going to be a different path? Where am I hotspot is going to be? Be able to model that for every single user. It's actually not difficult, and with both the understanding of what it was before If you can get an idea of what it's going to be, you know, afterwards, instead of just making a change, I had a customer recently come to me say that they had gotten to a point where they had moved over, say, 10s of 1000s, of course, employees, but they moved over something like 100,000 employees, and they were starting to have performance issues. And when they went back and said, Let's build this model and take a look at they saw that that was exactly what was going to happen. And the reason that it was starting to have performance problems was nothing to do with the application, but the environment, and then their lack of understanding of how it was going to affect it. So I'm sorry, long winded on that one. But it, your statement had brought that thought to me.
Donald Kaiser 20:39
So yeah, and I can't say I will say it the other way, where you introduce an application, it looks good in your test environment, you've put it through all your proper testing, you've got it out there, your users are starting to use it, performance is looking good. Until demand on the business application begins to be applied. A good example is a time and attendance system. But a lot of employees not really entering time and attendance information until a little bit later in the pay periods. And so that's an application to industry a really good example of use of it, our demand on it is low at the start of a pay period. But toward the end is being used to a great extent by by workforce. And so if you don't model your testing around that, then you're likely to experience what we did at the FBI, which was several weeks of pain of trying to get past a new software release, which had fundamentally changed workload was with the application, it was commercial software application, we simply implemented it in our environment. But what happened is that the of course the software development company changed where performance demand was at, and they moved it from the database layer up to the application layer. And we didn't understand that. So we were really hosting this thing in the incorrect environment. The moment that we change that to the correct environment, we fixed our performance problem. But of course, we had several weeks of real pain inside of our organization, as we went through this, what would have been very, very nice to have had was a simulation tool so that we could do synthetic transactions, and push it to the point where it couldn't go any further. And then we would have understood with clarity that we were hosting in the wrong environment, we would have known don't release this thing into the environment to really re hosted it into something different than the mainframe that it was writing. So. So lessons like that are painful for an organization to go through. I would say today, you don't have to live that pain, because there are solutions that are available, good solutions that can be used by your entire team, to drive organizational transparency. More importantly, it's not just driving the transparency inside of your own team. It's really about sharing that information with your, with your organization with your customers about this. This is what performance looks like in our system. It's not it telling you this or this is the real world and how well we're doing or where our challenges are. I think that the big attribute of this too is it does tell me when I've got a bad application. It allows me to focus on that and fix that problem. There's probably been there for a long, long time. Nobody ever really got to solve it.
Marlin McFate 23:51
Yeah, absolutely. What you just said, I have a feeling I know at least a partial answer to the next question. And I think we're going to be in violent agreement. But what are some of the biggest areas where the IT organization as a whole needs to think differently about responding to performance issues?
Donald Kaiser 24:08
I think that we have to understand, to respond as a unified team. We, we can't have it resting on the shoulders of our network team to try to tell the rest of the components of our organization, an enterprise IT organization, where the problem is that and so I think that it changes the nature of how information is shared inside of there. It also allows your enterprise IT team to be able to engage in a different sort of way with what I'm going to call shadow IT which that probably evokes some smiles and some frowns at the same time with our audience because the reality is you may have an enterprise organization but you've got to shadow IT running inside your organization as well with business applications and services that have been To find your individual components of the organization, and oftentimes, you get held accountable for it, because it's hosted inside your inside your enterprise hosted environment, but you're not responsible directly for that business application. But you get all the blame, when there's a problem. Wouldn't it be nice, if you're positioned to be able to have a different kind of conversation with your customer that is hosting that application, and say, you know, here's, here's the real deal with your business application performance issues, where before, they are simply able to point their finger and say, Okay, well fix this, and you don't have real clarity around what's going on with their application with a tool like this, which is an end user monitor, until you have that ability to do something like that, where before, you couldn't have had that conversation, you could have, but you wouldn't have been able to do it with fact based information that you can gain today with, with something like user mark.
Marlin McFate 26:02
You have mentioned a couple of times, now, I you know, the sharing of data, or, you know, the larger team, understanding or being able to view the data, and one of the things that you said to was hinted at all these groups are kind of measured, obviously, in different ways. You know, if I have the security team, I have the network team, the end user team, the database network, what have you. But in the end, really, what the major measuring stick for, if they're doing a good job is, you know, the, you know, the end user productivity and user experience, the their ability to do their job and such. And it's, and it's interesting that, oftentimes, we don't see I'll go into an organization to, you know, their, their knocks, let's just say, and there's different people, you know, working, they have the TVs on the screens, and you know, the network teams looking at a network problem, they're looking at network data. And there's a kind of unintentionally siloing themselves within this, this one realm, even if that data would be useful to another part of the organization or another team within the organization, say the SOC, they're not necessarily actively, you know, collaborating and sharing that information. And oftentimes, what I get is customers say I have problems, I know I have a problem, I just don't know where it is. And that problem usually lies within the periphery or the edges between two silos, right, or domains within the organization. So I like to I like what I'm hearing you say about, you know, fostering more of the sharing, we all have the same goal, we are all measured, in the end, by the same a measuring stick, or at least the same end goal. And so it only makes sense that we would be a sharing and the data as well,
Donald Kaiser 27:54
I'll say that I believe in that so profoundly, that what it did is for all of the supervisors for my entire division, at the OIG, their performance measures were one of the same, and they were all in user base performance measures at large. And, and by sharing that responsibility and sharing that ownership, it forced more of a team dynamic of what I expected. The same held true even though at the working level, which was even at the working level, across different components of it, there were shared obligations that every person had with regard to trying to improve the user performance environment for us as an organization. And so and so I speak to this because culturally, there are things that we can do as an IT organization, to try to help ourselves in instill that kind of behavior corporate with ourselves, because it's not a natural behavior, the what we've been trained to do for many, many years is okay, your network team, your server, team, your, whatever you are inside of the organization. And really, we want to move away from that and view ourselves all as having a collective responsibility for trying to improve that user experience.
Marlin McFate 29:21
I'm really, I'm really glad to hear that that's one of the things that I speak about, quite often about is you know, tear down silos and start to develop domains. You're never going to get away from domains you need domain expertise, there can't be one vanilla skill set across the entire organization. But starting off with measuring everyone with the same measuring stick eliminates things like the desire to do mean time to Innocence rather than mean time to resolution. You know, getting in a room, we're all gonna get in a room and we're all going to, you know, manually correlate this data until we figure out what the problem is, you know, war rooms and such. And I find that that mentality shift that you're describing eliminates quite a bit of that with an organization, so.
Donald Kaiser 30:06
Yeah, it really, it really moves you out of that reactive mode, you become much more of a proactive organization. And you combine that now with, with a good tool that allows your team to, to be able to delve into where the problem is. And you're going to change the outcome for your user base in a significant way, they're going to see less outages, because you're going to identify on before they actually happen, you're going to identify problems before it becomes an outage. And then you're going to do a corrective action. And then when you do have a problem inside your environment, more likely to resolve it faster than and under the old model that we always worked under.
Marlin McFate 30:46
Very much so agree. So talk to me about the importance of when we've touched on this, you know, a little bit, maybe a little bit more in depth touch on the importance of benchmarking product performance across an enterprise environment and why that information may be critical. Like I said, we've touched on it, but it may be critical for multiple IT teams within their sorry, that are involved with it within the organization. And how that relates to continual product, performance improvements.
Donald Kaiser 31:19
Okay, well, with everything that we seem to do in this world today, it's benchmarked in some way. You know, you lay down in a smart bed, it tells you how you slept at night, you know, you go through your day using different tools, that tells you how things are performing. And so it's critically important for us, as an organization to really understand how our services are performing on any given day. And without a benchmark. Again, we're going to go back into the we're not quite sure where our problem is they act with a with a business application problem, or a database problem that may be occurring with certain database queries, but not with all database work. And so, so I think that it, it is really important that we understand that the more clarity that we have around how our systems perform on a regular basis, and that we measure performance over time. In other words, I'm one on one to compare this information against how it was performing 30 days ago, and three months ago. And with that, now I've got trend data that allows me to look at and say, Okay, well, maybe I've got a problem that is occurring in some of our design characteristics of our application, maybe we've got some core SQL queries that we've developed over time, we need to go in and clean that up. So allows me to do that. But also, if you want to advocate for change inside of your environment, it's best to actually have performance measurement data to go with. If you want to make an argument for changing from one type of a business application to another, or you've got a customer that wants to do that. If you can't resolve performance characteristics with a commercial application, I can't think of a better reason to go look for something different than I can look at it and tell you that it performs badly in our environment. And there's no way to resolve it, because it's just a bad application design. That's going to cause you to really want to go on look for something different watching. So I think it helps in decision making moralizing that that's one piece of it.
Marlin McFate 33:44
So that actually kind of begs the question. So if an organization wants to correlate an ROI set for, you know, an IT investment that focuses on improving application or network performance, workforce productivity, what have you how I think that's part of it, but broader picture, how would one go about measuring?
Donald Kaiser 33:44
Okay, the return on an investment for a business application is really measured in productivity. And so what I want is a one application is, is performing well, that the user is not clicking the Send button and then having to standby for minutes while some transaction is occurring in the application, whether they're in the office or on the road. And so return on investment years, it can be measured in workforce productivity for each one and your business applications. And so that's really the largest expense that we have inside of an organization is a workhorse. And so wherever we can do performance increases for business apps, then we're going to see a return on investment there and we should be able to measure that I should be able to telling you that, okay, I have improved the speed of the system. And now instead of having an analyst or an agent that's sitting there waiting for a file upload that in the past might have taken 20 minutes from some remote location, it can be done in five. That's measurable time that you can equate that to performance. So your workforce. And I think that sometimes we nit, we kind of missed the point, we talked about our technologies. But we're able to demonstrate that we can improve performance for our organization through workforce productivity. That's a powerful state, it really equates to organizational leadership, and being willing to invest in tools and technologies that allows us to do things like that.
Marlin McFate 35:53
Absolutely, I can remember back, back when I was an SEO, so many years ago, and age myself here, I can remember I had built an ROI for an organization. And one of the things that I added in there was productivity. I mean, you're absolutely correct. I mean, the more productive that we can be the better. But from an ROI, you know, without being able to do something like monitor and user performance, and therefore their productivity, it's very difficult to utilize that as a solid ROI. It's always kind of called a soft, you know, ROI. I could remember I think at one point in time, I said, you know, I'm gonna save everyone a minute, a day, but only one out of four people was actually going to take advantage of that one minute a day, like two or three other people would wasted. I don't know what the water cooler, what have you. Because it was soft, it was very difficult to utilize that, even though it is what we're all looking to achieve, very difficult to utilize that in ROI. By doing this, would you agree that correlating that productivity ROI is now no longer, you know, the matter of being a soft ROI item, it can actually be considered a very hard and firm, this is the productivity gains that organization this organization is getting from these people on this application.
Donald Kaiser 37:00
Yeah, I think that it becomes very easy to be able to demonstrate that. And what it does the most important piece of this at the same time as it transforms that relationship with you, with your it and your customer base, because they feel it, they sense the approvement. And it's appreciated. And oftentimes, you know, what happens is, it only hears the negative news. But when you can begin to, to put capabilities in place that can change performance of a business application and improve, improve the Workday for your for your average employee inside of the organization. It's a measure of productivity, but it's also a measure of workforce satisfaction that comes into play. And let's be honest about this, if you're an IT, there's a lot there a lot of days, you're not being thanked for what you do. And if I used to know a DoD, IT leader and he used to say if you're if you're in the business of looking for thank you’s, and it, you need to pick a new career field. But I think that what has changed since he's made those sorts of statements was, we do have the ability to change the environment for our employees, and make it to where they can be satisfied with delivery of goods services, by IT, because we can measure what that performance looks like. And we can change it because we can measure, we can improve services that would have otherwise been hidden for promise.
Marlin McFate 37:10
Right. And we'll make changes we can immediately know whether or not that change was, you know, a positive or a negative to you know, the end user experience and their ability to be productive and, and such. Go ahead.
Donald Kaiser 39:07
I was gonna say this, you're probably sensitive for me that my view of the relationship between it and the customers one where it's hand in glove. And I think that this is always an essential element of the success of any IT organization is direct customer engagement, continuous feedback through accepting of criticism, whether fair or unfair, and, and then being able to action where we can do things that will logically improve performance over time. Inside of the system. Some steps can be big steps, some are going to be small. But the objective here by continuing to focus on that end user experience and improve the quality of our business applications and services. That that that that end user is experiencing, we're going to change the dynamics of the relationship between it and the customer base in a meaningful way. So with that, how should CIOs and IT teams think about the gaps in their capabilities, applications and our systems? And how does one go about thinking about where to invest? Well, I think that for many of us, that were in the CIO, see, what we were looking to do was to, in many instances, get ourselves moved to the cloud, because it offered agility, we wanted to do things like try to try to implement Zero Trust, and, and do things like that. And today, I think that the investments are more around tools that give us clarity about what's happening inside of our environment, to try to improve that experience. But what we need, of course, is to be ready for the this time of year, especially for the calls that do happen inside of organizations, when your CIO is called and told, okay, we've got $5 million? And do you have a business plan that can address your most critical need at this point, if I give you that $5 million, can you execute, and oftentimes, IT organizations are caught a little bit flat footed, when those calls happen, although it shouldn't occur, because it happens over here. But if you are read, you already know what that investment is going to look like. And you've already socialized with the leadership team around what it is that you want to invest in to try to improve the customer experience, then you're going to go far and get into funding resources that might come towards the end of the financial year, as we're approaching right now. So my counsel would be that regardless of your position and sign of an IT organization, it's incumbent upon you to identify those critical needs that you have right now, and to develop for yourself and for your organization. A sound business case, it doesn't have to be big or complicated. It just needs to communicate what that need is. So when, when your leadership team is asked about what can and execute on right now, try to improve the business environment. You've got a plan in place, right now.
Marlin McFate 42:42
Yeah, I've heard of it. Have the story ready. Right? you know, you, you, you, you just know what the problem is. And you can describe it. Well, you can describe what the solution is why that is the solution. And then more importantly, what is the effect of not doing, you know, X, Y, or Z? is, it's an important skill. And it's also important when talking to leadership. I would say that it doesn't happen by happenstance, either there's some strategic planning that goes into this effort, some strategic thought that goes into it. And really a dialogue between the employees inside of an enterprise organization and their leadership team around what is fundamentally needed here in order to transform that business environment, how to improve that customer experience. Absolutely. So with that, I think that we are getting close to quest question time here. So Brian, I'd like to invite you to come back on and start us off with some of the questions that are in the Q&A.
Brian Lake 43:47
Sure thing, Marlin, thank you, again, both Don and yourself. So we do actually have some questions coming in, they're looking pretty great. So I'm gonna go ahead and ask the first one. So we keep speaking about identifying and resolving issues by implementing changes, but how quickly can these changes be actually manifested?
Donald Kaiser 44:07
I think that the changes I touched on this and our current environment is never static any longer. For an enterprise IT organization, we have releases that are occurring continuously inside of them. That's a reality. And that that's all going to increase over time. And so, so that's, that's here to stay. And, and so really, at this point, Brian, I think that the answer on it is be ready for this new world order. In that regard, by arming yourself with capability allows him to keep up with that by understanding what's going on inside your environment.
Marlin McFate 44:49
So the way I am going to answer this may slightly differently just rereading it to make sure I understand here so we can keep speaking about identifying, resolving issues by implementing changes, how quickly can these changes be manifested? So I think of the what I would add to that, Donald is one of the things that we've, we've in it have kind of all, I think, thought at one point in time, and probably still do is that oftentimes, especially with our very large agency networks, these massive environments is they're not necessarily the most flexible and agile thing. And they're not flexible, because, you know, little tiny changes, you know, can cause unforeseen problems, we've got plenty as probably very smart people on this call, we can probably think of every failure scenario possible that could happen during a change or what have you. But they're still pretty fragile. But what we're trying to move toward by implementing these different changes, is one to continually provide excellent end user experience for our end users. But then also, these changes should be moving us in a direction of creating a more agile and flexible environments. So what I would say is, is to this is identifying and resolving issues by implementing changes those changes. Oh, and how quickly can manifest what we would hope is, is that if we're doing things in the right way, each one of these changes that we're talking about is going continually allow us to move even quicker, and manifest changes, or the right changes even quicker as we move into the future.
Brian Lake 46:21
Yeah, okay. Second question here for both of you. Are there any specific challenges when it comes to measuring and user experiences, if my and users will be shifting between the office or their branch office and then from working remotely in their home?
Donald Kaiser 46:39
The short answer on this is that a good end user performance, my tool is going to work, regardless of your location, whether you're in a in a facility connected directly to your network, you're working from home are your own road somewhere, it should be something that allows me to measure performance, regardless of where you're at.
Marlin McFate 47:01
Yeah, I'll add to that, I'll say that it shouldn't, right, it should be able to do both, if you're using the right tool should be able to do both. What is needed in that particular instance, is something that understands one, that that end user is now in two different places, or that devices in two different places, and being able to apply, you know, the correct set of analytics to understand, you know, I, I've always said this, was talking to a DoD customer, you know, email in Afghanistan, is going to have a very distinctly different performance, you know, set of performance characteristics than say, if I'm sitting on a base in Texas, right. So and then the, the, the system has to understand that those are two different environments, and be able to provide feedback to that end user experience based on the environment that that person is in. With that said, we should be trying to make the end user experience in any one of those environments better, again, with the changes from the last question, as time moves on.
Donald Kaiser 48:05
I would, Marlin, I would agree with you. The example that comes to mind is that within the FBI, we had about 65 different locations around the world Liga program, and there were unique challenges with bandwidth connectivity to a lot of those lead, get locations around the world. And in specific instances like that, your performance measurements are going to look different, you need to account for that. And, and recognize that there are situations where it is going to be a different type of an environment and you need to be able to measure against that specific environment.
Marlin McFate 48:50
Absolutely, it would, it would, it wouldn't be a good thing for an end user monitoring solution to say, look, Tom's email is doing great right now. And then he goes, I don't know on a trip and he's at a hotel. And now it's less and what am I going to do generate an alert to that we would be inundating our IT teams with a bunch of useless one data and alerts that up that don't need to be there needs to be an understand those are two different places and what good looks like and each one of those.
Donald Kaiser 49:19
Yeah, but the one truth that holds true in this is that it shouldn't be a tool that will work regardless of location. Yes, agree.
Brian Lake 49:29
So next question I've got here, can I build dashboards to constantly evaluate and quantify my end user improvements? And is this something that should be shared amongst other line of business teams or how do I build advocates to cross leverage this data?
Marlin McFate 49:48
There's a couple of questions in there. Do you want to take for shot Donald?
Donald Kaiser 49:51
Yeah. Okay. I will definitely take a first crack at this. One of the biggest things that I always saw was to actually have a good set of KPIs that I can illustrate what performance look like for our business services, regardless of where it was located at. And a dashboard view of what a system performance looks like, is very, very valuable for the IT team. But it's very valuable in terms of informing leadership inside of our organization, what performance looks like, I can't stress enough that many of our challenges that we have with our customers is that lack of transparency, and we're in we're able to break down the barriers that separate us by being transparent with the information that we're getting us and IT workforce through dashboards that are understandable. We're only going to improve our ability to get the kind of investments we need to continuously improve it. That's my take of it.
Marlin McFate 51:01
Yeah, I would, I would agree as well. I'll take maybe a little bit more of the second piece, but I'll bring in something that you had talked about earlier, Donald which was, you can sometimes look like a sunk cost are a necessary evil. To your point earlier, where it was saying, we said, the good things that are done are necessarily, you know, spoken about or shared with the rest of the organization. I feel that if we're looking because we've talked about some cultural changes here, in both thought process and mentality, I think this all goes in that particular direction. We can bring ITG teams to be in collaborative teams, as opposed to siloed teams, collaborative domains instead of silos. But that cultural change also needs, you know, a very positive backing. So, you know, just letting the rest of the alleged other lines of business, know that that it is understood that there is a problem, this is what we're doing to work on it. These are the improvements that we've made. But then also historically, to be a say, here are all the improvements that our investment in it have made and user experience to your constituents to your end users. And sharing that throughout the organization, I think is very important.
Donald Kaiser 52:13
Yeah, it builds relationships.
Marlin McFate 52:16
Yeah.
Brian Lake 52:17
Okay, I think we have time for one last question. And then we will go ahead and close out. I've been advocating for examining this type of data for several years, can you go into greater detail on what I can do as a network administrator, to convince my leadership that this is an important investment?
Donald Kaiser 52:34
Sure, I think that the best way to describe an investment like this is in terms of how we're going to transform the user experience inside of the organization, we transformed ourselves by moving in the cloud, we've done things to improve security around our data. And it is really time for us to focus on how to make the user experience inside of our government systems as good as what you might have, if you're sitting there, and you're connecting to your bank, through your through your local network, or to something else, you want to be able to highlight where you're at, and what the challenges are in the system. So that the organizational leadership is going to know what resources that you need, you're going to be able to communicate it, they're going to be able to see it with clarity around where we can take steps to improve services inside of the organization. Today, keeping our customers satisfied is a critical element. In my opinion, it's something that if we don't do that, then they're wanting to look for solutions outside of either the enterprise organization that you're part of, or the team that you're part of. That's a reality. And we shouldn't be surprised by that today.
Marlin McFate 54:03
And in the interest of time, because we are here one minute left, what I'll say just to add to that as a, you know, a former network engineer, at one point in time, you know, I was said what packets don't lie, right? If you're a network person, you've probably heard that before. That's very true. But the measuring stick that we're all measured on is end user experience. Regardless of what portion of the IT organization you are, at least it should be, and end user experience doesn't lie.
Corey Baumgartner 54:31
Thanks for listening. If you'd like more information on how Carahsoft or Riverbed Technology can assist your federal agency, please visit www.carahsoft.com, or email us at riverbedmarketing@carahsoft.com. Thanks again for listening and have a great day.