Forecasting the projected volumes is one of the key elements for work load modeling. The forecasting process is a combination of using judgment and application of mathematics. The mathematical process takes past history and uses it to predict future events. Both these components should be used in order to come out with an accurate forecast.
For a Performance Engineer to intelligently negotiate an SLA, he or she must understand business- related concepts such as customer/employee population, inventory stocks, and business transaction rates. This kind of volume data is expressed in business units, the language of the user community. Performance Engineer must be able to communicate with the users in these terms. However, they also must be able to translate business units into measurements related to intensity of service demand (e.g. system arrival rates, transaction-processing time, etc.). Furthermore, these must be translated to system performance metrics such as utilization, queuing levels, and response time. In addition, Performance Engineers should realize that business trends are changing over time and that business growth rates will cause changes to system requirements. Therefore, estimating the computing needs of an organization requires application of forecasting techniques and relating these to the system performance metrics mentioned above. The combination of these skills form a discipline called capacity planning, is the determination of predicted future system needs which impacts system acquisition decisions and service level expectations.
Workload Forecasting Techniques:
There are two broad categories of forecasting techniques:
1. Quantitative Methods
Quantitative methods are based on algorithms of varying complexity. Relies on the existence of historical data to estimate future values of workload parameters
2. Qualitative Methods
Qualitative methods are based on educated guessing analogy/commercial knowledge. It’s a subjective process, based on judgments, intuition, expert opinions, historical data etc.
Quantitative Versus Qualitative:
Quantitative forecasting looks more into statistics and past trends to make predictions, whereas qualitative analysis relies more on managerial or judgmental opinion. Qualitative analysis is also often used when quantitative data is absent.
Quantitative forecasting techniques are generally more objective than their qualitative counterparts which are more subjective. Qualitative techniques are more useful in the earlier stages of the product life cycle, when less past data exists for use in quantitative methods
For a Performance Engineer to intelligently negotiate an SLA, he or she must understand business- related concepts such as customer/employee population, inventory stocks, and business transaction rates. This kind of volume data is expressed in business units, the language of the user community. Performance Engineer must be able to communicate with the users in these terms. However, they also must be able to translate business units into measurements related to intensity of service demand (e.g. system arrival rates, transaction-processing time, etc.). Furthermore, these must be translated to system performance metrics such as utilization, queuing levels, and response time. In addition, Performance Engineers should realize that business trends are changing over time and that business growth rates will cause changes to system requirements. Therefore, estimating the computing needs of an organization requires application of forecasting techniques and relating these to the system performance metrics mentioned above. The combination of these skills form a discipline called capacity planning, is the determination of predicted future system needs which impacts system acquisition decisions and service level expectations.
Workload Forecasting Techniques:
There are two broad categories of forecasting techniques:
1. Quantitative Methods
Quantitative methods are based on algorithms of varying complexity. Relies on the existence of historical data to estimate future values of workload parameters
2. Qualitative Methods
Qualitative methods are based on educated guessing analogy/commercial knowledge. It’s a subjective process, based on judgments, intuition, expert opinions, historical data etc.
Quantitative Versus Qualitative:
Quantitative forecasting looks more into statistics and past trends to make predictions, whereas qualitative analysis relies more on managerial or judgmental opinion. Qualitative analysis is also often used when quantitative data is absent.
Quantitative forecasting techniques are generally more objective than their qualitative counterparts which are more subjective. Qualitative techniques are more useful in the earlier stages of the product life cycle, when less past data exists for use in quantitative methods
Quantitative methods come in two main types:
Time-series methods
Time-series methods make forecasts based purely on historical patterns in the data. E.g. you want to forecast site visitors over the next few weeks. Time-series methods only use historical site visit data to make that forecast. Time-series methods are probably the simplest methods to deploy and can be quite accurate, particularly over the short term. Most quantitative forecasting methods try to explain patterns in historical data as a means of using those patterns to forecast future patterns. It can be further divided in to
Last period demand (often called the "naive" forecast)
Arithmetic Average
Simple Moving Average (N-Period)
Time-series methods
Time-series methods make forecasts based purely on historical patterns in the data. E.g. you want to forecast site visitors over the next few weeks. Time-series methods only use historical site visit data to make that forecast. Time-series methods are probably the simplest methods to deploy and can be quite accurate, particularly over the short term. Most quantitative forecasting methods try to explain patterns in historical data as a means of using those patterns to forecast future patterns. It can be further divided in to
Last period demand (often called the "naive" forecast)
Arithmetic Average
Simple Moving Average (N-Period)
Weighted Moving Average (N-Period)
Simple Exponential Smoothing
Calculating Multiplicative Seasonal Indexes
Simple time-series methods include moving average models. In this case, the forecast is the average of the last "x" number of observations, where "x" is some suitable number. If you're forecasting monthly sales data, you might use a 12-month moving average, where the forecast for the next month is the average over the past year.
E.g. Ft+1 = Yt + Yt−1 + · · · + Yt−n+1/ n
where Ft+1 is the forecast value at time t + 1, Yt is the observation at time t and n is the number of observations used to calculate Ft+1
Trouble is, simple averaging methods don't tend to work well when there's either a trend in the data or seasonal effects. E.g. In case of marketing data, other techniques such as exponential smoothing may be more appropriate.
With moving averages, every data point carries equal weight in making the forecast whereas in case of smoothing methods, more importance is placed on the most recent data than on the historical data. If there's a trend in the data, it'll use the recent observations to make up the bulk of the forecast, and the forecast is more likely to reflect the trend.
Moving averages and simple exponential smoothing techniques are available in Excel and easy to execute. That's part of the great advantage of time-series methods: they're generally simple, cheap to run, and relatively easy to interpret.
Explanatory methods
Explanatory forecasting methods take the process a step further and allow us to relate changes in marketing activity to changes in such outputs as sales, brand awareness, and registrations.
Explanatory models assume that the variable to be forecasted exhibits an explanatory relationship with one or more other variables. For example, we may model the electricity demand (ED) of a hot region during the summer period as
ED = f (current temperature, strength of economy, population, time of day, day of week, error)
The relationship is not exact—there will always be changes in electricity demand that cannot be accounted for by the variables in the model. The “error” term on the right allows for random variation and the effects of relevant variables not included in the model. Models in this class include regression models, additive models, and some kinds of neural networks.
The purpose of the explanatory model is to describe the form of the relationship and use it to forecast future values of the forecast variable. Under this model, any change in inputs will affect the output of the system in a predictable way, assuming that the explanatory relationship does not change.
Qualitative forecasting methods can be categorized further as below
Delphi method: It is an iterative technique for obtaining a consensus forecast from a group of experts, without the problems inherent in group decision-making. The procedure works as follows: first, give a set of questions to each expert, who provides answers (forecasts) independently from the other experts. The responses are collected and numeric responses are statistically summarized. If a consensus was not obtained, return the summarized responses to the experts, along with any comments made by the experts (anonymously), and have them revise their forecasts based on this data. Repeat until either a consensus is reached (the answers converge) or else a "stalemate" occurs (no convergence can be obtained).
Simple Exponential Smoothing
Calculating Multiplicative Seasonal Indexes
Simple time-series methods include moving average models. In this case, the forecast is the average of the last "x" number of observations, where "x" is some suitable number. If you're forecasting monthly sales data, you might use a 12-month moving average, where the forecast for the next month is the average over the past year.
E.g. Ft+1 = Yt + Yt−1 + · · · + Yt−n+1/ n
where Ft+1 is the forecast value at time t + 1, Yt is the observation at time t and n is the number of observations used to calculate Ft+1
Trouble is, simple averaging methods don't tend to work well when there's either a trend in the data or seasonal effects. E.g. In case of marketing data, other techniques such as exponential smoothing may be more appropriate.
With moving averages, every data point carries equal weight in making the forecast whereas in case of smoothing methods, more importance is placed on the most recent data than on the historical data. If there's a trend in the data, it'll use the recent observations to make up the bulk of the forecast, and the forecast is more likely to reflect the trend.
Moving averages and simple exponential smoothing techniques are available in Excel and easy to execute. That's part of the great advantage of time-series methods: they're generally simple, cheap to run, and relatively easy to interpret.
Explanatory methods
Explanatory forecasting methods take the process a step further and allow us to relate changes in marketing activity to changes in such outputs as sales, brand awareness, and registrations.
Explanatory models assume that the variable to be forecasted exhibits an explanatory relationship with one or more other variables. For example, we may model the electricity demand (ED) of a hot region during the summer period as
ED = f (current temperature, strength of economy, population, time of day, day of week, error)
The relationship is not exact—there will always be changes in electricity demand that cannot be accounted for by the variables in the model. The “error” term on the right allows for random variation and the effects of relevant variables not included in the model. Models in this class include regression models, additive models, and some kinds of neural networks.
The purpose of the explanatory model is to describe the form of the relationship and use it to forecast future values of the forecast variable. Under this model, any change in inputs will affect the output of the system in a predictable way, assuming that the explanatory relationship does not change.
Qualitative forecasting methods can be categorized further as below
Delphi method: It is an iterative technique for obtaining a consensus forecast from a group of experts, without the problems inherent in group decision-making. The procedure works as follows: first, give a set of questions to each expert, who provides answers (forecasts) independently from the other experts. The responses are collected and numeric responses are statistically summarized. If a consensus was not obtained, return the summarized responses to the experts, along with any comments made by the experts (anonymously), and have them revise their forecasts based on this data. Repeat until either a consensus is reached (the answers converge) or else a "stalemate" occurs (no convergence can be obtained).
Market research: Questionnaires and interviews are used to solicit potential customers, current users, and others. One potential problem is that stated intentions (expectations) do not always translate into behavior. E.g. questionnaires, test markets, surveys, etc.
Product life-cycle analogy: forecasts based on life-cycles of similar products, services, or processes.
Expert judgment by management, sales force, or other knowledgeable persons
Product life-cycle analogy: forecasts based on life-cycles of similar products, services, or processes.
Expert judgment by management, sales force, or other knowledgeable persons