Home / Summaries / Simulation Modeling and Analysis / approach-ci-procedure

Output Data Analysis for a Single System - Statistical Analysis for steady-state parameters

Q: What is the tradeoff in choosing the warm-up period?

- If l and m are chosen too small, E(hat{Y}(m,l)) might be significantly different from v. - If l is chosen larger than necessary, hay{Y}(m,l) will probably have an unnecessarily large variance.

24 important questions on Output Data Analysis for a Single System - Statistical Analysis for steady-state parameters

What is the tradeoff in choosing the warm-up period?

- If l and m are chosen too small, E(\hat{Y}(m,l)) might be significantly different from v.
- If l is chosen larger than necessary, \hay{Y}(m,l) will probably have an unnecessarily large variance.

For what do we need the Welch procedure?

To decide upon the warmup period (l).
Determine a time index l such that E(Y_i) is approximately v, for i > l.

Advantages replication/deletion approach for means:

1. If properly applied, this approach should give reasonable good statistical performance.
2. It is the easiest approach to understand and implement.
3. This approach applies to all types of output parameters.
4. It can easily be used to estimate several different parameters for the same simulation model.
5. This approach can be used to compare different system configurations.
6. Multiple replications can be made simultaneously on different cores within a single computer or on different computers on a network, provided that the software being used for simulation supports this.

Objection against replication/deletion approach:

It uses one set of n replications to determine the warum period l, and then uses only the last m' - l observations from a different set of n' replications to perform the actual analyses.
(Usually not a problem due to the relatively low cost of computer time).

Criticism replication/deletion approach:

A 100(1-alpha) percent CI is actually being constructed for E(Xj) rather than for v. As a result, if we make a large number of replications n' in an effort to make the CI half-length small, then the coverage of the CI might be much less than the desired 1-alpha.

What are two general strategies for constructing a point estimate and CI for v?

1. Fixed-sample size procedures.
2. Sequential procedures.

Fixed-sample size procedure:

A single simulation run of an arbitrary fixed length is made, and then one of a number of available procedures is used to construct a CI from the available data.

List six fixed-sample-size procedures for constructing a point estimate and confidence interval for v.

1. Replication/deletion.
2. Batch means: approximate IID samples
3. Autoregressive : estimate auocorrelations
4. Spectral : estimate autocorrelation
5. Regenerative: regeneriation points, true IID
6. Standardized time series, batches, assumptions

Advantage: only one or no warmp-up period
Each method: variants for choices of parameters and confidence intervals

Potential difficulties of Batch means approach:

Choice of batch size, k.
Too small: correlated batch means, low coverage.
Too large: few observations, low precision.

What do the autoregressive method and the spectrum analysis have in common?

Rather than attempt to achieve independence, the two methods use estimates of the autocorrelation structure of the underlying stochastic process to obtain an estimate of the variance of the sample mean and ultimately to construct a CI for v.

Why is the spectrum analysis difficult?

- It requires a fairly sophisticated background on the part of the analyst.
- There is no definitive procedure for choosing the value of q.

Idea regenerative method:

To identify random times at which the process probabilistically starts over, i.e. regenerates, and to use these regeneration points to obtain independent random variables to which classical statistical analysis can be applied to form point and interval estimates for v.

Difficulties regenerative method:

- Real-world simulations may not have regeneration points, or (even if they do) the expected cycle length may be so large that only a very few cycles can be simulated.

What does the standardized time series method assume?

The process Y1, Y2,.. is strictly stationary with E(yi) = v for all i and is also phi-mixing.

Potential difficulties Standardized Time Series approach:

choosing the batch size k too small.

Why is it interesting to see how the five fixed-sample-size CI approaches perform in practice?

They all depend on assumptions that will not be strictly satisfied in an actual simulation.

Conclusion performance fixed-sample-size approaches:

1. If the total sample size is chosen too small, the actual coverages of all existing fixed-sample-size procedures may be considerably lower than desired.
2. The appropriate choice of the sample size would appear to be extremely model dependent and thus impossible to choose arbitrarily.
3. For m fixed, the methods of batch means, standardized time series, and spectrum analysis will achieve the best coverage for n and f small.

Why is a sequential procedure needed?

No procedure in which the run length is fixed before the simulation begins can generally be relied upon to produce a CI that covers v with the desired probability 1 - alpha, if the fixed run length is too small for the system being simulated.

Why do we consider the estimation of steady-state parameters other than the mean?

The mean does not always provide us with an appropriate measure of system performance.

Parameters of the steady-state distribution of interest:

1. Steady-state probability.
2. q-quantile.

What are the difficulties in applying Welch's procedure?

1. The required number of replications may be relatively large if the process Y1, Y2,... is highly variable.
2. The choice of l is somewhat subjective.
3. Choice of m (too large can be a waste of computer time)

Where must the length of the warm-up period be proportional to?

- Mean processing times.
- 1/(1-rho)^2, where rho is the load of the resource.
- Squared coefficients of variation.
- Size of network, i.e. number of nodes.

How does the Welch procedure work?

1. Make n (>= 5) replications of length m (large). Then, Yji is the ith observation from the jth replication.
2. Plot Yibar, which is the average of Yji per replication.
3. If further smoothing of the plot is necessary, define the moving average Yibar(w), where w is the window.
4. Plot Yibar(w), and choose l to be the i beyond which the line appears to have converged.

What is the goal of the Jacknife estimator of ratio of expectations and how to calculate the estimator?

Data y1,....,Yn IID X1,....., Xn IID

Goal: estimate E{y}/E{X}
Jacknife is less biased than the classical point estimator

The question on the page originate from the summary of the following study material:

Simulation Modeling and Analysis | 9781259254383 | Averill M Law

View summary

A unique study and practice tool
Never study anything twice again
Get the grades you hope for
100% sure, 100% understanding

Remember faster, study better. Scientifically proven.

Output Data Analysis for a Single System - Statistical Analysis for steady-state parameters

24 important questions on Output Data Analysis for a Single System - Statistical Analysis for steady-state parameters

What is the tradeoff in choosing the warm-up period?

For what do we need the Welch procedure?

Advantages replication/deletion approach for means:

Objection against replication/deletion approach:

Criticism replication/deletion approach:

What are two general strategies for constructing a point estimate and CI for v?

Fixed-sample size procedure:

List six fixed-sample-size procedures for constructing a point estimate and confidence interval for v.

Potential difficulties of Batch means approach:

What do the autoregressive method and the spectrum analysis have in common?

Why is the spectrum analysis difficult?

Idea regenerative method:

Difficulties regenerative method:

What does the standardized time series method assume?

Potential difficulties Standardized Time Series approach:

Why is it interesting to see how the five fixed-sample-size CI approaches perform in practice?

Conclusion performance fixed-sample-size approaches:

Why is a sequential procedure needed?

Why do we consider the estimation of steady-state parameters other than the mean?

Parameters of the steady-state distribution of interest:

What are the difficulties in applying Welch's procedure?

Where must the length of the warm-up period be proportional to?

How does the Welch procedure work?

What is the goal of the Jacknife estimator of ratio of expectations and how to calculate the estimator?

Summaries related to Basic Simulation Modeling

Simulation Modeling and Analysis

Supply Chain Management A Balanced Approach

Class notes - Philosophy of Science

Class notes - Microeconometrics

Indian Economics

Global politics

Essentials of international relations

Behavioral genetics

Management and organisational behaviour

Follow Up Engels idioom 4/5 H

International Business

Marketing fundamentals