Addition to Sampling with and without replacement

The difference between sampling with replacement and sampling without replacement is twofold:

1. In the former case a temporal order is essentially assumed during the sampling process. Elements {a_1, a_2, .....,a_n} are drawn one after another, and at any time during the process, the same set of elements to draw from (S, say) is available.
2. Sampling without replacement can also be described as `grabbing', taking all the elements {a_1, a_2, ...,a_n} of the sample at the same time. If done sequentially, that set has to be adapted since the element that has been drawn should be removed. If  the initial set of elements to draw from is indicated by S_0, at each draw the set  from has to be adapted in the following way: S_i = S_(i-1) \ a_i.   

Funnily enough, sampling without replacement as occurs in real life research is essentially sequential and the above formulation in 2 seems counterintuitive since there is no distinct background set S apart from the sampling frame (Hand, 2008) and the population P (Think of a clinical trial in which patients are included one after another).  

In short, the two sampling mechanisms are very different and are applied in different situations.
Sampling without replacement is the sampling procedure that is commonly used in all emperical research.
Sampling with replacement is used in resampling strategies like bootstrapping. In such cases the notion of population from which the sample is drawn, is remote.

However, sampling with replacement also occurs in real life sampling, for instance in Capture-Recapture sampling aimed at determining population size of animals in a habitat, see section 12.3.6 of Agresti's book.

References
Alan Agresti (2002). Categorical data analysis. Second edition. Hoboken, NJ: Wiley.
David J. Hand (2008). Sampling. In H. J. Adèr, G. J. Mellenbergh, & D. J. Hand, Advising on research methods: A consultant's companion, Chapter 5. Huizen, The Netherlands: Van Kessel.

Sampling with and without replacement

In the bootstrapping literature (Efron & Tibshirani, 1993), the idea of ‘sampling with replacement’ is essential. It indicates that different bootstrap samples of size n are drawn from the same data set of size n, without producing the original data set over and over again. For instance, bootstrap samples from the data set of the numbers {1,9,11,12} may be: {1,1,11,12}; {1,9,11,9}; {1,9,11,12} and so on.
However, the procedure underlying sampling with replacement differs completely from sampling without replacement, in which the same unit can never appear twice.
In fact, sampling with replacement supposes that units are drawn one after another, and that a copy of the unit that was drawn is put back into the data (the actual replacement). In contrast, sampling without replacement allows to ‘grab’ n units at the same time.
The same effect as the sequential drawing described above for sampling with replacement would be obtained if n units were grabbed from a data set consisting of n copies of the original data. The data set would then be of size n².

References

Efron, B., & Tibshirani, R. J. (1993). An introduction to the bootstrap. New York: Chapman & Hall.

Multiple imputed missings in a dependent variable: How to analyze them?

During the analysis of the International PIRLS project, a comparative study on reading comprehension of students of Grade 4 and 5, the following problem was encountered during the analysis phase:
Some of the dependent variables (the plausible scores) are obtained by doing multiple impution on several scores (5 times). Usually, properly analyzing the data sets resulting from multiple imputation makes it necessary to do the analysis of each data set separately and then afterwards, combining the results in some way (as far as I know, the only standard statistical package that does this for you automatically, is Mplus).
However, in case the dependent variable is imputed, one can use a repeated measurement design. In the case described above, an extra level was added to a multilevel analysis with the 5 plausible scores as a repeated measurement.
Drawback of this approach in multilevel analysis (MLwiN) is that the data sets which are already enormous in this case, are quintupled.

Herman

Distribution outside Belgium and the Netherlands

It turned out to be very difficult to get distribution of the ARM book (Advising on research methods: A consultant's companion by Adèr, Mellenbergh and Hand) via Barnes and Noble in the US up and running.
Until these problems are solved, readers outside Belgium and the Netherlands can order directly from the publisher, Johannes van Kessel Publishing at: Publishing@jvank.nl. They charge only local mailing costs (euro 6.20). Please contact them by email. To order via bol.com, van Stockum or Selexyz see: www.jvank.nl/ARMHome
The proceedings of the 2007 KNAW symposium `Advising on research methods' can also be ordered by contacting the publisher.

Herman

Proceedings of the KNAW colloquium on Advising now accessible via Google Books

We submitted the text of the Proceedings of the 2007 Colloquium `Advising on research methods' to Google Books. The book is now accessible. In Google Books, search for `Advising on research methods' or for the ISBN number 9789079418039.
Herman

Two announcements

I have two announcements to make:

1. November 11, 2008, the proceedings of the 2007 KNAW Colloquium `Advising on research methods' were published. The Colloquium was organized by Adèr and Mellenbergh and was funded by the Royal Netherlands Academy of Arts and Sciences (KNAW). The full reference is: Adèr & Mellenbergh (2008). Advising on research methods. Proceedings of the 2007 KNAW Colloquium. Huizen, the Netherlands: Johannes van Kessel (ISBN: 978-90-79418-03-9). The table of contents can be found at: http://www.knaw.nl/colloquia/advising/index.cfm . The commercial edition is sold at 40 euros. It can be ordered via http://www.bol.com/ , http://www.vanstockum.nl/ and http://www.selexyz.nl/ .
2. We (Don Mellenbergh and myself) are organizing a public course on Advising, using the ARM-book (http://www.jvank.nl/ARMHome) as material. It will be held May 13-20, 2009 nearby the Amsterdam central station. Pre-registration is already possible. For more information, see: www.jvank.nl/ARMCourse

Herman Adèr

Several Announcements

Finally, finally a new entry in this Weblog.

First, a few announcements:

1. Last February (2008), we (Don Mellenbergh, David Hand and myself) have published a (hand-/text-) book on methodological advising. The full reference is: Adèr, H. J., G. J. Mellenbergh and D. J. Hand (2008). Advising on research methods: A consultant's companion. ISBN 978907941801-5. Huizen: Johannes van Kessel. The book has its own webpage: www.jvank.nl/ARMHome . It is also accessible via Google books (http://www.books.google.com/ ; type: `Advising on research methods' or something of the kind).
2. During a colloquium, also called `Advising on research methods', held in March 2007 in Amsterdam and sponsored by the Royal Netherlands Academy of Arts and Sciences (KNAW), a masterclass was held during which participants of the colloquium functioned both as advisors and as clients. This masterclass was recorded and is now available on DVD (free of charge). It represents unique material of excellent quality. To access the colloquium website, click: http://www.knaw.nl/colloquia/advising/index.cfm . To order the DVD click `DVD of the consultation interviews'.
3. We are working on the proceedings of the above colloquium. If it becomes available, it will be announced here.

Herman Adèr.