September 27, 2007; Joanne Loewy (loewyjoey@aol.com)
Response to Jinah Kim-Dropouts in Research
What a great article Jinah. I am sure so many can empathize with your struggle. I think one of the most difficult elements of research is the drop-out impact. I go through it with every research project , and I can tell you that I have learned so much along the way. Maybe it first just helps to release…meaning, do what you need to do with someone you respect, and trust enough to admit to yourself and your colleague-to say “man this really really stincts!”. It is a mourning of sorts…and it may sound crazy-but rather than stuffing the anger, or holding onto it, or blaming yourself–admitting frustration lets you go to the next step-It also provides a venue to instill in yourself and another: “hey this is not my fault-in life-stuff happens.”
This leads to the second reminder-that research, as much as we like to control it and to have a handle on the variables…it is occurring in a social context…esp research involving human beings. So…in my work-even with our most rigorous scientific studies (or I should qualify this and write, especially with our most rigorous scientific studies), it is essential to ‘use’ the drop out incidences as data. What does this mean? It means that there may be an unexpected trend that arises in the elements that you chose not to measure. And this may change the course of how the research is pursued.
For instance-we learned from our many ‘no shows’ in one study that it was simply too difficult for our subjects to get to us. Whereas we might have analyzed this as avoidance, or resistance…another prospect was to shift the contingencies of our design and go to our subjects…and then when all is said and done, this shift will have to be tabulated in the final analysis and out come.
I do believe that when we are researching human beings-and esp within the therapeutic process, we need to consider the no-shows and drop-outs as part of the data-even if it was not part of our pre-projected course of study and intended measurement. In all honesty, it is part of the data if it happens in the context of the research. I will try to take a stab at some of your amazing questions.
How can we prevent the participants from dropping-out in the middle of the clinical trials?
**We really cannot. Incentives may help. Expanding how we get to our subjects-being flexible about time…but we cannot prevent it. What we can do, once we have bent over backwards trying to make it possible to keep them in is delve into why they needed to drop out….was it really the drive or time? What is their resistance to other things in life and how is the very contact with you and pull out indicative of something you might not have chosen to measure….
2. What does help us to keep both our research subjects, and assistants from dropping out?
***Incentives. We offer movie passes to our kid subjects. For adult assistants…making them really feel par of the team…get articles for them-invite them to present on what they are doing…keep a love of the topic brewing…research takes love and nurturing.
3. How can we deal with dropouts and attrition in research? Do we simply accept the fact as characteristics of a real world research?
***Yes and furthermore….i think we include it as a finding and speculate how and why we think it’s occurred. Is it non-compliance? Is it denial? Is it that being involved is simply to taxing? Does being a subject remind someone that they are ill?
4. What do we do with the incomplete data from the dropout cases?
***We include it and make hunches. Drop out is a source of data. True, it may not be the data we set out to catch, but it is rich in and of itself.
Maybe the study design is too complex….we need to also ask this of ourselves. his is why a pilot run of 1-5 cases is always a good idea before beginning a research project. Pilot data allows us to view a preliminary run and then to shift and change…to adapt and understand the pitfalls before ensuing the real thing.
THANK YOU for this inspiring column. It reminds me that we are not alone in our struggles…and that we can be honest and pursue and shift and adapt our course of investigating…In this way, research can be true to life-which always means that there are unexpected variables and outcomes- and this simply ‘is’…it does not mean that we failed or did anything wrong. It simply makes a challenge or two (or twenty)…(!), but in the long run, it makes our research more honest and interesting.
September 20, 2006; Jinah Kim (kim_jinah@hotmail.com)
Dealing with Dropouts and Attritions in Research
Carrying out clinical research often requires rigorous planning and close collaboration with the people with whom one is involved. Researchers may encounter difficulties in dealing with unexpected aspects of a real world situation regardless of how well prepared and planned the research was. The following explains my own experiences that I would like to share with researchers in the field.
I have recently completed three years of a doctoral research study investigating the effects of improvisational music therapy on the joint attention and social engagement of autistic children, which involved seven to eight months of clinical trials with each of the ten participants. Even though autism requires comprehensive long term treatments, seven to eight months was an unusually long duration for the controlled clinical trials for this population. Recruitment of the children (subjects) who satisfied the inclusion criteria for the project was a long and painstaking process. Throughout the two years of the research period, I had a total 20 referrals from the department of child and adolescent psychiatry of a general hospital in Seoul, Korea and 15 out of 20 referred children enrolled to the project. A total five children dropped out for different reasons at different phases of the trials. The children were age between three to six, and most of them came from a long distance. As expected, small children are prone to minor periodic illnesses and are not easily adapted (especially children with ASD) to long distance traveling. Therefore, attending seven to eight months of clinical trials was a challenge to most of the children and to their families. In all cases, the mothers were the primary caretaker of the children and their levels of willingness for cooperation, understanding and trust shown to the researcher turned out to be important factors for the successful completion of the clinical trials for each child. Even then, there were still unexpected factors leading to the mother’s decision for discontinuation.
The reason for the first dropout case was quite complex. The child’s mother drove her car from their home to the clinic for almost two hours each week through very dense traffic jam in the city of Seoul and got completely exhausted and frustrated by the time they arrived to the clinic (their home was less than an hour away in normal traffic situation). Following the research design, the child was randomly assigned to have play sessions with toys for the first 12 sessions, and then was expected to have same length of music therapy sessions later. During the toy play sessions, the child often went to the corner of the room and played with toys on his own. The mother observed how solitary her child was in toy play sessions from the TV monitor in the waiting area. As expected and turned out, most autistic children in this study inclined to play with their toys alone rather than readily sharing toys with the therapist, while the same children were more readily engaged with the therapist through music making process in music therapy sessions. The child was a five year-old high-functioning autistic boy and his mother had very strong opinion on what the therapist/ adults should do with her child. She was very controlling, directive and often intrusive towards her child. Almost each week after toy play sessions, she called me in exasperation and asked why the therapist allowed the child to do what he did, and did not administer strict rules in toy play sessions (such as pulling out her child from the corner of the room, making the child to sit down on the chair, giving instructions, etc). She seemed not only to feel threatened and disappointed to see her child’s solitary play in play sessions, but also she could not approve the child-centred approach even though from the first interview the principles and purpose of the child-centred approach were explained fully. What the mother did not recognize was the fact that the child was responding to the therapist’s sensitive and attentive approach more and more in his own way, however slowly. I emphasized the importance of nurturing the child’s own motivation and spontaneous interaction through joining the child’s focus of attention and play rather than directing and giving instructions to the child, however in vain. Shortly after the dropout, I was informed from a colleague that the child began to display very strong resistance and disruptive behaviours towards his mother and in other highly structured intervention programmes his mother assigned him to attend. The mother, as a result, had no choice, but to cancel most intervention programmes and to have a break for a while. This child showed certain potentials for musical intervention (such as his aptitude for singing and rhythmic playing), however, never reached the point where he was supposed to have music therapy intervention.
Another example, a mother of a three year old boy informed me of her decision to suddenly dropout after the child’s periodic illness. The mother had previously shown a positive and trusting attitude towards the researcher and the music therapy-research project as her child responded well to the therapy programme. However, when the child fell ill on one occasion (acute enteritis), the existing conflicts among the family members arose and the mother-in-law and her husband blamed the mother for taking the child to the distant clinic regularly as the cause of the child’s illness. In fact, acute enteritis was not related to long distance traveling in this case. Since the child had shown good potential to develop further in music therapy, which he had not shown previously in other interventions, and had been already in the trial for five months, I tried to persuade the mother to continue her child in the study until the completion of the therapy trials by offering her a free transportation (to drive from their home to the clinic back and forth by myself) for the rest of the trial period. The mother, however, chose to follow her husband and the mother-in-law’s wish and said her relationship with her family was in crisis and it was more important for her save the family relationship. Culturally, submission to the mother-in-law and husband’s will is still considered as a good virtue for the married women in Korea.
From the beginning of the research project, I was well aware of the possibility of conflicting interests between the researcher and the participants that I was prepared to work for the participants’ interest and their benefits as well as ensuring my own purpose of collecting the necessary research data, however, did not expect such circumstances. This mother was well-aware of the fact that dropping out was not only my loss but also their loss too. Still, she followed what she thought the best solution for the family.
Throughout the process of managing the project, I also experienced that the problem of dropout was not confined to the children. Two of the research assistants dropped out without any prior notice for different personal reasons at different research phases too. The work they were doing was an important part of the trials such as video recording assistance, carrying out pre/post tests, etc. Therefore, when entrusted research assistants did not turn-up without notice on the day of clinical trials, it became a major crisis to the management of the project at that time.
Now, looking back, these were probably unexpected variables that real world research entails. At the beginning of this project, I had expected there would be some drop-outs, but not to that high rate. As a result, I had to recruit more participants (both children and research assistances). Therefore, the whole project took more time to complete. Near the end of clinical trials, my own patience and endurance was running out too.
After completing the research project, I wonder what I could have done differently in order to minimize the potential dropouts, and attritions in clinical research. Could I have predicted some of the characteristics of the caregivers/ research assistances, who were prone to drop-out and screened them out before-hand? Or is it better to train myself with certain types of meditation to better deal with stresses of unexpected attritions of a real world research?
I’d like to invite researchers to share their experiences and how they dealt with their situations. Discussion may evolve freely, or around these questions below;
- How can we prevent the participants from dropping-out in the middle of the clinical trials? What does help us to keep both our research subjects, and assistants from dropping out?
- 2) How can we deal with dropouts and attritions in research? Do we simply accept the fact as characteristics of a real world research?
- What do we do with the incomplete data from the dropout cases?