Which of the following is one of the downsides of a computer-based training (cbt) approach?

Depress Anxiety. Author manuscript; available in PMC 2010 Jul 19.

Published in final edited form as:

PMCID: PMC2906395

NIHMSID: NIHMS90304

Michelle G. Craske, Ph.D.,1 Raphael D. Rose, Ph.D.,1 Ariel Lang, Ph.D., M.P.H,2 Stacy Shaw Welch, Ph.D.,3 Laura Campbell-Sills, Ph.D.,2 Greer Sullivan, M.D.,4 Cathy Sherbourne, Ph.D.,5 Alexander Bystritsky, M.D.,6 Murray B. Stein, M.D., M.P.H.,7 and Peter P. Roy-Byrne, M.D.3

Abstract

Objectives

This paper describes a computer assisted CBT program designed to support the delivery of evidenced-based cognitive-behavioral therapy (CBT) for the four most commonly occurring anxiety disorders (panic disorder, posttraumatic stress disorder, generalized anxiety disorder, and social anxiety disorder) in primary care settings. The purpose of the current report is to (1) present the structure and format of the computer-assisted CBT program, and (2) to present evidence for acceptance of the program by clinicians and the effectiveness of the program for patients.

Methods

13 clinicians using the computer-assisted CBT program with patients in our ongoing Coordinated Anxiety Learning and Management (CALM) study provided Likert-scale ratings and open-ended responses about the program. Rating scale data from 261 patients who completed at least one CBT session were also collected

Results

Overall, the program was highly rated and modally described as very helpful. Results indicate that the patients fully participated (i.e., attendance and homework compliance), understood the program material, and acquired CBT skills. In addition, significant and substantial improvements occurred to the same degree in randomly audited subsets of each of the four primary anxiety disorders (N=74), in terms of self ratings of anxiety, depression and expectations for improvement.

Conclusions

Computer-assisted CBT programs provide a practice-based system for disseminating evidence-based mental health treatment in primary care settings while maintaining treatment fidelity, even in the hands of novice clinicians.

Anxiety disorders are common, afflicting almost 29% of the population by 75 yrs (1) and over 10% in any given year (2). There is good evidence for the efficacy of medication (3, 4) and cognitive behavioral therapy (CBT) (5, 6). However, the evidence is based mostly on studies in clinical research settings. Adapting the delivery of these treatments to real world settings remains a challenge and there is a need for service delivery models to support the application of these evidence-based treatments.

We are currently testing a collaborative-care model of care delivery for anxiety disorders in primary care settings in our ongoing study (Coordinated Anxiety Learning and Management, CALM; 7) of the effectiveness of evidence-based treatments (psychopharmacology and CBT). We chose primary care because anxiety disorders in this setting are particularly prevalent (8), costly (9) and poorly treated (10,11), with anxious patients often dissatisfied due to perceived unmet needs (12,13).

In collaborative care models, patients typically remain under the care of a primary care physician, while mental health specialists, usually master's level clinicians (e.g., nurses, social workers), coordinate and deliver specialized care in consultation with psychologists and/or psychiatrists. The collaborative care approach has been tested extensively in primary care settings for depression (14,15,16,17) and, to a lesser extent, panic disorder (18, 20) and GAD (19). The “user-friendliness” of these interventions is believed to be central to successful implementation (21), and was emphasized in the feedback from primary care providers from our prior treatment study of panic disorder in primary care (20).

Based on these considerations, our CALM intervention, and, in particular, its critical CBT component, was developed with the goal of maximizing user friendliness and ease of implementation. We aimed to create a brief intervention that could be adopted in a variety of primary care settings that lack existing mental health expertise. To this end, we developed a novel, computerized system for supporting delivery of CBT for anxiety disorders by novice clinicians, called CALM Tools for Living.

This CBT program is novel in a number of ways. First, in contrast to separate, uniquely tailored CBT manuals for different disorders, we developed a CBT approach that singly addresses the four most common anxiety disorders in primary care settings (12): panic disorder with or without agoraphobia (PD), generalized anxiety disorder (GAD), social anxiety disorder (SAD), and posttraumatic stress disorder (PTSD). In this program, the core elements of CBT are the same across the four anxiety disorders, whereas other elements are tailored to the features unique to each anxiety disorder through branching mechanisms.

In its application, the focus is maintained upon the most distressing and impairing of the four anxiety disorders, as rated by the patient. Comorbid anxiety as well as mood disorders (22,23,24) are presumed to improve as a result of this focused attention, given the evidence that CBT for a targeted anxiety disorder yields positive benefits upon comorbid anxiety and mood disorders (25-30). Moreover, we chose not to focus upon more than one anxiety disorder at the same time given the preliminary evidence that CBT programs that simultaneously target a principal anxiety disorder and a comorbid anxiety or mood disorder are less effective than CBT targeting a principal anxiety disorder only (31).

Another novel feature is that the computer program guides both the clinician and patient. The choice to guide the clinician by the computer program was driven by two factors. First was the need to minimize clinician training to enhance eventual implementation in primary care and other real world settings. Second was the need to develop a system that would support minimally trained clinicians' implementation of a relatively sophisticated CBT for not just one but four anxiety disorders. Thus, we developed a program in which a computer program guides the clinician, session by session, in the delivery of CBT. Computer-assisted programs already have been used in training clinicians in CBT (32, 33), but they have not before been used for ongoing assistance for delivery of CBT. The program aids novice clinicians by not only providing the structure for delivering CBT for four different anxiety disorders, but also by helping them to remain on target (i.e., focus upon CBT), and to maintain CBT fidelity.

The computer program also guides the patient. Stand-alone (i.e., without clinician involvement) computerized versions of CBT are well researched, and have been shown to be generally acceptable to patients and effective in treating depression and anxiety (34,35), as well as specific anxiety disorders, including PD (36,37,38), SAD (39,40), PTSD (41), and obsessive compulsive disorder (42,43). However, they are associated with higher rates of drop-out or refusals, and lower rates of satisfaction with therapy, compared to a live clinician (44). In contrast, computerized programs are more acceptable and more successful when clinician involvement is offered (37,43,38,45). Hence, our choice to engage the clinician with the patient as they both proceed simultaneously through the computer program is expected to benefit patient involvement and satisfaction.

The purpose of the current report is twofold. The first is to present the structure and format of the computer-assisted CBT program. The second is to present evidence for acceptance of the program by clinicians and the effectiveness of the program for patients.

Method

Participants

CALM Tools for Living was implemented as part of our ongoing CALM study that is being conducted across four sites, Seattle, San Diego, Los Angeles, and Little Rock (details of study methodology are provided in [7]). Thirteen clinics across the four sites participated. The clinics were selected based on considerations of provider interest, space availability, size and diversity of the patient population, and insurance mix. Anxiety management clinicians who delivered the intervention (n=13) had some patient care experience (although only 7 had prior mental health care experience) and some exposure to primary care settings, but without expertise in anxiety management or CBT. The clinicians ranged in age from 25 to 59 years, were mostly women, and typically had master's degrees in social work or nursing. Some were selected by study personnel and employed by the study site academic institution, while others were working clinically at affiliated sites and were employed by the health care provider organization operating the clinic (7).

The clinician training (detailed in a separate report, Rose et al., in preparation) involved readings and five days of workshops addressing CBT in general, CBT in CALM, and the tailoring of CBT to each of the four anxiety disorders. Interspersed between workshops were role-plays where clinicians practiced CBT skills and use of the computer-assisted program. In addition, clinicians took on two to four “training cases” and their proficiency at delivering CBT was evaluated by expert psychologists. Lastly, clinicians delivering the program received on-going group supervision for approximately one hour per week from an expert psychologist.

Patients were either primary care physician/nurse referred or responded to advertisements within each clinic. Eligible patients were at least 18 years old, met diagnostic criteria for generalized anxiety disorder, social anxiety disorder, panic disorder or posttraumatic stress disorder (as determined through administration of the Mini International Neuropsychiatric Interview (MINI; Version 5.0) (46). In addition, eligible patients were willing to participate in CALM. Exclusion criteria were serious alcohol or drug use, unstable medical conditions, cognitive impairment, active suicidality, psychosis or bipolar 1 disorder, or ongoing participation in CBT. Also, persons without telephone access or who could not speak English or Spanish were excluded. In addition to meeting criteria for the disorder, patients had to score at least 8 on a scale from 0 to 20 (indicative of moderate or greater severity) on the Overall Anxiety Severity and Impairment Scale (OASIS) (see below).

Patients who entered the CALM study were randomized to either treatment as usual or our collaborative care intervention. The latter involved either CBT, medication management or both, a decision that was made jointly by the patient and the clinician. Data from patients who elected CBT, either alone or in combination with medication management, are described below.

Materials

CALM Tools for Living is interactive and clinician guided. It contains eight modules; Calm Recording (i.e., self monitoring); Education; Calm List (i.e., fear hierarchy); Calm Breathing (i.e., breathing retraining); Calm thinking (i.e., cognitive restructuring); Calm Living (i.e., exposure to external cues); Calm Feeling (i.e., exposure to images, memories, sensations); and Keep Going (i.e., relapse prevention). The Education, Calm Thinking, Calm Feeling and Calm Living modules are tailored to each of the four anxiety disorders through branching mechanisms, whereas the remaining modules are mostly generic to all four anxiety disorders.

Each module includes text (e.g., session goals and summaries, educational information, review of principles presented earlier), review of homework since the last visit, in-session practice of new skills, instructions for skills practice between sessions, and video demonstrations of specific CBT strategies for each anxiety disorder. An assessment at the end of each module includes self ratings of understanding, multiple choice quizzes, and clinician ratings of the patient's demonstrated skill acquisition and is designed to assess understanding of and competency with the material.

To reinforce what was learned in session, printouts of the material presented are given to patients. These printouts are individualized; individual examples that the patient describes in session are recorded into the computer program by clinicians. The individualized data are accessed in subsequent sessions to keep the clinician and client on track and promote progress. Furthermore, these data allow for what has recently been termed “measurement based care” (47), where ongoing monitoring of clinical state can help clinicians easily determine a patient's progress and adjust their future treatment accordingly.

The clinician sits side by side with the patient as they both view the program on screen. Throughout, the program provides prompts to clinicians to engage in specific tasks, such as helping patients to establish a fear hierarchy, demonstrating breathing skills, or designing in vivo exposure assignments. In addition, the clinician is trained in advance to direct the patient regarding which sections of the computerized treatment to read; inquire about and assess the patient's understanding of the material; summarize and rephrase sections of the text as necessary to provide clarification; assist in the application of CBT principles to the patient's idiosyncratic thoughts and behaviors; demonstrate skills; and reinforce in-session skills practice and between-session homework practice.

The program is intended to be completed as quickly as within 6 to 8 visits although flexibility is allowed in the time taken to completion and more advanced modules are available if additional care is appropriate. For instance, focus can be shifted to a comorbid anxiety disorder if it does not resolve during the treatment of the first anxiety disorder, without initiating a totally separate treatment protocol or repeating information. Suggested timing of the basic modules of the program is presented in Figure 1. The program is available in both English and Spanish languages.

Overall treatment plan for Calm Tools for Living over six sessions

Patient Activation = skills for enhancing motivation and reducing barriers to treatment involvement; Education = psycheducation about fear and anxiety; CALM Recording = self monitoring anxiety and fear; CALM List = generation of a hierarchy of feared situations; CALM Breathing = breathing retraining as a coping tool for interrupting anxiety; CALM Thinking = identifying anxious thinking and learning skills for generating more evidence-based, rational thinking; CALM Living = exposure to feared situations; CALM Feeling = exposure to feared sensations, images, or memories; Keep Going = skills for relapse prevention.

Even though depression often subsides as targeted anxiety disorders improve (20,48,49), it may occasionally remain unchanged or worsen (25,50) and therefore interfere with anxiety treatment. In CALM, levels of depression are monitored using self report and clinician observations at each visit. When depression significantly worsens or interferes with anxiety treatment, clinicians are taught to shift emphasis to behavioral activation and cognitive restructuring tailored to depressed mood. Return to the target anxiety disorder is encouraged as soon as is appropriate.

Finally, throughout delivery of CALM Tools for Living, clinicians are taught to recognize indicators of decreased patient engagement, and to respond by addressing: new or ongoing barriers to treatment (e.g., scheduling issues, family issues); negative reactions to the treatment thus far (e.g., side effects, unpleasant emotions, disparities between initial expectations and process of treatment); negative outcome expectancies (e.g., judging that treatment will not work); and negative self efficacy (e.g., judging inability to carry out the treatment). These issues are addressed through motivational enhancement strategies tailored to anxious patients (see 51,52), as well as direct problem solving and cognitive restructuring as appropriate.

Measures

Clinicians responded to survey questions regarding their judgments of the CALM Tools for Living program, after a year or more of experience with the program. Survey items included a series of 7 point (1-7) Likert ratings about ease of use and overall judgment, as well as a series of open ended questions (What did you like most about the computer program?, What did you like least about the computer program?, What changes, if any, would you make to the computer program?, What did your patients like most about the computer program?, What did your patients like least about the computer program?, What changes, if any, would your patients make to the computer program?).

Patient measures included attendance (number of CBT sessions attended before moving into the maintenance phase of the study), self ratings and performance on quizzes at completion of each module of CALM Tools for Living program as an indication of understanding of the material presented, involvement in CBT (i.e., number of homework assignments completed), and symptom and expectancy data. The latter was assessed at each visit with the clinician using the Overall Anxiety Severity and Impairment Scale (OASIS; 53); a five-item scale (each rated on a 1-4 point anchored scale, total score 0-20), which we have shown to be internally consistent and to possess good convergent and discriminant validity. The OASIS rates frequency and intensity of anxiety and avoidance, and the degree to which anxiety interfered with functioning over the last week.

Depression was measured using the two depression items plus the one fatigue item from the Patient Health Questionnaire (54). Expectancies for positive outcomes and for being able to do what is necessary to make the treatment successful (i.e., self efficacy) were rated using 0-8 point Likert scales devised specifically for the purposes of this study. Symptom data and expectancy data were collected at each visit with the clinician.

Results

Clinician Impressions

The clinicians (n=13) judged the CALM Tools for Living program very highly. Five questions were rated on a 1-7 point Likert scales. The mean rating to the question “How well did the computer program work during your meetings?” was 6.23 (SD=0.73). Corresponding items and mean responses are as follows: “How easy was it for you to use the computer program?”, 6.15 (SD=0.69); “How easy was the language used in the program for your patients to understand?”, 5.39 (SD=0.77); “What was your patients' overall opinion of the computer program?”, 5.69 (SD=1.25); and “What was your overall opinion of the computer program?”, 6.08 (SD=0.95). In responses to open-ended questions, clinicians appreciated the way in which the program provided a clear agenda per session and overall structure, as well as an easy-to-use guide for CBT concepts. They also cited the comprehensiveness of the information, the print functions for creating individualized-patient “workbooks”, the video demonstrates of specific skills, and the interactive nature of the program as strengths of Calm Tools for Living. Clinicians noted areas for improvement, including simplification of the language, reduction in the amount of text and elimination of redundancy, and greater flexibility. Future versions will be modified based on this type of feedback to further optimize user-friendliness of the program.

Patient Characteristics and Attendance

To date, 290 patients completed 12 months (i.e., the maximum duration of the intervention phase of the study) since they entered the CALM intervention. Of those, 261 received at least one CBT session, and of these, 53.6% were taking antidepressants, 27.6% benzodiazepines, and 18% other psychotropic medications. 69.7% were taking one or more medications, although not necessarily at optimal doses or for optimal durations. This group of 261 patients attended on average 7.63 (SD = 3.39) computer-assisted CBT sessions with a clinician (range = 1 - 19). The average number did not differ across the four targeted anxiety disorders (F(3, 74) = .10, ns). The rate of attendance indicates patients' willingness to participate in the program, especially given our general aim of limiting the number of CBT sessions to 8 to 10. The number of sessions attended did not correlate with initial session scores on the OASIS (r=-.15), PHQ (r=-.18), outcome expectancies (r=.02) or self efficacy expectancies (r=.15).

Assessment Module Data

Self rating and quiz performance data from the 261 patients who received at least one CBT session indicate that the material was well understood. In terms of self rating (0-100 point scale) of understanding the material, mean ratings for each module were as follows: Education, 80.3 (SD=21.4); Calm Recording, 87.2 (SD=12.9); Calm List, 85.4 (SD=14.2); Calm Breathing, 91.6 (SD=9.3); Calm Thinking, 91.4 (SD=9.0); Calm Living, 90.7 (SD=11.9); Calm Feeling, 91.5 (SD=9.8). The Keep Going module did not include self ratings of understanding. In terms of quiz performance, rates of percentage accurate for each module were as follows: Education, 97% (SD=.08); Calm Recording, 97% (SD=.09); Calm List, 90% (SD=.20); Calm Breathing, 95% (SD=.09); Calm Thinking, 86% (SD=.14); Calm Living, 95% (SD=.09); and Calm Feeling, 94% (SD=.11). There were no differences in self ratings of understanding or quiz performance across patients targeting the different anxiety disorders.

Patient Involvement

Of the 261 patients who received at least one CBT session, engagement in the CALM Tools for Living program was substantial and did not vary based on the specific anxiety disorder targeted. For example, they practiced CALM breathing skills on average 6.5 times (SD=3.6) between when they were first introduced to this skill and their next session with the clinician (typically this interval was one week). The corresponding numbers were 4.2 times (SD=3.5) for in vivo exposure practices to feared situations, and 4.1 times (SD=1.6) for exposure to feared memories, images or physical sensations, again with no differences across the four anxiety disorder targets. Thus, compliance with homework assignments was very good.

Patient Symptom Data

Self reported levels of anxiety and depression symptoms as well as expectancies for improvement were evaluated across randomly selected subsets whose treatment targeted generalized anxiety disorder (n=20), social anxiety disorder (n=20) and panic disorder (n=20), and all patients whose treatment targeted posttraumatic stress disorder (n=14). Means and standard deviations for the first and last CBT session are presented in Table 1.

Table 1

Patient ratings of anxiety, depression, and expectancies from the first to the last CBT session in randomly selected participants whose treatment targeted panic disorder, generalized anxiety disorder, and social anxiety disorder, and all participants whose treatment targeted posttraumatic stress disorder

OASIS scores decreased significantly and substantially from the first to the last Calm Tools for Living session (F(1, 66) = 56.3, p < .001, ES = .46), and did not differ across the four targeted anxiety disorders (F(3,66) = .59, ns) Nor were the results moderated by the presence (n=49) or absence (n=25) of taking psychotropic medication (F(1,66) = .58, ns). OASIS scores reduced on average by half. Nor did presence of benzodiazepines in particular moderate effects (F(1,66 = .26, ns). Scores for PHQ depression items also decreased significantly (F(1, 65) = 33.3, p < .001, ES = .34), and were not moderated by the type of anxiety disorder targeted in CBT (F(3,65) = .56, ns) or the presence of any psychotropic medication (F(1,65) = 1.0, ns) or, in particular, benzodiazepines (F91,65) = .33, ns). Expectancies in general were already at high levels at the first CBT session. Nonetheless, expectancies for the success of treatment increased significantly from the first to the last session (F (1, 65) = 27.5, p < .001, ES = .30). These changes were not moderated by type of anxiety disorder targeted (F(3,65) = .97, ns) or presence of any psychoptropic medication (F(1,65) = 1.56, ns) or, particularly, benzodiazepines (F(1,65) = 3.1, ns). The same was the case for expectancies for being able to carry out the requirements of treatment, which increased significantly, F(1, 65) = 3.92, p = .05, ES = .06. Increases again were not moderated by the anxiety disorder targeted (F(3,65) = .31, ns) or the presence of any psychotropic medication (F(1,65) = .55, ns). However, the presence of benzodiazepines, in particular, significantly moderated changes in self efficacy (F(1,65) = 6.1, p < .05, ES = .10). Self efficacy increased from the first to last session for those not taking benzodiazepines (t (52) = -3.59, p < .01), whereas it did not significantly change for those taking benzodiazepines (t(19) = .77, ns).

The degree to which OASIS scores changed from the first to the last CBT session correlated with patient ratings of the degree to which they understood the following CALM Tools for Living modules; Calm List, r=.36, p < .05; Calm Breathing, r=.37, p < .05; Calm Living, r = .43, p < .05; and Calm Feeling, r = .44, p < .05. Other module ratings of understanding (i.e., Education, Calm Recording, and Calm Thinking) did not correlate with outcomes. Also, outcomes did not correlate with quiz performance for any module, possibly due to the truncated range of scores.

Finally, a hierarchical regression approach was used to predict OASIS scores at the final session: first session OASIS scores were entered on Step I, type of anxiety disorder targeted and number of sessions attended were entered on Step II, self efficacy and outcome expectancies at the first session were entered on Step III, and patient rated understanding of the module that most highly correlated with outcomes (i.e., Calm Living) was tested on Step IV. Only patient rated understanding of the Calm Living module predicted final session OASIS scores, even after controlling for all other variables. Understanding contributed 21% unique variance, R2 = .21, Beta = -.48, p < .05.

Discussion

CBT is an evidence-based treatment for anxiety disorders that has not been well disseminated to primary care settings. The goal of our program is to develop a practice-based system that will support implementation of CBT in primary care settings. We developed a computer-assisted CBT program that singly addresses four anxiety disorders that are frequently encountered in primary care, and guides both clinician and patient. During computer-assisted CBT, clinicians maintain focus on the primary anxiety disorder, which in many cases will help to ameliorate comorbid anxiety and mood disorders. In addition, our program addresses comorbid depression that worsens or warrants direct intervention, as well as strategies for managing flagging patient involvement in treatment. Finally, our program was developed with an eye to future implementation in settings where personnel do not have extensive training or experience in CBT or psychotherapy in general. Thus, our computer-assisted CBT is designed to facilitate delivery and to maintain fidelity of CBT by novice clinicians.

Current data indicate that clinicians rate the program positively, although with some recommendations for improvements, and patients both engage in the program, and understand and practice the CBT skills. Also, patients report significant and substantial reductions in anxiety and depression symptoms, and significant increases in expectancies for improvement and self efficacy from their first to their last session of treatment, in ways that do not differentiate among the four targeted anxiety disorders. Also, results did not vary as a function of whether or not patients are also receiving psychotropic medications, with one exception being a limiting effect of benzodiazepines upon improvements in self efficacy expectations. Patient rated understanding of the material (Calm Living, which focuses on exposure to feared situations) not only correlated with changes in anxiety symptoms from the first to the last session, but was the only significant predictor of OASIS scores at the final session, above and beyond first session OASIS scores, type of anxiety disorder targeted, number of sessions attended, and efficacy and outcome expectancies at the first session. These data provide some (albeit indirect) evidence for the symptom outcomes to be attributable to the Calm Tools for Living program rather than nonspecific factors. The ultimate test of this approach's clinical effectiveness will be determined by results from the RCT, which will not be complete until December 2009.

Several questions warrant further investigation, including the degree to which the results generalize to different types of clinicians with different levels of training. Moreover, the generalizability of the findings from primary care patients who are offered both CBT and medication to those offered only CBT warrants investigation. Another question is the extent to which modifications to the computerized program, based on feedback from the clinicians described herein as well as refinement in measurement using quiz questions, improves outcomes. Future research also will be needed to evaluate the cost-effectiveness of computer-assisted programs – with varying levels of clinician involvement – in primary care. Prior research (37, 43, 38, 45) has indicated greater satisfaction with and engagement in computerized programs when they are complemented by clinicians than when they are conducted without a clinician. Hence, it is reasonable to assume that our computer-assisted program involving a clinician would also improve outcomes relative to a self-directed version of CALM Tools for Living. By providing on-line guidance, the computer assisted program may keep novice clinicians on task and thereby maintain CBT integrity. Whether it is superior to CBT conducted by novice therapists with the aid of manuals or workbooks remains also open to future investigation.

Acknowledgments

This work was supported by the following National Institute of Mental Health grants: U01 MH070018, U01 MH058915, U01 MH057835 and K24 MH64122, UO1 MH057858, U01-MH070022 and K24 MH065324.

The authors thank Lori Zoellner, Ph.D. Brett Litz, Ph.D., and Edna Foa, Ph.D. for their contributions to the PTSD module of the cognitive behavioral program, and to Kristin Bumgardner for her help with data preparation.

Footnotes

There are no competing interests for the authors of this manuscript.

References

1. Kessler RC. Lifetime Prevalence and Age-of-Onset Distributions of DSM-IV Disorders and the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62:593–602. [PubMed] [Google Scholar]

2. Silverstone T, Turner P. Drug Treatment in Psychiatry. London: Rutledge; 1995. [Google Scholar]

3. Stein MB. Anxiety Disorders: Somatic Treatment. In: Kaplan HI, Sadock BJ, editors. Comprehensive Textbook of Psychiatry. Baltimore, Maryland: Williams & Wilkins; 2005. [Google Scholar]

4. Roy-Byrne PP, Cowley DS. Pharmacological treatments for panic disorder, generalized anxiety disorder, specific phobia and social anxiety disorder. In: Nathan PE, Gorman JM, editors. A Guide to Treatments that Work. 3nd. New York: Oxford University Press; 2007. [Google Scholar]

5. Norton PJ, Price EC. A meta-analytic review of adult cognitive-behavioral treatment outcome across the anxiety disorders. Journal of Nervous and Mental Disease. 2007;195:521–531. [PubMed] [Google Scholar]

6. Butler AC, Chapman JE, Forman E, et al. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clinical Psychology Review. 2006;26:17–31. [PubMed] [Google Scholar]

7. Sullivan G, Craske MG, Sherbourne C, et al. Design of the Coordinated Anxiety Learning and Management study: Innovations in collaborative care for anxiety disorders. General Hospital Psychiatry. 2007;29:379–387. [PMC free article] [PubMed] [Google Scholar]

8. Mergl R, Seidscheck I, Allgaier A, et al. Anxiety & Somatoform Disorders in primary care: Prevalence and Recognition. Depression and Anxiety. 2007;24:185–195. [PubMed] [Google Scholar]

9. Marcus SC, Olfson M, Pincus HA, et al. Self-reported anxiety, general medical conditions, and disability bed days. American Journal of Psychiatry. 1997;154:1766–1768. [PubMed] [Google Scholar]

10. Stein MB, Sherbourne CD, Craske MG, et al. Quality of care for primary care patients with anxiety disorders. American Journal of Psychiatry. 2004;161:2230–2237. [PubMed] [Google Scholar]

11. Young AS, Klap R, Sherbourne CD, et al. The quality of care for depressive and anxiety disorders in the United States. Archives of General Psychiatry. 2001;58:55–61. [PubMed] [Google Scholar]

12. Kroenke K, Spitzer RL, Williams JB, et al. Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection. Annals of Internal Medicine. 2007;146:317–325. [PubMed] [Google Scholar]

13. Craske MG, Edlund MJ, Sullivan G, et al. Perceived unmet need for mental health treatment and barriers to care among patients with panic disorder. Psychiatric Services. 2005;56:988–994. [PubMed] [Google Scholar]

14. Katon W, Russo J, Sherbourne C, et al. Incremental cost-effectiveness of a collaborative care intervention for panic disorder. Psychological Medicine. 2006;36:353–363. [PubMed] [Google Scholar]

15. Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care: A randomized controlled trial. Journal of the American Medical Association. 2000;283:212–220. [PubMed] [Google Scholar]

16. Simon GE, Von Korff M, Ludman EJ, et al. Cost-effectiveness of a program to prevent depression relapse in primary care. Medical Care. 2002;40:941–950. [PubMed] [Google Scholar]

17. Unutzer J, Rubenstein L, Katon WJ, et al. Two-year effects of quality improvement programs on medication management for depression. Archives of General Psychiatry. 2001;58:935–942. [PubMed] [Google Scholar]

18. Roy-Byrne PP, Katon W, Cowley DS, et al. A randomized effectiveness trial of collaborative care for patients with panic disorder in primary care. Archives of General Psychiatry. 2001;58:869–876. [PubMed] [Google Scholar]

19. Rollman BL, Belnap BH, Mazumdar PR, et al. A Randomized Trial to Improve the Quality of Treatment for Panic and Generalized Anxiety Disorders in Primary Care. Archives of General Psychiatry. 2005;62:1332–1341. [PubMed] [Google Scholar]

20. Roy-Byrne PP, Craske MG, Stein MB, et al. A randomized effectiveness trial of cognitive-behavioral therapy and medication for primary care panic disorder. Archives of General Psychiatry. 2005;62:290–298. [PMC free article] [PubMed] [Google Scholar]

21. Pincus HA, Pechura C, Keyser D, et al. Depression in Primary Care: Learning Lessons in a Narional Quality Improvement Program. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33:2–15. [PubMed] [Google Scholar]

22. Brown TA, Barlow DH. Comorbidity among anxiety disorders: Implications for treatment and DSM-IV. Journal of Consulting and Clinical Psychology. 1992;60:835–844. [PubMed] [Google Scholar]

23. Brown TA, Chorpita BF, Barlow DH. Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. Journal of Abnormal Psychology. 1998;107:179–192. [PubMed] [Google Scholar]

24. Kessler RC, Nelson CB, McGonagle KA, et al. Comorbidity of DSM-III-R major depressive disorder in the general population: Results from the US National Cormorbidity Survey. British Journal of Psychiatry. 1996;168:17–30. [PubMed] [Google Scholar]

25. Brown TA, Antony M, Barlow DH. Long-term outcome in cognitive-behavioral treatment of panic disorder: Clinical predictors and alternative strategies for assessment. Journal of Consulting and Clinical Psychology. 1995;63:754–765. [PubMed] [Google Scholar]

26. Tsao JCI, Lewin MR, Craske MG. The effects of cognitive-behavioral therapy for panic disorder on comorbid conditions. Journal of Anxiety Disorders. 1998;12:357–371. [PubMed] [Google Scholar]

27. Tsao JCI, Mystkowski JL, Zucker BG, et al. Effects of cognitive-behavioral therapy for panic disorder on comorbid conditions: Replication and extension. Behavior Therapy. 2002;33:493–509. [Google Scholar]

28. Tsao JCI, Mystkowski JL, Zucker BG, et al. Impact of cognitive-behavioral therapy for panic disorder on comorbidity: A controlled investigation. Behaviour Research and Therapy. 2005;43:959–970. [PubMed] [Google Scholar]

29. Borkovec TD, Abel JL, Newman H. Effects of psychotherapy on comorbid conditions in generalized anxiety disorder. Journal of Consulting and Clinical Psychology. 1995;63:479–483. [PubMed] [Google Scholar]

30. Stanley MA, Beck JG, Glassco JD. Treatment of generalized anxiety disorder in older adults: A preliminary comparison of cognitive-behavioral and supportive approaches. Behavior Therapy. 1996;27:565–581. [Google Scholar]

31. Craske MG, Farchione TJ, Allen LB, et al. Cognitive behavioral therapy for panic disorder and comorbidity: More of the same or less of more? Behaviour Research and Therapy. 2007;45:1096–1109. [PubMed] [Google Scholar]

32. Carroll KM, Rounsaville BJ. A vision of the next generation of behavioral therapies research in the additcions. Addiction. 2007;102:850–869. [PMC free article] [PubMed] [Google Scholar]

33. Gega L, Norman IJ, Marks IM. Computer-aided vs. Tutor-delivered teaching of exposure therapy for phobia/panic: Randomized controlled trial with pre-registration nursing students. International Journal of Nursing Studies. 2007;44:397–405. [PubMed] [Google Scholar]

34. Gega L, Marks I, Mataix-Cols D. Computer-Aided CBT Self-Help for Anxiety and Depressive Disorders: Experience of a London Clinic and Future Directions. Journal of Consulting and Clinical Psychology. 2004;60:147–157. [PubMed] [Google Scholar]

35. Proudfoot J, Goldberg D, Mann A, et al. Computerized, interactive, multimedial cognitive behavioural program for anxiety and depression in general practice. Psychological Medicine. 2003;33:217–227. [PubMed] [Google Scholar]

36. Carlbring P, Bohman S, Brunt S, et al. Remote treatment of panic disorder: A randomized trial of internet-based cognitive behavior therapy supplemented with telephone calls. American Journal of Psychiatry. 2006;163:2119–2125. [PubMed] [Google Scholar]

37. Carlbring P, Nilsson-Ihrfelt E, Waara J, et al. Treatment of panic disorder: Live therapy vs. self-help via the Internet. Behaviour Research and Therapy. 2005;43:1321–1333. [PubMed] [Google Scholar]

38. Richards J, Klein B, Carlbring P. Internet based treatment for panic disorder. Cognitive Behavior Therapy. 2003;32:125–135. [PubMed] [Google Scholar]

39. Botella C, Hofmann SG, Moscovitch DA. A Self-Applied, Internet-Based Intervention for Fear of Public Speaking. Journal of Clinical Psychology. 2004;60:821–830. [PubMed] [Google Scholar]

40. Carlbring P, Furmark T, Steczkó J, et al. An open study of internet-based bibliotherapy with minimal therapist contact via email for social phobia. Clinical Psychologist. 2006;10:30–38. [Google Scholar]

41. Lange A, van de Ven J, Schrieken B, et al. Interapy. Treatment of posttraumatic stress through the Internet: A controlled trial. Journal of Behavior Therapy and Experimental Psychiatry. 2001;32:73–90. [PubMed] [Google Scholar]

42. Greist JH, Marks IM, Baer L, et al. Behavior therapy for obsessive-compulsive disorder guided by a computer or by a clinician compared with relaxation as a control. Journal of Clinical Psychiatry. 2002;63:138–145. [PubMed] [Google Scholar]

43. Kenwright M, Marks I, Graham C, et al. Brief scheduled phone support from a clinician to enhance computer-aided self-help for obsessive-compulsive disorder: Randomized controlled trial. Journal of Clinical Psychology. 2005;61:1499–1508. [PubMed] [Google Scholar]

44. Christensen H, Griffiths K, Jorm A. Delivering interventions for depression using the internet randomized controlled trial. British Medical Journal. 2004;328:265. [PMC free article] [PubMed] [Google Scholar]

45. Spek V, Cuijpers P, Nyklicek I, et al. Internet-based cognitive behavioural therapy for symptoms of depression and anxiety: a meta-analysis. Psychological Medicine. 2007;37:319–328. [PubMed] [Google Scholar]

46. Lecrubier Y, Sheehan DV, Weiller E, et al. The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. Eur Psychiatry. 1997;12(5):224–31. [Google Scholar]

47. Trivedi MH, Rush AJ, Gaynes BN, et al. Maximizing the adequacy of medication treatment in controlled trials and clinical practice: STAR(*)D measurement-based care. Neuropsychopharmacology. 2007;32:2479–2489. [PubMed] [Google Scholar]

48. Mitte K. Meta-Analysis of Cognitive-Behavioral Treatments for Generalized Anxiety Disorder: A Comparison With Pharmacotherapy. Psychological Bulletin. 2005;131:785–795. [PubMed] [Google Scholar]

49. Laberge B, Gauthier J, Cote G, et al. Cognitive-behavioral therapy of panic disorder with secondary major depression: A preliminary investigation. Journal of Consulting and Clinical Psychology. 1993;61:1028–1037. [PubMed] [Google Scholar]

50. McLean PD, Woody S, Taylor S, et al. Comorbid panic disorder and major depression: Implications for cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology. 1998;66:240–247. [PubMed] [Google Scholar]

51. Miller WR, Rollnick S. Motivational interviewing: preparing people for change. 2nd. New York: Guilford Press; 2002. [Google Scholar]

52. Newman CF. Understanding client resistance: Methods for enhancing motivation to change. Cognitive and Behavioral Practice. 1994;1:47–69. [Google Scholar]

53. Campbell-Sills L, Norman SB, Craske MG, et al. Validation of a brief measure of anxiety-related severity and impairment: The Overall Anxiety Sensitivity and Impairment Scale (OASIS) Journal of Affective Disorders. 2008 Epub ahead of print. [PMC free article] [PubMed] [Google Scholar]

54. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine. 2001;16:606–613. [PMC free article] [PubMed] [Google Scholar]