Interest has increased in recent years in "comparative effectiveness," that is, assessing the efficacy of new or established medical interventions, with particular emphasis on head to head comparisons of established therapies, or understanding their real-world performance. Randomized controlled trials (RCTs), while the ostensible gold standard for establishing efficacy and sometimes effectiveness, have well recognized liabilities, most notably the time and expense it often takes to mount them, as well as the sometimes limited scope of the questions they address. Alternatives to RCTs include a variety of observational designs. Those attracting considerable attention are typically derived from very large databases, often assembled for non-research purposes, such as hospital billing, reimbursement, prescription data, electronic patient records, etc. Studies derived from such data sources promise real-world relevance, and relatively rapid results, compared to some RCTs. The middle ground is occupied by observational designs with original data gathering and RCTs that utilize surrogate endpoints-for example, death versus tumor progression, elevated LDL or coronary artery narrowing versus MI or death. A dilemma facing patients, physicians, regulatory entities, insurance providers, guideline developers and others with an interest in evidence assessment involves: (1) how pertinent existing RCT evidence is to the decisions they have to make and (2) how informative and reliable results from either observational designs or RCTs that use surrogate outcomes are in determining either efficacy or effectiveness. It is generally recognized that observational designs are subject to subtle biases that can have large effects (e.g., WHI), and that data not gathered for research purposes often lacks the precision or validity to make reliable inferences. The main approaches to these problems currently being discussed are three-fold; improving the quality and completeness of the underlying data, using innovative statistical methodologies to diminish the effects of confounding, and the development of evidence grading schemes to distinguish reliable from unreliable evidence. The ultimate goal of such efforts is to derive conclusions through these approaches that are nearly as reliable and perhaps more relevant for policy purposes than RCTs. What is notably absent from these conversations is the role that should be played by knowledge of mechanism, and how this can help in the evaluation of observational evidence, including the detection of effect modification (e.g., "personalized medicine"). With the ascendance of the evidence-based medicine, there is no formal role for mechanistic knowledge in the evidence-evaluation framework. At best, mechanistic knowledge comes in indirectly, through the choice of endpoints, target populations, and perhaps under the vague rubric of "biological plausibility." But nowhere in any of the evidence grading schemas, new statistical methodologies or other technology assessment guidelines do we have a formal language and structure for how knowledge of how an intervention works should enter the process. The closest we have is in the prior probability distribution functions of Bayesian approaches, but this begs the question of how to reliably determine how much mechanistic knowledge is worth. Our objective was to identify and pilot test a framework for the evaluation of the evidential weight of mechanistic knowledge in evidence-based medicine and technology assessment.