Individuals with autism spectrum disorders (ASDs) have difficulty communicating in ways that are primarily for initiating and maintaining social relatedness (i.e., social communication). We hypothesized that the way researchers measured social communication would affect whether treatment effects were found. Using a best evidence review method, we found that treatments were shown to improve social communication outcomes approximately 54% of the time. The probability that a treatment affected social communication varied greatly depending on whether social communication was directly targeted (63%) or not (39%). Finally, the probability that a treatment affected social communication also varied greatly depending on whether social communication as measured in (a) contexts very similar to treatment sessions (82%) or (b) contexts that differed from treatment on at least setting, materials, and communication partner (33%). This paper also provides several methodological contributions.