Depression and low self-efficacy are both associated with worse glycemic control in adults with diabetes, but the relationship between these variables is poorly understood. We conducted a cross-sectional study examining associations between depressive symptoms, self-efficacy, and glycemic control among men (n = 64) and women (n = 98) with type 2 diabetes to see if self-efficacy mediates the relationship between depression and glycemic control. Correlational and mediational analyses examined the relationship between these three variables for the sample as a whole and separately by sex. A significant association between depressive symptoms and glycemic control was found for men (0.34, P < 0.01) but not for women (0.05, P = 0.59). Path analysis suggested that, among men, self-efficacy mediates the relationship between depressive symptoms and glycemic control. We conclude that men with depressive symptoms and type 2 diabetes may need tailored interventions that improve their self-efficacy in order to achieve glycemic control.