Phosphorylation of Smad1 at the conserved carboxyl terminal SVS sequence activates BMP signaling. Here we report the crystal structure of the Smad1 MH2 domain in a conformation that reveals the structural effects of phosphorylation and a molecular mechanism for activation. Within a trimeric subunit assembly, the SVS sequence docks near two putative phosphoserine binding pockets of the neighboring molecule, in a position ready to interact upon phosphorylation. The MH2 domain undergoes concerted conformational changes upon activation, which signal Smad1 dissociation from the receptor kinase for subsequent heteromeric assembly with Smad4. Biochemical and modeling studies reveal unique favorable interactions within the Smad1/Smad4 heteromeric interface, providing a structural basis for their association in signaling.