Brain atlases contain a wealth of information that could be used in radiation therapy or neurosurgical planning. Until now, however, when large space-occupying tumors and lesions drastically alter the shape of brain structures and substructures, atlas-based methods have been of limited use. In this work, we present a new technique that permits a brain atlas to be warped onto image volumes in which large lesions are present. First we show that a method previously used for atlas-based segmentation of normal brains can also be used for brains with small lesions. We then present an extension of this technique for brains with large lesions. This involves several steps: a global registration to bring the two volumes into approximate correspondence; a local registration to warp the atlas onto the patient volume; the seeding of the warped atlas with a tumor model derived from patient data; and the deformation of the seeded atlas. Global registration is performed using a mutual information criterion. The method we have used for atlas warping is derived from optical flow principles. Preliminary results obtained on real patient images are presented. These results indicate that the proposed method can be used to automatically segment structures of interest in brains with gross deformation. Potential areas of application for this method include automatic labeling of critical structures for radiation therapy and presurgical planning.