Quantification of fat and muscle on clinically acquired CT scans is critical for determination of body composition, a key component of health. Manual tracing has been regarded as the gold standard method of body segmentation; however, manual tracing is time-consuming. Many semi-automated/automated algorithms have been proposed to avoid the manual efforts. Previous efforts largely focused on segmenting 2D cross-sectional images (e.g., at L3/T4 vertebra locations) rather than on the whole-body volume. In this paper, we propose a fully automated 3D body composition estimation framework for segmenting the muscle and fat from abdominal CT scans. The 3D whole body segmentations were reconstructed from a slice-wise multi-atlas label fusion (MALF) based framework. First, we used a low-dimensional atlas representation to estimate each class for each axial slice. Second, the abdominal wall and psoas muscle were segmented by combining MALF with active shape models and deformable models. Third, skeletal muscle, visceral adipose tissue (VAT) and subcutaneous adipose tissue (SAT) were measured to assess the areas of muscle and fat tissue. The proposed method was compared to manual segmentation and demonstrated high accuracy. Then, we evaluated the approach on 40 CT scans comparing the new method to a prior atlas-based segmentation method and achieved 0.854, 0.740, 0.887 and 0.933 on Dice similarity index for the skeletal muscle, psoas muscle, VAT and SAT, respectively. Compared with the baseline, our method showed significantly ( < 0.001) higher accuracy on skeletal muscle, VAT and SAT estimation.