Pancreatic cancer occurs in the setting of a profound fibrotic microenvironment that often dwarfs the actual tumor. Although pancreatic fibrosis has been well studied in chronic pancreatitis, its development in pancreatic cancer is much less well understood. This article describes the dynamic remodeling that occurs from pancreatic precursors (pancreatic intraepithelial neoplasias (PanINs)) to pancreatic ductal adenocarcinoma, highlighting similarities and differences between benign and malignant disease. Although collagen matrix is a commonality throughout this process, early stage PanINs are virtually free of periostin while late stage PanIN and pancreatic cancer are surrounded by an increasing abundance of this extracellular matrix protein. Myofibroblasts also become increasingly abundant during progression from PanIN to cancer. From the earliest stages of fibrogenesis, macrophages are associated with this ongoing process. In vitro co-culture indicates there is cross-regulation between macrophages and pancreatic stellate cells (PaSCs), precursors to at least some of the fibrotic cell populations. When quiescent PaSCs were co-cultured with macrophage cell lines, the stellate cells became activated and the macrophages increased cytokine production. In summary, fibrosis in pancreatic cancer involves a complex interplay of cells and matrices that regulate not only the tumor epithelium but the composition of the microenvironment itself.