Plasmon-resonant gold nanorods have been demonstrated recently as contrast agents for optical coherence tomography (OCT). To evaluate their ability to produce contrast in a structurally heterogeneous environment, nanorods were injected at discrete locations into an excised sample of human breast invasive ductal carcinoma. The distribution of nanorods within the tissue was revealed using spectroscopic OCT imaging techniques, by analyzing the evolution of the backscattered light spectrum over tissue depth. We compare a variety of signal processing methods including spatial averaging and least-squares fitting to the a priori extinction spectrum of the nanorods, with the goal of optimizing the detection sensitivity to the nanorods in these tissues. Because nanorods can be conjugated with proteins specific to biomolecular targets, they may potentially be used with these imaging methods to provide molecular contrast in human tissues.