Trabecular bone is a porous nanocomposite material with a hierarchical structure. In this study, a multi-scale modeling approach, addressing scales spanning from the nanometer (collagen-mineral) to mesoscale (trabecular bone) levels, was developed to determine the elastic moduli of trabecular bone. Then, the predicted modeling results were compared with experimental data obtained by compression testing of bovine femur trabecular bone samples loaded in two different directions; parallel to the femur neck axis and perpendicular to that. Optical microscopy, scanning electron microscopy and micro-computed tomography techniques were employed to characterize the structure and composition of the samples at different length scales and provide the inputs needed for the modeling. To obtain more insights on the structure of bone, especially on the interaction of its main constituents (collagen and mineral phases), trabecular bone samples were deproteinized or demineralized and, afterwards, tested mechanically in compression. The experimental observations were used, in turn, to fine-tune the multi-scale model of bone as an interpenetrating composite material. Good agreement was found between the theoretical and experimental results for elastic moduli of untreated, deproteinized, and demineralized trabecular bones.