The study aimed to elucidate the relationship between sex steroids and muscle mass, muscle strength, and trabecular bone score (TBS) in a community-dwelling aged population. We analyzed 922 men > 60 years of age and 1244 postmenopausal women. Weak muscle strength was defined as hand grip strength < 26 kg for men and < 18 kg for women, whereas degraded bone microarchitecture was defined as a TBS ≤ 1.2. The mean age was 70.2 ± 6.8 years for men and 71.2 ± 6.7 years for women. Participants within higher dehydroepiandrosterone sulfate (DHEAS) and free testosterone (FT) tertiles were likely to be younger, have greater muscle mass, and have stronger hand grip strength. Based on logistic regression models, men within the lowest FT tertile had weaker muscle strength compared to those in the highest tertile (adjusted odds ratio [OR] 2.28; 95% confidence interval [CI] 1.33-3.91). Women within the lowest DHEAS and FT tertile had weaker muscle strength compared to those in the highest tertile (adjusted OR for DHEAS 1.42; 95% CI 1.02-1.99; adjusted OR for FT 1.77, 95% CI 1.26-2.48). Moreover, men within the lowest FT tertile exhibited degraded bone microarchitecture compared to those in the highest tertile (adjusted OR 2.57, 95% CI 1.46-4.51). However, estradiol was not related to muscle strength or bone microarchitecture in both sexes. In conclusion, in aged men, serum FT was closely associated with muscle strength and bone microarchitecture and in postmenopausal women, serum DHEAS and FT were related to muscle strength.