Identification of the causative microorganism is important in the management of pyogenic vertebral osteomyelitis (PVO). The aim of this study was to investigate whether culture positive rates differ between needle biopsy sites in patients with PVO, and which tissues are best for microbiological diagnosis. Between January 2005 and December 2013, we conducted a retrospective cohort study of PVO patients who had soft-tissue abscesses (paraspinal or psoas abscesses) and who received needle biopsy for microbiological diagnosis. Needle biopsy sites were classified into two anatomical categories: vertebral bodies, or soft tissues (intervertebral discs, paraspinal abscesses, or psoas abscesses). A generalized estimating equation model was developed to identify factors associated with tissue-culture positivity. During the study period a total of 136 tissues were obtained by needle biopsy from 128 PVO patients with soft-tissue abscesses. The culture positive rates of vertebral bodies and soft tissues were 39.7% (29/73), and 63.5% (40/63), respectively (p < 0.05). In a multivariate analysis, male gender (adjusted odds ratio (aOR) 2.24, 95% CI 1.00-5.02), higher C-reactive protein (aOR 1.07, 95% CI 1.01-1.15), positive blood culture (aOR 2.57, 95% CI 1.01-6.59), and soft tissues as biopsy site compared with vertebral bodies (aOR 2.28, 95% CI 1.08-4.78) were independent factors associated with tissue culture positivity. Soft tissues were the best sites for microbiological diagnosis in PVO patients undergoing needle biopsy.