Early differentiation of mycobacteria in sputa is crucial. This study was set to evaluate the usefulness of a newly developed duplex polymerase chain reaction (PCR) for hsp65 gene-based method in differentiating mycobacteria in sputum with a positive acid-fast bacilli (AFB) smear before culturing. One hundred forty-seven sputa with positive AFB smear were included for the analysis. Mycobacterial species were identified using a newly developed duplex PCR for hsp65 gene followed by a nested PCR-direct sequencing and the conventional colony-based method. Final decision of mycobacterial species were made based on 1) results of species identification based on mycobacterial colonies or 2) results of species identification of other sputa from the same patients and clinical findings. The duplex PCR-based method correctly identified 83.2% sputa from tuberculosis patients and 82.2% sputa from nontuberculous mycobacteria patients, whereas the colony-based method correctly identified 86.1% and 77.8%, respectively. Sensitivity and specificity of the colony-based method for Mycobacterium tuberculosis were 86.1% and 100%, respectively, whereas those of the duplex PCR-based method were 83.2% and 95.6%, respectively. The duplex PCR-based method, to differentiate mycobacterial species in sputa, produced comparable results as those of the colony-based identification method.