Cholangiocarcinoma (CCA), especially intrahepatic CCA, is known to share several risk factors with hepatocellular carcinoma (HCC) and liver cirrhosis has been proposed as a common pathogenic factor. We aimed to identify the risk factors of CCA and to examine differences in risk factors between CCA and HCC. We followed 510,217 Korean adults who underwent health checkups during 2002−2003 until 2013 via linkage to national hospital discharge records. Hazard ratios (HRs) were calculated after adjustment for confounders. During the mean follow-up of 10.5 years, 1388 and 2920 individuals were diagnosed with CCA and HCC, respectively. Choledocholithiasis (HR = 13.7; 95% confidence interval (CI) = 7.58−24.88) was the strongest risk factor for CCA, followed by cholelithiasis (HR = 2.94) and hepatitis B virus (HBV) infection (HR = 2.71). Two of the strongest risk factors for HCC—liver cirrhosis (HR = 1.29; 95% CI = 0.48−3.45) and hepatitis C virus infection (HR = 1.89; 95% CI = 0.49−7.63)—were not significantly associated with the risk of CCA. HBV infection and diabetes increased the risk of both HCC and CCA, but the HRs were lower for CCA than for HCC (Pheterogeneity < 0.001 for HBV; Pheterogeneity = 0.001 for diabetes). The magnitudes of the effects of age, sex, obesity, alcohol consumption, and smoking on the development of both cancers were different (Pheterogeneity < 0.05 for each variable). In conclusion, choledocholithiasis, cholelithiasis, HBV, older age, male sex, diabetes, smoking, alcohol drinking, and obesity were found to be potential risk factors of CCA. Liver cirrhosis did not increase the risk of CCA. The magnitudes of the potential effects of common risk factors were generally different between CCA and HCC.