Although hepatitis B virus (HBV) and hepatitis C virus (HCV) are hepatotrophic viruses, they may affect pulmonary diseases. The purpose of this study was to assess whether chronic viral hepatitis (CVH) infection was associated with a rapid decline in lung function. Repeated measurements of lung function were obtained from a well-curated health check-up database. A case was defined as an individual positive for HBsAg or anti-HCV antibody. A control was randomly selected (from the same dataset) after 1:1 matching in terms of age, sex, height, the body mass index, and smoking status. Separate analyses of non-smokers and smokers were performed. A total of 701 cases were enrolled (586 with HBV and 115 with HCV). In cross-sectional analysis, both forced expiratory volume in 1 s (FEV1) and forced vital capacity (FVC) decreased significantly only in smokers (smoking cases vs. smoking controls) (adjusted p = 6.6 × 10^-5 and adjusted p = 2.2 × 10^-3, respectively). In longitudinal analysis, smoking cases showed significantly greater FEV1 and FVC decline rates than did smoking controls (adjusted p = 8.5 × 10^-3 and adjusted p = 1.2 × 10^-5, respectively). Such associations were particularly high in smoking cases at intermediate-to-high risk of hepatic fibrosis, as evaluated by the non-invasive Fibrosis-4 index. In summary, CVH was associated with both decreased lung function and accelerated lung function decline in smokers. A non-invasive measurement of hepatic fibrosis may be useful in predicting rapid lung function decline in smokers with CVH.