Background: Physical fitness assessment of older adults is essential because it is a key component of functional independence and healthy aging. Aim: To establish physical fitness reference values for physically active older Chilean adults of both sexes and identify the variables associated with the deterioration of their physical condition. Material and Methods: Cross-sectional study that included 342 older adults aged 60 and over. Their physical fitness was assessed with the Senior Fitness Test (SFT). The timed up and go (TUG), chair stand (CS), arm curl (AC), and aerobic resistance (2 min) tests were evaluated. Results: Performance in the physical fitness tests by age group decreased in all tests as older adults advanced in age. Scores for men were more evenly distributed across the different age groups. The main risk factors for the deterioration of physical fitness were age, sex, and body mass index (BMI) (p-value < 0.05). The primary risk factor for men was age and for women age and BMI. Conclusions: Performance of both men and women in the different SFT tests decreased as older adults aged. Age, sex, and BMI were the main risk factors for the deterioration of the physical fitness of physically active older adults.