People with very low levels of physical activity may have smaller brains later in life, a new study found.

“We found a direct correlation in our study between poor fitness and brain volume decades later, which indicates accelerated brain aging,” said study author Nicole Spartano, PhD, with Boston University School of Medicine in Boston.

The study spanned over two decades, with 1,583 people; they took a treadmill test and MRI brain scans and repeated the same things 20 years later. After researchers excluded participants who suffered from heart diseases and other similar factors, they were still left with 1,094 people – a pretty big sample size.

Their peak VO2, or the maximum amount of oxygen the body is capable of using in one minute was correlated with brain size. For every eight units lower on the VO2 scale, their brain volume two decades later was smaller, equivalent to two years of accelerated brain aging. Furthermore, people who had very high heart rates during the tests also had smaller brains later in life.

This study is only observational and reports a correlation – not a causation. In other words, it doesn’t prove that poor physical fitness causes smaller brains.

“While not yet studied on a large scale, these results suggest that fitness in middle age may be particularly important for the many millions of people around the world who already have evidence of heart disease,” she said.