How easy is it to pursue a science career later in life?

I currently work in the public sector but learned later in life that my real passion is in the sciences. I'm considering reentering university to pursue a science degree. In all of the science podcasts and media I consume, I don't hear much about people entering sciences as a second career. Is this a realistic goal; if so, would you have any recommendations. Massive fan of the Naked Scientists team!