I’ve definitely been in your shoes, trying to figure out the best way to count based on dates in Excel. I think you're on the right track with COUNTIF, but it sounds like you need to set up the criteria properly for it to count dates within specific ranges. What I did in a similar situation was use a formula like this:
=COUNTIF(A:A, ">"&TODAY()-365)
This counts all devices over 1 year old, and you can adjust it for 3 and 5 years by changing the number (e.g., 1095 for 3 years). I also picked up a few great tips from excel.tv on handling date-based conditions that really helped me get a more accurate count.