Most tire manufacturers recommend warehouse inventory is OK up to about 5 years. Within that time, age isn't an issue for tires sold. Another way to look at it is most manufacturers have tires with 80k to 90k wear, and this is probably the most popular segment of all season tires.
Average drivers are looking at between 12k and 15k miles per year. If manufacturers are making 80k warranty tires, it would take the average consumer 6 to 7 years to reach worn.
There are savy consumers, such as yourself and many others that replace their tires well before worn, but many only replace them when they have a catastrophic failure or fail to pass inspection.
When tire are stored out of the sun and away from ozone generating equipment, cracks are not likely to appear. Ozone is the enemy, and road hazards that get improperly repaired allowing water to reach the belt package and cause failure (delamination). Tires can fail in the bead area, but most fail in the tread.
I hope that makes it more clear, you are doing fine by replacing early, but honestly most don't.