Its bees, so its multifactorial, AND individuals have their own set ways …
1/ Bees don't like new. So a new box, with new frames and foundation is UNattractive.
Hence my suggestion of putting such boxes UNDER the brood for a week or two before 'use' so that the enforced bee traffic through the box knocks a bit of newness out of it and makes it more readily accepted.
2/ Its getting a bit late in the season for starting new boxes. Certainly for hoping to get a crop in new boxes. So don't expect too much!
If your brood box isn't fully drawn out, THAT is what you should be concentrating on. It will be needed for stores of course, but before that you want to build the colony numbers as much as possible.
The best insurance going into winter is more bees. (Bees are even more important than stores! You can, if needs be supply extra feed - fondant - before winter ends.)
3/ At this time of year, only look to getting shallow frames drawn AFTER the brood box is fully drawn.
If your shallow ("super') frames have hoffman spacers, you could put a couple in the brood box for a week or so (maybe less) to get them started with comb-drawing. Don't leave them there too long - you just want them drawn, not used. A see-through cover board makes this easier to keep a check on. Potentially replace with another couple for the next week or so. If you don't have hoffman shallow frames, you can get "hoffman converter clips" from Thornes to temporarily facilitate this move.
To make room, remove stores-only frames for the drawing period, and put one shallow frame each side between brood and stores.
If your supers have 10-slot castellations, the wide spacing itself isn't making it as easy as possible for comb-drawing. Nowadays, I use those hoffman converter clips briefly and fit 12 frames to a box, to get them started - before moving the frames onto 10-slot castellations (and retrieving the converter clips) once they are drawn (and definitely before they are capped).
One downside to working on getting a super drawn before Autumn is that you will have to protect that drawn comb from wax moth until you can bring it into play next Spring.
4/ In Spring, stores in the brood box, restricting brooding space, increases the risk of swarming, but in Autumn (soon!) the game is to get as much stored in the brood box as possible. And in Autumn, getting more comb drawn isn't easy for the bees.
So ensure the brood box is fully drawn, ASAP.
Getting the brood box fully drawn will be easier
without an empty cold super above.
As mentioned before, between brood and stores is the best place to put a frame to get it drawn. Don't add a load of foundation outside the stores frames and expect the bees to be interested in them.
5/ Sadly, they won't automatically use all the available volume "if they need it".
Oftentimes, their need for comb increases (because of brood expansion plus a sudden nectar flow) faster than the rate that the bees can make extra comb - particularly with an understrength colony. This results in backfilling the comb with nectar as the brood emerges, vacating cells. Cramping the usable brood space reduces the rate of brooding (reducing your beepower) and potentially promotes swarming. Autumn swarms do happen, even if they shouldn't - the latest that I've had a swarm callout was Sept 27th last year …
6/ The bees don't always know best. Late Sept swarming is just one example. However the beekeeper SHOULD know (even better than the bees) what is best for them - and help them to achieve it. (That is what beekeeping is about.) Right now, their prime need is a fully-drawn (and thus fully-usable) brood box.
