I would say that birmingham is still to far north for an olive grove to be sustainble, the trees will grow ok but you will get poor fruit set a evenworse fruit development.
I live in morecambe lancashire and have a 100+ year old olive tree in my back yard it flowers well set some fruit and...