We kept bees in Italy for several years and had olive trees as well. We occasionally saw bees visiting olive blossom, but it was never systematically worked. For all I know, what we saw were new foragers having a quick check and then deciding, "Nah, can't be bothered," and moving on.
As Juanito...