Under fire for their alleged involvement in illegal mining in Karnataka and Andhra Pradesh, the Reddy brothers of Bellary asserted on Monday that Karnataka Governor H R Bhardwaj had no authority to refer a petition seeking their disqualification from the state legislature to the Election Commission.