The Madurai district police on Sunday arrested sand quarry baron P L Padikasu and four others in connection with illegal granite quarrying.