A federal appeals court ruled that the Bush administration did not properly study the environmental impact of expanding oil and gas drilling off the Alaska coast and canceled a program to find new reserves.