While disaster studies researchers usually view risk as a function of hazard, exposure, and vulnerability, few studies have systematically examined the relationships among the various physical and socioeconomic determinants underlying disasters, and fewer have done so through seismic risk analysis. In the context of the 1999 Chi-Chi earthquake in Taiwan, this study constructs five hypothetical models to test different determinants that affect disaster fatality at the village level, namely seismic hazard intensity, population, building fragility, demographics and socioeconomics. The Poisson Regression Model is used to estimate the impact of natural hazards and social factors on fatality. Results indicate that although all of the determinants have an impact on the specific dimension of seismic fatality, some indicators of social inequality, such as gender ratio, dependency ratio, income and its SD, are the driving determinants deteriorating vulnerability to seismic risk. These findings have strong social implications for policy interventions to mitigate such disasters. This study presents an interdisciplinary investigation into social and physical determinants in seismic risk.