Leaders should recognise and act on dangers facing people living on the margins in urban centres, a report states.