This article investigates the complex entanglement of neutrality and displacement in Hong Kong and Macau with a focus on the impact of and responses to an unprecedented influx of refugees during the Second World War. Displaced persons were of central importance in shaping the ambiguous experience of neutrality before Hong Kong's occupation by Japan in late 1941 and until the end of the war in Macau. Building on Elizabeth Sinn's conceptualization of Hong Kong as an ‘in-between place’, this article considers these two foreign-ruled territories as ‘in-between places’ where multi-layered transborder flows developed in an ‘in-between time’ of neutrality. Highlighting similarities and connections between Hong Kong and Macau, it argues that neutrality was shaped by the movement of refugees and that refugees often experienced neutrality differently depending on perceptions of race, class, and nationality. The presence of diverse communities of refugees shaped multiple dimensions of urban life, with colonial concerns for spatial order and social control co-existing with humanitarian co-operation. The discourses and practices around refugees are an important precedent to understanding post-war refugeedom in these territories.