The cortical organization of the semantic network has been studied extensively in neuropsychological and neuroimaging studies. Recent theories have heavily relied on the observation of category-specific activations, i.e., the preferential activations in brain regions for specific semantic categories. With decades of research, a full understanding of the organization has not yet been reached, since little is known about the factors that contribute to the variances in observed activation patterns across numerous neuroimaging studies. In this study, we first reviewed 97 published papers that reported category-specific activations for living or nonliving concepts in the past two decades. Then, using the Activation Likelihood Estimate (ALE) method, we characterized the brain activation associated with living and nonliving concepts, revealing the influences of relevant factors (e.g., neuroimaging mode, task demands, and stimuli modality), and analyzing these findings in relation to theoretical accounts of cortical semantic networks.