Real-world intelligence includes the ability to handle complex, uncertain, dynamic, multi-modal information in real time. In order to pursue the artificial realization of such "human" or "intelligent" information processing, a novel system of representing and interpreting knowledge must first be developed. This book collects the results of ten years of research at six laboratories, focusing on the theoretical and algorithmic foundations of the intelligence we find in the real world.