Qualitative spatial reasoning (QSR) pursues a symbolic approach to reasoning about a spatial domain. Qualitative calculi are defined to capture domain properties in relation operations, granting a relation algebraic approach to reasoning. QSR has two primary goals: providing a symbolic model for human common-sense level of reasoning and providing efficient means for reasoning. In this paper, we dismantle the hope for efficient reasoning about directional information in infinite spatial domains by showing that it is inherently hard to decide consistency of a set of constraints that represents positions in the plane by specifying directions from reference objects. We assume that these reference objects are not fixed but only constrained through directional relations themselves. Known QSR reasoning methods fail to handle this information.