The traditional representations of games using the extensive form or the strategic form obscure much of the structure of real-world games. In this paper, we propose a graphical representation for noncooperative games --- \emph{multi-agent influence diagrams (MAIDs)}. The basic elements in the MAID representation are \emph{variables}, allowing an explicit representation of dependence, or relevance, relationships among variables. We define a decision variable $D'$ as \emph{strategically relevant} to $D$ if, to optimize the decision rule at $D$, the decision maker needs to consider the decision rule at $D'$. We provide a sound and complete graphical criterion for determining strategic relevance. We then show how strategic relevance can be used to decompose large games into a set of interacting smaller games, which can be solved in sequence. We show that this decomposition can lead to substantial savings in the computational cost of finding Nash equilibria in these games.