Sound cyber security testing is a critical challenge, in particular for large and complex systems such as the smart grid. In this paper, we explore the need for, and specific issues involved in, security testing for smart grid components and standards and how testbeds play a critical role in that environment. We present three main problems; the need for a methodology to define the appropriate tests, the need for a means of comparing and measuring the security of the system under scrutiny, and the current lack of tools and instrumentation with which to carry out those tests. We present work addressing those problem areas through approaches in methodology, quantification of security, formal methods, and tool creation. We illustrate our approach through a case study showing applications of these techniques to the Advanced Metering Infrastructure (AMI) protocol space and discuss advances in cyber-physical testbed experimentation that ease this testing at scale.