The implementation of electricity markets in various U.S. jurisdictions has resulted in multiple markets at different time scales for trading the same electricity commodity. This multi-settlement system allows the participation of players with physical resources and loads, as well as financial entities. Given the strong interactions of market and system operations, the multiple markets and the participation of financial players make the operational reliability management an increasingly challenging task. While the nature of the interactions between the system and market operations is well understood on a qualitative basis, the quantification of the operational reliability impacts on the overall economics of electricity markets is, typically, not performed. In this paper, we develop a general approach to quantify the market performance as a function of operational reliability criterion in a multi-settlement environment while taking into account the participation of players with physical resources and load, as well as financial entities. The quantification provides meaningful measures of the monetary and the resource dispatch impacts of the compliance with the operational reliability criterion in force. We illustrate the application of the proposed approach to the ISO-NE system in the quantification of the comparative impacts of two different security criteria for the 2005 - 2006 day-ahead and real-time markets. Through this study, we gain insights into the distinctions between day-ahead and real-time markets, as well as the impacts of participating financial entities in addition to those with physical generation assets or loads.