Gröbner geometry of vertex decompositions and of flagged tableaux

We relate a classic algebro-geometric degeneration technique, dating at least to Hodge 1941 (J. London Math. Soc. 16: 245-255), to the notion of vertex decompositions of simplicial complexes. The good case is when the degeneration is reduced, and we call this a geometric vertex decomposition. Our main example in this paper is the family of vexillary matrix Schubert varieties, whose ideals are also known as (one-sided) ladder determinantal ideals. Using a diagonal term order to specify the (Gröbner) degeneration, we show that these have geometric vertex decompositions into simpler varieties of the same type. From this, together with the combinatorics of the pipe dreams of Fomin-Kirillov 1996 (Discr. Math. 153: 123-143), we derive a new formula for the numerators of their multigraded Hilbert series, the double Grothendieck polynomials, in terms of flagged set-valued tableaux. This unifies work of Wachs 1985 (J. Combin. Th. (A) 40: 276-289) on flagged tableaux, and Buch 2002 (Acta. Math. 189: 37-78) on set-valued tableaux, giving geometric meaning to both. This work focuses on diagonal term orders, giving results complementary to those of Knutson-Miller 2005 (Ann. Math. 161: 1245-1318), where it was shown that the generating minors form a Gröbner basis for any antidiagonal term order and any matrix Schubert variety. We show here that under a diagonal term order, the only matrix Schubert varieties for which these minors form Gröbner bases are the vexillary ones, reaching an end toward which the ladder determinantal literature had been building.