Bilinear inverse problems (BIPs), the resolution of two vectors given their image under a bilinear mapping, arise in many applications, such as blind deconvolution and dictionary learning. However, there are few results on the uniqueness of solutions to BIPs. For example, blind gain and phase calibration (BGPC) is a structured bilinear inverse problem, which arises in inverse rendering in computational relighting, in blind gain and phase calibration in sensor array processing, in multichannel blind deconvolution (MBD), etc. It is interesting to study the uniqueness of solutions to such problems. In this paper, we define identifiability of a bilinear inverse problem up to a group of transformations. We derive conditions under which the solutions can be uniquely determined up to the transformation group. Then we apply these results to BGPC and give sufficient conditions for unique recovery under subspace or joint sparsity constraints. For BGPC with joint sparsity constraints, we develop a procedure to determine the relevant transformation groups. We also give necessary conditions in the form of tight lower bounds on sample complexities, and demonstrate the tightness by numerical experiments.