Tuning three-dimensional higher-order topological insulators by surface state hybridization

Higher-order topological insulators (HOTIs) are a novel class of materials that exhibit exotic boundary states. The finite size effect induced hybridization between the boundary state HOTIs, however, remains largely unexplored. In this work, we analytically and numerically study the hybridization in films of three-dimensional chiral second-order topological insulators (SOTIs). We show that the gaps of two gapped surfaces (Chern insulators) are increased upon reducing the film thickness to a finite value. Meanwhile, in the presence of the hybridization of chiral edge states, we find that the gapless chiral hinge states (i.e., the boundary states of the surface Chern insulators) can be gapped, destroying the SOTIs. Moreover, we show that the gapless surfaces would in general be gapped due to the hybridization of surface states for a finite-sized system. Our work demonstrates that HOTIs exhibit a rich variety of physics under the influence of finite-size effects