Supervisor of reality:
...no, it doesn't, not from that table; if anything, the fact that a Gargantuan creature takes up a 20 ft space pretty solidly suggests that Gargantuan is half the size needed here. The building is also about 50 ft tall, since it's a five floors building, so that's a 50 ft cube.
Ah right, missed that.
Looking at volume, 50 x 50 / 64 is just a little below 40, which is more than 20, so a house would indeed be Colossal.
Wondering though, if the goal is to go into the house and not be found by the opposing team, could anyone do something that makes it impossible for the seekers to actually enter the house? Like turning the doors and windows into solid wall?