Yes, unless there is some motion like falling dust and debris after an explosion, etc. then you don't need to over-crank to create a fake sense of mass. Many spaceship models, for example, are shot at very low frame rates with very slow-moving cameras in order to get more exposure on the models, because it is important to get as much depth of field as possible.
Think of it this way, if your miniature was a real building and was 50' tall and 100 feet away, then you'd expect that most of it would fall into focus on a camera lens even at a wide aperture. But if your miniature is only 1' tall and only 2' from the camera lens, then you'd have to use a very deep f-stop to hold most of it in focus.
You also want to think of camera height in terms of scale, do you want to get the lens down to the height of a person taking a picture if this were a real building.