What is the effect of leaving the narrator's crime unspecified?

Great question.  By leaving the narrator's crime unspecified, Poe has enlisted the help of the reader's imagination.  A reader's imagination will always take things way past what a writer is generally capable of writing about.  We just assume the absolute worst.  The narrator's punishment is quite awful.  He's put in a dark room with a pit.  The jailers are hoping he falls in.  Then there's the whole pendulum part and trying to cut him in half.  Finally, there's the burning and squeezing walls.  Whatever the narrator did, his crime must have been terrible to deserve such horrors!  When a reader thinks along that line, the reader then starts hypothesizing all of the horrible crimes that the narrator might have committed.  And my imagination can be quite vivid.  Modern film occasionally uses an audience's imagination effectively as well.  I think back to the original Jaws and Alien. Those two films were terrifying, and they both hardly ever allowed the audience to see the death giving monster.  My imagination made the fear so much more visceral.  That's the effect of Poe leaving things unknown to his readers. 

The other effect of leaving the narrator's crime unknown is that the reader is at times just as confused as the narrator himself.  Consequently, we are better able to relate and empathize with the narrator. We understand his confusion, because we are confused and "kept in the dark" ourselves.