A good pilot:
1. Introduces characters we can identify with in some way.

2. Has a complete episode arc that tells exactly what the series will be.

We get humor ("Bitch." "Jerk." and "Is anything real about you, boy?" "My boobs.") and fake rock IDs and action and mystery.
3. Asks story questions that make us want to tune in next week, and the week after, and...
Not all good shows have a major story arc, but most do. If they don't, viewers know right away that the show will get stale and repetitive, and that can make us decide not to bother investing time and care into watching now.
But Supernatural easily avoided that fate. First, there's the catalyst that launches the events: Dad's on a job, and he hasn't been home in a while. Where is he? What will the boys find when they catch up with him? Then the neatly bookended mystery--whatever killed Mom has now killed Jessica, and the reluctant hero accepts the call to adventure. We've got to know what it was, and why it did it.
There's a danger in setting up a long-term mystery, too, because if it drags on too long without answers, viewers will also get bored and tune out. But at the start, we're full of anticipation and curiosity and we'll eagerly sign up for the ride.
4. Just enough backstory, but not too much.
All fiction has this balance to maintain, and it's always a struggle. We know Sam and his father have a contentious relationship. We know they had a very unconventional upbringing, that the brothers are closer than most because of it, despite their current rift, and that their mother's death drives everything they do, every choice they make. We know these things from several short lines and one powerful prologue, but the emphasis of the show is on the future. There isn't a chance to get bored.

Like anything, the episode wasn't perfect. How did Dean get to the farmhouse from the police station? And later, why did he return to Sam's apartment? His timing is just a little too coincidental. Of course, a deleted scene on the DVD shows him driving away and changing his mind, one quick shot showing us that he's not accepting Sam's decision as he turns the car around. But we didn't get that originally. Still, those small errors or oversights aren't enough to override all the good in this pilot. It made me really happy that I was a late starter and could watch as many episodes in a row as I wanted!

~~~~~~~~~~
Your turn! Post your favorite lines or details from the pilot, or tell us what your first impression of this episode was. Did it hook you right away, or grow on you? Did you see the pilot first, or start watching on TV later in the show's run, and go back?
Screencaps from Supernatural Fan Wiki and Supernatural Caps.