It is very rare for prosecutors fail to secure indictments. But you're right, the Grand Jury did hear a lot more evidence than the public.
However, because the burden of proof is so low and the are more than enough public allegations / evidence to support such a burden they would basically need to be able to show beyond a reasonable doubt or at clear and convincingly that no wrong doing occured which is very unlikely.
Much more likely is the fact that the DAs office has a vested interest in not prosecuting local police whom they work closely with or the grand jury, technically drawn from a pool of registered voters (but most often the DAs golfing buddies) disn't want to indict a police officer.
See recent Houston Chronicle investigation into this very issue at
http://m.chron.com/local/investigati...rt-1-24419.php