Shock cord, like a rubber band, but way more durable (about 15¢/ft). Get it at Rei, Neptune, or any reputable outdoor store.
Tie the shock cord into the lanyard hole at the bottom of the knifes handle. Tie a loop into the other end. Make the length of the cord just right so that the loop fits snugly over the top of the sheath after wrapping the cord around the knife 1/2 to 1 turn. Now your knife dosen't fall out of the sheath. Slipping the shock cord loop off the top of the knife is fast and easy.
I can post a picture of how this works if my description dosen't make sense. I am over 6000 miles, 2 life jackets, and 8 seasons using this setup and still on my first knife - it works.
I also use a thin piece of cord to tie the sheath to my PFD (backing up the attachment point, which has failed on my jacket). This has saved my knife many times.