According to the makers of Handbrake, you can get comparable DVD video quality by using Constant Quality with a setting of RF:20. Note that you should not use RF:0. This will result in huge files with no visible quality improvement. You can also choose a fixed file size or average bit rate. I have found RF:20 creates files of 700mb to 1.5 gb depending on the source video. Older movies with lots of video noise will result in larger files. I have also read that using average bit rate of over 1,000 is good enough for most video content. You can encode a small video clip to see what setting you like the most for your particular output device. Better quality usually means longer encode time and larger files but there is a point where you will not be able to see the difference depending on your player.
I decided to increase three fields in the advanced to the default settings. I can’t tell you what they do but it seemed like a default is better than less than default.