I've never, ever had a problem finding a place to go that isn't a "mob scene," you just have to do a little research. A few years ago, for example, I went with some friends to Diablo Mexican restaurant at Monte Carlo. The first floor had food and two bars with a DJ, while the second floor (which has a covered patio that overlooks the strip) had a live band with a large, centrally located bar. We paid $50 and it included food AND They had a set amount of tickets so they didn't get overcrowded. $175 bought you all you can drink tickets.
Several years ago I went to a party at Sushi Roku restaurant. For $75 you took part in a champagne toast at midnight and could celebrate on a huge dance floor with a great DJ and lots of dancing. The crowd was completely controlled (limited admission) and the restaurant offered a perfect view of the strip.
If you are willing to spend more money, all of the clubs in Vegas offer tickets for admission. Almost every party is hosted by a celebrity or pseudo celebrity (last year's hosts included Avril, Paris, and JayZ). Whether you are looking to mingle with the stars or just find a cool, less expensive bar or restaurant scene, check out http://www.vegas.com/ before you book. The website provides a list of club, bar and restaurant NYE offerings around the end of October, beginning of November.
If you absolutely want to check out Vegas but you just can't stand the thought of a huge crowd, check out one of the off-strip hotels that offers a view of the strip. Some suggestions-- JW Marriott, Red Rock, and Green Valley Ranch. All three will likely host parties at their hotels that are more low key than the strip events, yet you will still have an opportunity to view the strip fireworks from these resorts. I went with friends to GVR for NYE about 8 years ago and it was great! We gathered out by the pool for a full 360 view of the fireworks on the strip and downtown. Then we went back inside to a very inexpensive dance party to complete the night.
One last option for the cheapest way to do Vegas is to check out either Fremont Street or the far south end of the strip, like the M, Silverton, or South Point. Silverton and South Point both have great parties for about $50-80 admission. M has an amazing view of the strip from just about every spot in the resort. The Freemont Street party is still pretty popular and can be a bit crazy, but Silverton and South Point are both mellow. For South Point, I'm 99% sure the cost of admission includes drinks! It's a great way to do Vegas on the cheap.
Finally, if you are looking for a room and don't want to pay a pretty penny, keep an eye on the Vegas hotel deals thread on http://www.fatwallet.com/. Otherwise, check out priceline and hotwire. You can score a great deal around late November at the earliest, with even better deals offered about 2 weeks before NYE. I've seen people score rooms on the strip for as little as $80 at 4* hotels just by using priceline in the weeks before NYE. If you need help identifying one of the hotels on which you are bidding, check out http://www.betterbidding.com/.
And now for my biggest secret that I'm only willing to share now that we've decided to start leaving Vegas for NYE--timeshare deals. If you know you want to come to Las Vegas for NYE and you are willing to be patient, you can really score some sweet deals on timeshares via ebay. For the past 7 years this is the route we've gone. Individual owners and brokers start selling their NYE weeks for great prices about 2 weeks before the holiday weekend. For the past two years we've stayed at Polo Towers on the strip in a two bedroom timeshare (sleeps 6) for under $500. The timeshares are usually sold as a full week (7 nights), with a standard minimum of 3 nights. You can't get a single bed hotel room for $500 for 3 nights! So definitely check out ebay at the very last minute for some sweet deals on timeshares.
You can't just show up in Vegas for the busiest weekend of the year and expect everything to fall into place. Do a little research and a litle planning using my suggestions and I really think you can have a wonderful time at NYE.