Vinyl***What If***

I would not buy a thing at this point if you are on the fence -- do you have a dealer who would be willing to loan you a better than entry level set up for a week or so? And where are you going to get records to listen to? Buy them and get rid of them if you decide it isn't for you? I assume you have some friends who'd be willing to provide some records for you to play temporarily....
I'm not trying to be negative about this-- I've been playing LPs all my life (well, since I was a child) and am fairly well 'invested' in the medium. Ultimately, that means money in records. Not just gear. Part of this is meant to be fun- looking for better sounding pressings, finding stuff in bins, not just hitting "BUY" on the latest remaster. 
I'd also be concerned that the total cheapy stuff probably isn't going to give you any sense of the potential of the medium. Nor will some records, no matter how how heavy the vinyl weight or fresh off the press remaster. (Nothing against re-do's, some actually do sound better than early pressings). 
In other words, I'm not sure you can get a real sense of what the potential is by dipping your toe in tentatively. So, see if you can get a dealer to loan you something decent and set it up for you and get a few good records to try. In the long run, it may be cheaper to pay for a dealer's time to do this than for you to buy something mediocre, declare every one who is into records delusional and then try to sell what you bought. 

@n80- I’m not advocating a ’spend money’ approach, but reality intrudes. (I think we are now well beyond the OP’s mandate). Older turntables can be fine. But once you get into changing arms, playing with different cartridges (right now, I’m playing with an old Monster Alpha Genesis that was rebuilt- it tracks like a motha, but doesn’t have the midrange of my long in the tooth Airtight Supreme, nor did I expect it to), cleaning records and buying decent playing copies (bin cheapies can often sound better than new re-masters but condition and pressing are outcome determinative), you are unavoidably deep. I’m sure there are people who can buy one of those USB turntables of the type sold at record stores, some less than stellar copies of used records or new reissues and be perfectly happy, but I don’t think that reveals the potential of the medium.
With some exceptions for people that jumped into LP playing in the last 10 years, I think most of us have been doing this a while-- not that this means much in terms of knowledge- but I did the bulk of my record acquisitions before the prices went nuts and clean copies were hard to find among all the dross now being thrown up online; there is as much luck as there is knowledge in finding those gems on vinyl. (I happened on a sealed promo of Nathan Davis’ 6th Sense in the 11th House purely by luck when it popped up as a new listing during a bout of insomnia-- not an easy find). Some of the golden era hard rock, from UK Island, Vertigo, Decca, Philips, etc. are now stratospherically priced; there the reissue makes sense if only to have a copy, assuming the record has been reissued legitimately and many of those are not particularly great sounding compared to the early pressings. The knowledge of records alone keeps people who do this full time constantly busy. (I’m a dilettante, and retired, so I have the time, but it is a constant learning curve, and an enjoyable one for me-- for someone who doesn’t want to bother it is probably an unnecessary headache). It isn’t about the "experience" or the "lifestyle"- it is a combination of learning about music that has been neglected, not reissued, pressing variations, how to shop vinyl (no magic formula I’ve found- it’s time, effort and knowledge which can be acquired) as well as sonic outcomes.
I only started ’doing’ digital recently and have found a lot of good stuff on the silver disc, musically and sonically, so it also isn’t about the vinyl vs. __ format arguments for me either.
If I were doing this from scratch, I’d probably shop a quality used table with appropriate arm. But there’s still the issue of records--which is the biggest cost long term. And doing this in a way that results in a good sonic outcome does require some money, time and effort. Can you skirt one of the three? Sure. But at what cost, given the results? I think that’s up to the individual.
That’s why I wouldn’t advocate diving into this with plug and play expectations on the cheap. Not an elitist attitude, just reality as I know it.
OCD- sure. You’ve mentioned that you’ve tracked cars. Would you take a stock rental car to a track (forget the old Hertz Shelby) compared to what you did to work the suspension and do whatever you felt was necessary to maximize your results, even if done on a cost-effective basis? Just another hobby with deep crevasses and accumulated knowledge. If done for enjoyment, rather than making money, or getting all myopic on the details to the point where you aren’t using the thing as intended, that’s where I part ways with the collectors, the garage queen owners and the snob factor. I actually play those thousand dollar plus records. But, would I go out today and buy them at those prices? No way.

@inna- no doubt on tape. The problem is source material and cost. I'm in it for more music, not necessarily always the 'tippy top' best at this point, so I guess I lose a few rank privileges on my audiophile badge. So be it. 
I'm getting past the point of acquisitiveness, partly because I'm past my big earning years and partly because --while not exactly complacent- I've got other things I want to spend my money on than gear (or crazy priced records). But, alas, that siren call....
My philosophy (not that you asked): if it isn't fun, stop doing it! 
:)

Yeah, @michaelgreenaudio and @cleeds- I haven't found that to be the case either. My system, which is 'mature' in that I have been using roughly the same basic components in the primary audio system for about 12 years, was set up for vinyl only until quite recently, when I added digital and I didn't have to tweak the system to make it sound better- it was just fine on digital and I didn't go all spendy either, compared to the vinyl front end. I think digital is way more cost effective based on my limited experience with it as a serious medium (as opposed to using it as background, in cars, etc.). I didn't expect it to be as good as it is, but it is still very source material dependent in terms of sound quality; true of both media.  

cool, thanks @Frogman. I've got a few options and i'll add that one to the list! 

Hey Schubert is entitled to whatever view he wants to take on the subject of music. I worked with a colleague some years ago who didn’t listen to any music after the beginning of the 19th Century- and we worked together on catalog and masters, etc. for such luminaries as Bob Marley. I did OD on Marley after a while just cause it was 24/7, but talent? The guy was gifted and took that reggae thing (with Chris Blackwell’s help) to a whole other level, making it popular with rock audiences. My colleague- couldn’t stand any of it. We used to tease him about it.
Don’t dig rock- no skin off me. To each his own. I do, however, listen to classical, jazz and all kinds of other stuff, though, truth be told, I like the excitement of a good hard rock band, rock n’ roll, and the passion of well-played, heartfelt blues (most of which is electric, pretty hard to find rural blues played on acoustic instruments these days, but there’s some).
Wasn’t Charley Christian the guy who first amplified his guitar to take it from rhythm to lead in a big band? (I’m sure there were others who went ’electric’ before him). Remember what a controversy it was when Dylan ’plugged in’? Joe Boyd has a good story about that- he was the poor stage hand who literally plugged him in.
Do what you like.
Oh, I forgot the punchline. For my friend’s Birthday, I hired an impersonator to come up to his office as Marley and sing to him. He was really pissed at me.