To make my point a bit clearer, if you haven't already done so, download the Unity3d free version. Install the package. Then put aside about 4 hours to watch and walk through their "Roll a ball" tutorial videos:
When you're done, you should have something that looks a bit like:
You are probably thinking "Sure, but that's not really a game." And indeed I suspect that even as a free game you'd get very little attention. But what may surprise you is how little more it would take to get you to the first level of a game. Starting with this, it took me about two hours to get to the following:
And there was virtually NO programming required to get from the top picture to the bottom. Simply adding some components and free assets, and then playing with placement. You now have something that would be a good first level for a "real" game.
If you look through the Unity Asset store, you will even see starter projects that will let you just tweak and extend engines to easily create car games, adventure games, and more. For a weekend and a couple hundred dollars of assets, or a week or two if you want to do it all yourself, you could have your first simple game ready to go.
It's not likely going to let you create a title that will compete with the AAA releases out there, but it will be enough to help you see how to build something that might.
So if you've had a game idea and thought, "I'd like to create a game, but it would be too hard because I'm not good at X" - well, now you know that shouldn't stop you. Start digging, and let's see what you can come up with!