How much planning a project requires is largely dependent on the size/complexity of the project and the number of people working on the project. The more complicated the project, the more planning things out will make things go smoothly. It certainly is possible to complete a complex project without any planning, but the time required to do so will be significantly higher than the time that would be saved by proper planning. Planning doesn't need to take the form of formal documents, though if you are working with large teams a formal design specification document is probably a good idea.
I tend to design by working out what screens a game needs then breaking down each of the screens. After breaking down my NES RPG game I quickly realized that I could turn this project into a number of mini-games. These pieces could then be assembled together to form the core of the final game.
The first game is totally unrelated to the RPG but will have common code that will be used by other projects. It will be a simple quiz game. This will let me get my multiple-choice prompts working, number printing, random numbers, and some of the math routines. The 6502 does not have multiplication or division (other than bit shifting) so this has to be done in software.
Next will be a outdoor scrolling maze game. This obviously will be used for the game map screen. This will rapidly followed by a treasure hunt game using a scrolling covered map that only reveals itself as the player explores the map.
The final pair of pre-game mini-games will be the combat engine. This will be developed as a series of arena games. The first arena will simply be the combat with pre-determined characters and monsters. The second game will add the character stat screens and a store where players can buy equipment for upcoming battles.
At this point all the major parts of the RPG exist so they can be combined into a proper RPG. This will be broken into separate chapters so each chapter can be released separately yet be combined into a single larger final game.