When the world generates, you'll see various texts appearing while waiting. These texts aren't random, the world is actually processing what the text says.
I've seen a video that showcases how the world generation looks like behind the scenes. So that gives you a rough idea how it's creation works: (https://www.youtube.com/watch?v=bF-_AeRRbmU)
It starts out with the surface, and the variation of the dirt and stone, then it start making caves, increasing in size each time. After that you'll also see sections been converted to biomes (jungle and ice specifically), and then more detailed structures, like granite, cabins and the dungeon happens afterwards.
The world is technically generated in one go, but it keeps generating new parts over itself, and finetuning the details afterwards.
As for rendering, while the world is loaded all at once, it will only draw (or render) the blocks when you're close to it. so when you're moving fast, you sometimes see the tiles ahead of you needing some time to appear.