The AI will expand. A human can get the drop on them a bit, though. Their ability to combine the necessary tasks to expand in certain cases is less than the human. A person can examine the map and, if "stuck" for the moment because a narrow strip of ocean is blocking further expansion, the human can target that crossing with a city placed there. Early ships can no longer enter Ocean plots. Period. So there are no more "Suicide Galleys(TM)" where your ship may or may not sink, and you send it out across the ocean in hopes of getting lucky.
However, if your cultural border expands out in to the ocean, you can cross safely. Any ocean plot inside your border is rendered "safe" and your ships may pass. You can also pass through the ocean plots controlled by any civ with whom you have Open Borders. Sometimes in order to keep expanding, you have to plant a city near one of these narrow crossings, where the ocean is one plot wide and there is another island over there, then produce culture in your city and expand its borders far enough to let you cross. Your borders can only expand one plot out in to the ocean, though, so anywhere that the ocean is two or more plots wide, you have to have ocean-worthy vessels to make it across. Or... if there is a civ on the other side, their borders can project out one plot, and yours can project out one plot, so if there is a friendly civ nearby, you may be able to use their help to cross ocean of two plots wide.
Three or more plots wide is an insurmountable obstacle until you obtain ocean-worthy vessels. If you happen to get stuck in a spot where there is no more room to expand, you may need to prioritize researching Astronomy! On the whole, though, this is not something to worry about. The fractals scatter the islands very evenly. I have played many games on the new Archipelago and conducted hundreds and hundreds of tests, checking the land forms. Some instances will be more kind than others, but they all offer you a chance to compete. For one thing, seafood resources (Fish, Clams, Crabs... and to a lesser extent, Whales) are in great abundance on these maps. You can use the Slavery civic to rush city projects, and you only need food to support your slaves. (To grow more population, to replace the slaves who died while working on whatever you selected.) Food can also support City Specialists.
So even if you have, for instance, a tiny island with two Seafood, your city, one hills plot and one grassland, that city can still be productive for you. The seafood will be worth five to six food per plot. The hills, when mined, would give you a few shields. The grassland could be turned in to a Workshop, for more shields, or a Cottage, for extra commerce. You still have enough extra food from the seafood sources to feed a slavery-driven economy, or to feed City Specialists. (If you choose Caste System instead of Slavery, for Labor civic, you can run as many Merchant or Scientist specialists as you can feed!) A few cities of this sort are possible in every case I've observed, and that is enough to research your way to ocean-worthy vessels if you need to do that. And that is the worst case scenario! Most maps are much kinder than this, letting you reach many islands, sometimes dozens, with Coastal-only ships.
Another favorite of mine is Highlands. Highlands is probably the most challenging single player map we have. While the AI is at its weakest on heavy water maps, it is at its best on large, sprawling land maps. The AI tends to favor the cottage and windmill tile improvements. These are the commerce-heavy improvements and they to be more urgent on maps that lack oceans. Oceanic maps boost commerce in several ways:
1) Coastal plots offer two commerce apiece. Ocean plots give one commerce each.
2) Harbors are only buildable along a salt water sea or ocean, and when built these boost Trade Route revenues. They also boost health from seafood resources.
3) The extra food from seafood resources can let you run cottages instead of farms, or feed more Specialists.
Oceanless maps lack all of that. You have to build more farms to feed your miners. Or... you have to forego the farms and mines and instead build cottages and windmills. Fewer shields, more commerce. Farms don't start to produce extra food until the late game, and by then you may be in a deep hole! Highlands has lots of hills and peaks, of course, and it has a region of cooler climate which can be particularly challenging if you start there. Also, because the map is larger than most land-heavy maps (to compensate for plots lost to the Peaks), the food resources that do exist tend to scatter more. So it is really tough to get a high food city going. You have to choose between farms and cottages, and that's tougher than it appears!
In any situation where food is in shorter supply, the AI's handicap bonuses on higher difficulty settings will have more impact. They can afford fewer farms simply because they grow faster. (This is not true on Noble setting or lower, though).
I lost my last test game on Highlands... at Monarch difficulty! I'll just let that speak for itself. 
BOB "SIRIAN" THOMAS | "Four Times the Charm"