Tukukkk

The answer is no; they'll not even be able to withstand normal human acceleration limits. Not because of the pressure (induced by constant acceleration), but because of the momentum change in their environmental medium (caused by the sudden changes in acceleration, especially at the beginning of the launch).

A simple experiment that explains this (don't try this at home edit - for ethics reasons, not because I'm worried about the results); take a small fishbowl, and put a lizard in it, seal the top and shake it. Afterwards, the lizard will be sore and more than a little pissed off at you, but it will be alive. Take a fishbowl full of water and a goldfish, seal the top and shake it. Your fish will be dead, almost instantly.

Further Edit - Another analogy that you might consider here is the shockwaves caused by Blast Fishing, which also causes large changes in momentum over a short time.

Why? Because water is a non-compressible medium (and it's very dense). In any collision, the most deformable object is the one that also absorbs as much of the kinetic energy of the impact as it can. This is why modern cars are so 'flimsy' by comparison to older cars, and consequently so much safer. In an accident, the car breaks so that it absorbs as much of the kinetic energy as it can before transferring the balance to you. Older, more rigid cars don't do that and as a result many people found out what being the most deformable object in a collision actually meant right before they died.

Air is very compressible (read as deformable in this answer) meaning that it can absorb a lot of energy by comparison to its mass. Unfortunately, that doesn't mean much because its density is very low, so the two tend to balance each other out. Still, our lizard only has to worry about being the deformable object every time he hits a wall on the fishbowl. Strap him in, and he's going to fair a lot better (hence seatbelts).

Your fish on the other hand is in a medium that is not only non-compressible, but very dense. This means that it's going to take a LOT more energy to launch all that water because of the mass, and more importantly, the sudden acceleration change is going to throw all that non-compressible mass your way as soon as you start your acceleration, likely crushing you. This also means that any form of maneuvering in space or engine issues that cause significant vibrations or shaking are of concern because of rapid and large momentum changes other than the constant acceleration along your line of axis.

The problem with the reasoning in your question is that buoyancy is not the same thing as pressure. Buoyancy is relative density, whereas pressure is the force of a mass being applied against you (this is a simplification but functionally correct). Sure, the pressure deep in the ocean depths can be extreme, but it can also be introduced to your creatures slowly - they don't instantly go from 10m depth to 1000m depth, and it would kill them to do so. But, in a rocket, that's exactly what you're asking them to endure.

They'd be far better in some form of gel, that allows them to breathe but is lighter weight and can absorb most of the impact for them. Water is not the best medium to find oneself in when face with sudden acceleration.

Assuming that the spaceship is totally filled with water (i.e. no air gaps) then there will be two effects experienced by the aliens.

Firstly, the effect of the acceleration on the water mass in the spaceship. For any significant acceleration there will a gradient in the water pressure along the axis of acceleration. Assume a 200 m long spaceship, at 10 g acceleration. At the 'front' of the ship, the water-pressure will be negligible, at the back end it would be equivalent to that found on Earth at a depth of about 2 km (around 200 Atmospheres). If the spaceship then decelerated at the same rate there would be an almost instantaneous reversal; of the pressure gradient (now negligible at the rear of the craft and 200 Atmospheres at the front). These pressures would scale up in a longer vessel or with higher accelerations - 500 m long and 20 g would give you an almost instantaneous 1000 atmospheres. Equivalent to almost instantaneously diving to the bottom of the Marianas trench! So if the creature relied on swim-bladders for buoyancy moderation they would be in big trouble (deep-sea fish don't to well when they are dredged up to the surface quickly). Internal baffling in the spaceship could solve this problem but then it would be very difficult to open/close doors and move around during acceleration.

Secondly, there would be the effect of acceleration within the body of the creature itself. Land-based creatures often experience significant local acceleration and jarring as a result of jumping, falling etc, so have evolved so internal organs of varying density are constrained in position against these forces. A sea creature, depending on its normal mode of transport within water may not be subjected to these acceleration/jarring effects in its normal life (consider a jelly-fish for example). So if they are used to being cushioned in water their internals may be more sensitive to acceleration than ours.

So the answer would depend on the specifics of the creature. Muscular, deep-diving porpoise - maybe. Floppy, floating jelly-fish - probably not.

Will they merely experience an increase in water-pressure? They are used to dealing with extreme changes in pressure when they dive deep and return to near the surface of their ocean. Therefore surely high acceleration won't inconvenience them.

The biggest problems will arise wherever there are density differences within their body. Think of putting a steel cube inside a mold of jello. As you subject them to higher pressure, nothing bad happens (there are no air pockets to be compressed).

But as you accelerate it, you are changing the forces at the boundary where the density changes. The denser steel cube wants to be at the "bottom" of the pan. As you increase the local gravitational field, the greater the stresses required to maintain out-of-order densities.

Do the creatures have bones? Sensitive, but light organs? Complex organs with multiple tissues of different densities? The greater the acceleration, the greater the forces that appear within them.

In human studies, the greatest damage at the limits reached wasn't with lungs or other aspects of empty spaces, but with the retina. That takes place entirely within an enclosed liquid container, but is still subject to damage at high accelerations.

Yes, they will be able to handle higher accelerations. This has been answered by Steve and AlexP in the comments (possibly others as well)

First, it's useful to turn the question around, why do air breathing creatures suffer from acceleration more than aquatic creatures? Imagine standing under 10g of acceleration in air. Also let's just say your blood is water to simplify things. The pressure increase at your feet:
$$ begin{align} Delta P&= rho g Delta h \
& approx(1000)(10times 10)(2) \
& =200kPa\
& =2bar
end{align}$$

Whereas the external air pressure increase at your feet:
$$ begin{align} Delta P&= rho g Delta h \
& approx(1)(10times 10)(2) \
& =200Pa\
& =2mbar
end{align}$$
There is nearly 2 bar of pressure difference between the blood in your feet and the air outside them.
Lots of blood pools in your feet and legs, your heart won't be able to pump it up to your head, you lose consciousness.

If instead of air, you are surrounded by water, the pressure difference between the blood in your feet and the surrounding medium is zero, no blood pools. You stay conscious.

But there is still nearly 2 bar of pressure between your head and your feet, you might worry that your heart has to work hard to pump against that pressure gradient. It doesn't really, provided everything is incompressible. Let's simplify your circulatory system to be a simple loop. It now looks like this:
$${Huge 0}$$
With your heart as an infinitely thin pump on one side. As your heart pumps water up one side, it is replaced by water moving down the other side. The replacing water arrives at the pump at very very nearly the same pressure as the water moving up - as it's all a sealed closed loop with an incompressible fluid - so it doesn't have to overcome a high pressure difference, as it is effectively being feed water at a high pressure to begin with. This is basically how pressure suits in fighter aircraft work.

Lastly some answers suggest that the pressure changes in a deep column of fluid will kill them, this is true if they are bad engineers. If they build their space ship as a 100m continuous water column, they will have a bad time at high accelerations. If instead they break that 100m into 100 1m high sealed rooms, with no water column to the floors above, then they experience much lower pressure rises. At some point 10cm of water column above them will kill, but it'll take a lot of acceleration.

Provided they use a fluid with similar density to their blood, correctly design their spaceship for high accelerations they will be able to tolerate higher accelerations than land animals. It'll also help if they don't regulate buoyancy with an air bladder like arrangement.

Yes, so long as they can withstand high pressures they will be able to withstand much higher accelerations than a human could (in air.)

Acceleration is equivalent to gravity, and your intuition may work better thinking of it that way (I know mine does.) Increasing gravity on a canister of water is going to increase water pressure linearly. So for example, if you can withstand 10x the pressure at 1g, you can withstand 10gs of acceleration.

Another factor that hasn't been mentioned yet--water creatures are generally going to be a lot weaker than similar land creatures. A water creature doesn't have to support it's own mass, let alone it's own mass taking a fall.

if you want an advanced aquatic like species, just make the homeworld have an insanely dense atmosphere.
the way buoyancy works are (tldr) you need to be less dense than what you are floating in (that's ignoring displacement and stuff)
the density of water is 997kg/m³ while humans are about 985 kg/m³.
now, I know what you are thinking. "if the gas will be as dense as water anyway, why don't I just limit the pressure, and just go aquatic anyway?"

the reason is a running joke in the forums of a game called 'thrive'.

ITS IMPOSSIBLE FOR A OCEANIC SPECIES TO DEVELOP TECHNOLOGY.
combustion is impossible in water. they cant craft metal tools. just.... go to thrive and they will explain it better.