Not a chance. It’s a common misconception that gas giants are made of gas like our atmosphere, since that’s the only gas that the average person interacts with on a regular basis.
In actuality, pressures in the atmospheres of gas giants increase drastically over short distances as you go deeper, so the Hydrogen and Helium that make up the majority of their atmospheres eventually start behaving more like a liquid, or even a plasma.
At the visible top of the clouds in Jupiter’s atmosphere, the pressure is [around 0.1 bar](Chapter 11, Section 2), or about 10% of the pressure of Earth’s atmosphere at sea level. Around 10,000 km deeper than this level (6,000 miles; only about 1/7th of the way to the core), the pressure is already up to insanely high levels ([over 1,000,000 bar] or 1,000 times the pressure in the deepest ocean), and the substances we know as “gasses” start behaving in strange ways.
For instance, [Helium may condense into droplets and fall as rain into the core](Atmosphere of Jupiter – Wikipedia) through an atmosphere of lighter [metallic hydrogen](Metallic hydrogen – Wikipedia). Additionally, the temperature at that depth is already 5000 K (8500F, 4700C), and only gets hotter the deeper you go.
Furthermore, it is theorized that Jupiter has a “solid” core of rock and/or exotic ice. This core would likely be even larger than Earth as a whole.
So even if somehow there was an exotic magical spaceship that could fly (more like swim, since they would be [supercritical fluids](Supercritical fuid – Wikipedia)) through the insanely high temperatures and pressures of most of Jupiter’s atmosphere, there is still a giant [solid-ish core] to deal with.

Jupiter has a core, but I still enjoyed your question, so let’s say that Jupiter didn’t actually have a core, it was just a cloud of gas compressed by its own gravity into a sphere (which is probably what you assumed firsthand).
The rocket wouldn’t bode well. Technically, we don’t have to envision a core in this scenario because it ultimately won’t matter. You see, it’s hard for someone who spends his entire life engulfed in atmosphere to understand what it’s like to try to pierce one.
On earth, Atmospheric pressure is 14.7 pounds per square inch (that’s a lot, the reason you don’t feel it is because everything inside your body is kept under the same pressure). If we don’t return our shuttles at the right angle, they will disintegrate, or they will bounce right off like they hit a wall when bringing them back to earth, and that is with no additional (propulsive) velocity.
On Jupiter, atmospheric pressure is 150 pounds per square inch. That is insane. It’s funny you should mention flying a rocket into Jupiter, because it turns out that NASA essentially did just that – but they knew all about Jupiter’s crazy thick atmosphere.
See, NASA sent Galileo to go check out Jupiter’s moons (and Jupiter, but primarily Jupiter’s moons). Once the mission had concluded, they decided to destroy their spacecraft rather than crash land it on one of Jupiter’s moons (in order to preserve whatever life forms may have existed there, and to avoid contamination). How did they do it? They flew it directly into Jupiter, and it took almost no time at all for it to completely disintegrate.
I think NASA does a lot of cool stuff, and I thought the measures they took to avoid contamination of Jupiter’s moons (it was possible that earth bacteria may have been dormant on the craft) they flew it right into Jupiter.
An artist actually depicted this very event, painting Galileo as it burned up in Jupiter’s dense atmosphere.

Jupiter would most likely swallow everything we sent at it.
This is a somewhat simplified summary of what we currently think Jupiter’s internal structure is.

Metallic hydrogen, as the name suggests, is a solid.
And the core is thought to be most likely be a mix of silicate rock and iron-nickel, not dissimilar to the Earth, but weighing in at 20–40 Earth masses.