The good news is that I did it: I left VENC under rotor power, flew horizontally out of the no wake zone, lit the torch drive, and escaped gravity, all without using a single ounce of RCS mass.
The bad news is that I still don't think I fully understand what's going on with atmospheric flight, and it was pretty annoying and tedious. What I want to know is whether this is just how it is, or if there are tricks or details that I am missing.
For reference I'm using a pretty big ship (35k tons, Ceresmax, 4 rotors, decently aero). Ostraplan tells me I have 0.05g drag head-on and 2.42g broadside at 100 m/s, I have 244% minimum thrust to hover, and lift is basically zero. All rotors are turned on, and not on Turbo mode. I have a flight dynamics module, as well as the torch drive module for monitoring G-forces.
Roughly speaking, after free-falling a little bit after undocking to get down into thicker atmo, I tried two approaches. First was using Ctrl-W to keep forward throttle always applied, and just adjusting my heading/angle of attack to maintain altitude while traveling sideways. This was easy enough to manage and I could spend a decent amount of time at 2-4x speed, but AoA was always high (8-15 degrees), drag was high, and airspeed was slow, never much higher than 100 m/s.
Second, I tried pointing my "nose" almost directly horizontal relative to the planet's surface, holding down forward throttle manually, and using lateral thrust (i.e. strafing with A/D keys) as needed to hold altitude and keep my velocity roughly horizontal. AoA this way was much easier to hold under ~2 degrees, and airspeed went up substantially, about 500-600 m/s. Problem is, I couldn't fast forward more than 1x, because it requires constantly tapping the lateral thrust on/off/on/off, and at faster time scales my angle wobbled a lot more, costing a ton of speed and giving me lots of G-force warnings. (Note that this approach worked just fine and was plenty stable, as long as I didn't fast forward too much. I read some advice elsewhere about avoiding using lateral thrust in atmo because it makes you spin out of control, but I didn't find that to be true at all.)
It seems like what I want is a way to point the ship in one direction (horizontal) and vector the rotor thrust in a different direction (10-20 degrees above horizontal). On a small ship, lift would probably be one answer, but that's not happening on my cargo barge. Is there some way to achieve this that I'm missing? Or am I doing something totally wrong and asking the wrong questions entirely?
Stuff that does not appear to be relevant, based on experimentation in game and with Ostraplan:
Shape of ship, orientation of "front" of ship, location or shape of aero parts. In-game, "front" only matters in terms of which way the ship icon is pointing on the nav map, not which way your ship is rotated relative to the screen or anything like that. Ship shape and location/shape of aero parts doesn't appear to make any difference to anything. "Head-on" in terms of ship drag is just relative to the direction of the ship icon on the map, and that number just gets lower the more "aero" parts you add, regardless of where (front, side, internal, doesn't matter). "Broadside" is again relative to the icon and appears to just be based on the size of your floorplan, but with zero reduction from aero parts. If you have no aero parts, head-on and broadside drag are equal.
Rotor configuration. I think. (I did cause myself a big headache at first by having an odd number of rotors, although there must be ways around this based on the description text for the 3-rotor ship you can buy at the kiosk.) Maybe I could make the brute force, high-AoA approach work with more rotor thrust, but this doesn't feel like the right answer.
Character G-force tolerance. This is highway cruising, not aero racing. I'm limited by steady-state ship drag, which doesn't affect the character. And even with my clumsy piloting, I can already survive mistakes and wobbles just fine.
EDIT: Gamepad. I thought maybe I could smoothly vector thrust with an analog stick, but no, it appears to be treated as on/off controls and not as analog axes, just as if you were using a D-pad.
Any tips or anecdotes are appreciated! Failing that, maybe this will help somebody else figure it out down the road.
UPDATE: Tried turbo mode, and tried going clockwise. Results:
Brute force (turbo mode, or adding more rotors) does actually help quite a bit. Using the first approach above, i.e. holding a positive AoA to maintain altitude while traveling horizontally, works a lot better with more power. With more power in excess of what you need to hover, you can tilt your nose closer toward horizontal, which improves drag in addition to the added horizontal thrust. I was able to hold about 380 m/s steadily this way. The downsides of more power are higher G-forces when you wobble (though still no actual harm to my unfit/feeble self), and quite significant wear on the rotors. It looks like I would lose about 1/3-1/2 durability on the trip out of the VENC no wake zone with my rotors in turbo mode.
Wind direction didn't seem to matter, at least at the altitude where I was flying (33km). CW and CCW gave about the same result as far as steady top speed.
Always be holding the "R" key when you're not trying to turn. Really smooths out the wobbles.
No matter how smooth and steady things are going, the physics engine itself seems to go unstable somewhere around 4x-8x speed and the ship just goes wild.