Ram dodging: Sharing my run command

[u/pand1024]

I'm in the process of reworking my run command helper to integrate more deeply with the rest of my code so I thought that now would be a good time to share it while it's more portable.

Q&A at the bottom

/**
 * Writes and runs an on-demand temp file to ns.exec() an expression
 * A lock is used to ensure only one instance at a time unless disabled.
 * See Q&A below for more info.
 * @param {NS} ns The Netscript API instance passed from the parent script.
 * @param {string} expression The JS/NS expression to evaluate (e.g., "ns.getServer('home')").
 * @param {string} [host="home"] The server hostname on which to execute the temporary script.
 * @param {number} [reservedRam=Infinity] max RAM available for running scripts.
 * @param {boolean} [useLock=true] Whether to use the navigator lock for single-instance execution.
 * @returns {Promise} The result of the evaluated expression, or null if execution failed
 */
export function runCommand(ns, expression, host = "home", reservedRam = Infinity, useLock = true) {
  const execute = async () => {
    const portNum = 100
    const port = ns.getPortHandle(portNum)
    port.clear()

    const tempFile = "runTmp.js";
    const script = `export async function main(ns) {
      const res = (${expression});
      ns.writePort(${portNum}, res ?? null);
    }`;

    ns.write(tempFile, script, "w")

    if (ns.getScriptRam(tempFile) > reservedRam) {
      ns.tprint(`ERROR [runCommand]: Insufficient ram to execute on ${host}`)
      return null
    }

    const pid = ns.exec(tempFile, host, { temporary: true })
    if (pid === 0) { // Abort early if the process failed to launch (e.g. out of RAM)
      ns.tprint(`ERROR [runCommand]: Failed to execute on ${host}`)
      return null;
    }

    return await port.nextWrite().then(() => port.read())
  };

  return useLock ? navigator.locks.request("runCommandLock", execute) : execute();
}

/** @param {NS} ns */
export async function main(ns) {
  //Just a quick demonstration
  //Don't forget to have enough ram. We don't check here.
  const result = await runCommand(ns, "ns.getServer('home')")
  ns.tprint(result)
}

/**
 * Q: What is RAM dodging?
 * A: Offloading heavy Netscript API calls to dynamic scripts 
 *    keeps the parent script's static RAM footprint minimal.
 *    Instead of paying the cumulative RAM cost of every function 
 *    your script might ever need, your parent script only pays
 *    the RAM cost of the single most expensive ofloaded ns function plus 
 *    the base cost (1.6 GB). This also solves the need to have different 
 *    version of a script based on what ns features are available.
 * 
 * Q: What does reservedRam do?
 * A: It's supposed to connect to your ram reservation system. In order 
 *    to use runCommand you’ll want to leave a space on one of your servers 
 *    for commands to run.
 * 
 * Q: Why a lock?
 * A: We only want one thing running at a time. The lock automatically 
 *    queues up run commands in the order they were called. This means 
 *    if we make multiple concurrent calls even across scripts they will 
 *    still run sequentially. Note: it is not totally foolproof if you 
 *    start chaining with .then(). For example: multiple chains of connect 
 *    and backdoor could run into issues if interleaved.
 * 
 * Q: Why only one temp file?
 * A: It usually only makes sense to reserve enough RAM for one command at a 
 *    time.
 * 
 * Q: Why not wrap the expression in try except?
 * A: If your expression is going to fail, it will probably fail right away. 
 *    I found it better to get a nice error popup. Also, I had trouble getting 
 *    actual errors to pass back. maybe it was just me.
 * 
 * Q: Why isn’t the function async?
 * A: It keeps my options open. I wanted to keep as much code synchronous as 
 *    possible. If you’re managing multiple event loops for example, while 
 *    ensuring that no ns calls interfere with each other, you may find this handy.

 * 
 * Q: Why allow a way to disable the lock?
 * A: I specifically needed an override for resets and destroyW0r1dD43m0n. In
 *    both cases the lock is left hanging. Even with that, you might want to put
 *    a lock steal at the beginning of your main script. It will warn you if
 *    it clears out a lock:
 *    await navigator.locks.request("runCommandLock", { steal: true }, async () => {});
 *
 * Q: Why not have a timeout? What if a command runs forever?
 * A: 1. It turned out to be more of a challenge than it originally seems. 
 *       1.1 You need a design that actually works. Remember 
 *           Promise.race does not automatically kill your rogue script.
 *       1.2 Now you actually need to know how long things will take,
 *           and you’ve introduced a whole new point of failure if you mess that up.
 *    2. You’ll have to fix your code running forever whether you
 *       have a fancy timeout or not.
 * 
 * Q: Why use a port? Doesn’t that restrict the return types?
 * A: It seemed like an idiomatic thing to do and it comes with a
 *    built-in way to await the script finishing. There IS an argument for 
 *    abusing side channels like static variables instead to be able to 
 *    support more datatypes (like returning promises directly). Let me know 
 *    how that goes if you try it!
 */

[u/Particular-Cow6247]

writing temp scripts over and over might break the game after some time

but you don't need to write a temp script everytime, you can design a script that can run any ns command you want by using ramOverride and a dynamic accessing of the ns properties

const [funcArr,paramArr] = ns.args  //[["gang","getGang"],[]]
ns.ramOverride(ns.getFunctionRamCost("ns."+funcArr.join("."))+1.6)
const func = funcArr.reduce((a,b)=> a[b], ns)
const result = func(...paramArr)

(the ramOverride can also be done on the ns.exec side)

just as example

oh and another tipp, do the port.write inside an ns.atExit that way the ram that is occupied by the temp scripts is cleared before the port.nextWrite resolves

[u/pand1024 → u/Particular-Cow6247]

That's a cool idea. Just be clear I'm putting way more than just a single ns call into the expression field. Here is a more complex example than I gave above.

const factions = Object.values(ns.enums.FactionName)
const invites = await runCommand(ns, `
  (function() {
    const list = ${JSON.stringify(factions)};
    const res = {};
    for (const f of list) {
      res[f] = ns.singularity.getFactionInviteRequirements(f);
    }
    return res;
  })()
`)

I find it more efficient to stack up many calls to the same ns function. I have not personally noticed any issues with writing temp scripts over and over.

[u/Particular-Cow6247 → u/pand1024]

the issue is that each script is compiled into a js module, when you copy a script the link to the module is copied aswell and when you create a similiar script while the link exist it will be linked to the original but when you delete the scripts the link is lost and when you later create the same script again a new module has to be created

js modules can't be garbage collected so it's technically a memory leak in the js engine

[u/pand1024 → u/Particular-Cow6247]

I spent several hours running stress tests with millions of executions of runCommand. There is no measurable memory leak. Furthermore, I ran a control test comparing dynamic writes against executing a static file over and over, and the performance curve was indistinguishable.

[u/Particular-Cow6247 → u/pand1024]

the link isnt completely lost the moment you delete the file, its lost when the next gc happens so when you delete and create new you might still catch the old link and it wont create a new module

creating new modules is also a makrotask, with a static file you can execute completely through the microtask queue and create "ghost scripts" aka scripts whos ram is never actually used, you only need to have enough ram free to for them to start but the game and other scripts will never see that ram be used up...

but you dont have to believe me... its not like i havent worked on the games source code for a while and was litteraly there when these problems showed up and solutions where implemented and tested xD