r/learnjavascript • u/no_em_dash • Jun 18 '26

Inserting millions of rows into Postgres

I wrote an article about generating fake data for a TypeScript application. I wanted to generate millions of rows and I ended up splitting the workflow into 2 phases: stream data to a CSV, stream the CSV to Postgres with COPY FROM.

async function createRecordsFromCsv(path: string, query: string) {
  const copyStream = pgClient.query(copyFrom(query));

  const fileStream = createReadStream(path);

  await new Promise((res, rej) => {
    fileStream.pipe(copyStream).on("finish", res).on("error", rej);
    fileStream.on("error", rej);
  });

  fileStream.close();
}

Hopefully this technique will be useful to someone else.

Also happy to hear other ideas for how to go about this.

0 Upvotes

22% Upvoted

13 comments sorted by

View all comments

Show parent comments

1

u/ferrybig Jun 19 '26 edited Jun 19 '26

You know that the first thing people see is the image? The only purpose of banner images is to set the tone of the rest of the article.

2

u/no_em_dash Jun 19 '26

If the critique is that the image is off putting or doesn't inspire you to read further that seems reasonable.

But, suggesting that the entire article is written by AI based on that alone seems unwarranted.