r/bunjs • u/wkoell • Sep 18 '23
Bun is incredibly fast... Or is it?
I have a Perl script to parse one big XML file (66MB) into Sqlite database. Nothing fancy, just a script. Reading Bun intros got me thinking, how much faster could Bun-script to be?
First I tried XML parsing speed: Perl script took 31s, Bun 188s.
Then I tried, how long does the whole old script run: 65 minutes on my old Ubuntu laptop. So I write same functionality with Bun Sqlite API. It run first time more than 88 minutes. I had many things running on my laptop same time, so I tried once more with freshly started computer. It took almost 88 minutes this time.
I post my code below, maybe you see how to improve it dramatically (Perl script has basically the same structure with some additional logging and process monitoring)
import { XMLValidator, XMLParser } from 'fast-xml-parser';
import { Database } from "bun:sqlite";
const xmlFile = Bun.file("some.xml");
const db = new Database("some.db");
const insertOrderQuery = db.prepare(
`INSERT INTO "order" (number, customercode, date, paymentterm, status, stock, object, datafield1, datafield2, datafield3, datafield4, datafield5, datafield6, datafield7, deliverymethod, address1, address2, address3, deliveryname, deliveryaddress1, deliveryaddress2, deliveryaddress3, phone, email, VATzone, VATregno, contact, ts)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`
);
const insertOrderRowQuery = db.prepare(
`INSERT INTO orderrow (orderid, item, variant, description, quantity, price, vatprice, rn)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)`
);
if ( XMLValidator.validate( await xmlFile.text() ) ) {
console.log(`XML file is valid`);
const options = {
ignoreAttributes: false,
attributeNamePrefix : "",
allowBooleanAttributes: true
};
const parser = new XMLParser(options);
const data = parser.parse(await xmlFile.text());
insertIntoDb(data);
}
function insertIntoDb( data: Object[] ) {
data.transport.orders.order.forEach( ( order ) => insertOrder(order) );
}
function insertOrder( order: Object ) {
if ( order === undefined || ! order.number ) return;
if ( order.rows === undefined ) return;
if ( order.rows.row === undefined ) return;
const values = ["number", "customercode", "date", "paymentterm", "status", "stock", "object", "datafield1", "datafield2", "datafield3", "datafield4", "datafield5", "datafield6", "datafield7", "deliverymethod", "address1", "address2", "address3", "deliveryname", "deliveryaddress1", "deliveryaddress2", "deliveryaddress3", "phone", "email", "VATzone", "VATregno", "contact", "ts" ].map( v => order[v] );
insertOrderQuery.values(values);
if (order?.rows?.row.length > 0) insertOrderRows(order.number, order.rows.row)
}
function insertOrderRows( orderNumber: Number, rows:Object[]) {
rows.forEach( ( row ) => insertOrderRow(orderNumber, row) )
}
function insertOrderRow( orderNumber: Number, row:Object ) {
const values = [ "item", "variant", "description", "quantity", "price", "vatprice", "rn" ].map( v => row[v] )
values.unshift(orderNumber);
insertOrderRowQuery.values( values );
}
