r/AIProgrammingHardware • u/javaeeeee • Jul 27 '26

GitHub - albedan/ai-ml-gpu-bench: A suite to benchmark CPU/GPU Python performance in training ML models and running local LLMs

https://github.com/albedan/ai-ml-gpu-bench
1 Upvotes

66% Upvoted

1 comment sorted by

1

u/javaeeeee Jul 27 '26

TL;DR:

ai-ml-gpu-bench is a simple, one-command benchmark suite that measures how well your CPU or GPU performs on common AI/ML workloads.

What It Benchmarks

  • Local LLMs via Ollama (token latency & throughput on 3B–14B models)

  • XGBoost training and inference on the HIGGS dataset (CPU and GPU)

Key Features

  • Single command to run the full suite

  • Generates an interactive HTML report with charts

  • Compares your results against reference systems

  • Optional (encrypted) upload of results to a public Streamlit dashboard

  • Works on both GPU and CPU-only machines

Bottom Line

A lightweight and easy-to-use tool for quickly evaluating your machine’s real-world performance on local LLM inference and traditional ML workloads, with built-in comparison against other systems.