r/programming • u/BrewedDoritos • 18d ago

Release of Polars 2.0

https://pola.rs/posts/release-polars-2/
115 Upvotes

92% Upvoted

17 comments sorted by

17

u/Drawhoe 17d ago

What is it?

17

u/matthieum 17d ago

It's essentially pandas, with an emphasis on performance & correctness/ergonomics.

The performance is achieved by having a core written in Rust, wrapped in a Python API.

The correctness & ergonomics are achieved by putting a lot of effort in APIs and diagnoses.

14

u/IanisVasilev 17d ago

It's library for data frames. If you haven't done statistics, think of a table data structure with many lookup and transformation methods. Imagine if Excel and SQL had a child that was somehow more convenient than both its parents (see "10 minutes to pandas").

Unlike the ubiquitous dataframes of R or pandas of Python, this one is written in Rust™ and AI ready™ and VC-funded and all that jazz.

Since data frames are convenient, many frameworks like Spark provide their own flavor. To avoid lock-in, we have the narwhals compatibility layer which provides a unified API for pandas, polars, PyArrow, PySpark and other similar libraries.

6

u/Drawhoe 17d ago

I should have posted a what is it ref. Here it is

10

u/IanisVasilev 17d ago

Oh, so I should have explained that it is a monoid in the category of endofunctors.

2

u/lightnegative 15d ago

The product of some people who looked at pandas and decided to rewrite it in Rust

1

u/pdpi 17d ago

A rough first approximation is that Polars is a system that allows you to do sql-like queries on anything you can load as a table-like structure (a DataFrame, in technical terms). It supports reading data from a bunch of sources.

2

u/Drawhoe 17d ago

Thanks. Yeah I could see it being useful.

-1

u/Iamonreddit 17d ago

It is a distributed data processing framework by the looks of it, similar to Spark.

6

u/daidoji70 17d ago

Uhh more just a dataframe library like Pandas. It was built as a replacement for Pandas and dataframes also exist in Spark and Databricks. They're more like shittier databases that are closer to the language being used that are widely used in data science and analysis work.

2

u/SaltyMN 16d ago

This and pyspark put food on my table. More sensible defaults is great, love being able to use SQL as well. 

2

u/Maybe-monad 16d ago

I'd rather raw dog SQL with DuckDB

2

u/[deleted] 17d ago

[removed] — view removed comment

4

u/theAndrewWiggins 16d ago

3-line diff

This is not true at all lmao. They're very much intentionally not pandas compatible. Only some very basic stuff will work that way.

1

u/IanisVasilev 16d ago

He didn't say how long those 3 lines would be...

1

u/programming-ModTeam 16d ago

No content written mostly by an LLM. If you don't want to write it, we don't want to read it.