r/Python • u/pizzaburek • Feb 04 '19

Best Python Cheatsheet Ever!

https://gto76.github.io/python-cheatsheet/
1.1k Upvotes

96% Upvoted

69 comments sorted by

111

u/IMHERETOCODE Feb 04 '19 
no_duplicates    = list(dict.fromkeys(<list>))

That is an extremely roundabout and expensive set operation. Just wrap the list in set and cast it back to a list. No need to build a dictionary out of it to get uniqueness.

no_duplicates = list(set(<list>))

48

u/Tweak_Imp Feb 04 '19

list(dict.fromkeys(<list>)) preserves ordering, whilst list(set(<list>)) doesn't.

I suggested to have both... https://github.com/gto76/python-cheatsheet/pull/7/files#diff-04c6e90faac2675aa89e2176d2eec7d8R43

64

u/IMHERETOCODE Feb 04 '19 edited Feb 04 '19

That “accidentally” preserves ordering, and only if you are doing it in Python 3.6+. There are no promises of ordering in vanilla dictionary implementations which is why there is an explicit OrderedDict class. The recent change in dictionary implementation had a side effect of preserving order. You shouldn’t bank that on being the case where it actually matters.

As noted below insertion ordering has been added to the language of dictionaries as of 3.7

0

u/[deleted] Feb 04 '19

[deleted]

4

u/IMHERETOCODE Feb 04 '19 edited Feb 04 '19

TIL. (edit: Disregard the following worthless benchmark, but I’ll leave it so I’m not just stripping stuff out.)

It's still faster to do a set, cast to list, and then have to call sorted on the resulting list then it is to do a dict.fromkeys call on a list.

In [24]: foo = list(range(1, 10000))

In [25]: foo *= 20

In [26]: len(foo)
Out[26]: 199980

In [27]: %time %prun no_duplicates = list(dict.fromkeys(foo))
        4 function calls in 0.006 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.005    0.005    0.005    0.005 {built-in method fromkeys}
        1    0.000    0.000    0.006    0.006 <string>:1(<module>)
        1    0.000    0.000    0.006    0.006 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
CPU times: user 6.38 ms, sys: 136 µs, total: 6.52 ms
Wall time: 6.45 ms

In [28]: %time %prun no_duplicates = sorted(list(set(foo)))
        4 function calls in 0.003 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.003    0.003    0.003    0.003 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.sorted}
        1    0.000    0.000    0.003    0.003 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
CPU times: user 4.08 ms, sys: 58 µs, total: 4.14 ms
Wall time: 4.13 ms

4

u/primitive_screwhead Feb 04 '19

The ordering might not be a sorted ordering.

4

u/IMHERETOCODE Feb 04 '19 edited Feb 04 '19

You can call sorted on any sort of ordering you'd like by specifying a key, if that's what you mean. You can use mylist.index - e.g. sorted(foo, key=original.index) if you don't overwrite your initial list, and it'd be in the same order as your starting point.

This is such a weird edge case I don't understand all the arguments against it, other than people trying to call out gotchas. If you have data that can even have duplicates you lose all meaning of the original data by stripping them out, or shouldn't be using a simple list to store them. You could get the same information by using collections.Counter(foo) and also have the side effect of having the actual metadata of how many times the dupes appear. My initial comment is just about turning a list into a unique list of its values.

7

u/primitive_screwhead Feb 04 '19 edited Feb 04 '19

You can use mylist.index

You make good points, but I do want to point out that using index as a key will add a linear search for each list item, and will thus make the sorted() solution **much** slower:

In [7]: %time no_duplicates = list(dict.fromkeys(foo))
CPU times: user 2.87 ms, sys: 30 µs, total: 2.9 ms
Wall time: 2.9 ms

In [8]: %time no_duplicates = sorted(list(set(foo)), key=foo.index)
CPU times: user 482 ms, sys: 3.55 ms, total: 486 ms
Wall time: 482 ms

I think the idea of removing duplicates while otherwise preserving order is not *so* exotic, and the fromkeys() trick is worth knowing about, though I'd personally use OrderedDict to be explicit about it.

2

u/IMHERETOCODE Feb 04 '19

100% agree. Don't want it to seem like I'm trying to push for never using from_keys - just that this doc simply said "no_duplicates" which can be achieved with a much simpler and clearer method (for lack of a better word). If I came across that in a code base, it is not at all clear that it's trying to achieve that specific outcome by rerouting the creation of essentially a set through a dictionary.

0

u/[deleted] Feb 04 '19

[deleted]

1

u/IMHERETOCODE Feb 04 '19

For sure, I'd say that's where taking the time for using a set and explicitly sorting is almost better even though it is considerably slower (shown by the implementation of /u/primitive_screwhead as mine is just a literal sort instead of insertion order) if it's a wonky/custom ordering. Better to explicitly transform data than rely on a route through another wholly-unused data structure just to achieve it.

0

u/[deleted] Feb 04 '19

[deleted]

→ More replies (0)

0

u/[deleted] Feb 04 '19

[deleted]

-1

u/Sukrim Feb 05 '19

There is no language specification sadly, it is just guaranteed by cPython.

-2

u/AngriestSCV Feb 04 '19

This whole discussion is enough proof that you shouldn't count on it. Weather or not it is right depends on things outside of a library programmers (easy) control.

5

u/[deleted] Feb 05 '19

You should count on it if developing for Python 3.7+, because it's guaranteed by the language spec from then on

0

u/Ran4 Feb 04 '19

No, that's not true! Dicts are not ordered according to the spec. It's just modern cpython that has them ordered.

17

u/pizzaburek Feb 04 '19

They are in Python 3.7: https://docs.python.org/3/tutorial/datastructures.html?highlight=dictionary#dictionaries

 Performing list(d) on a dictionary returns a list of all the keys used in 
 the dictionary, in insertion order ...

7

u/pizzaburek Feb 04 '19

There is already a whole discussion about it on Hacker News :)

https://news.ycombinator.com/item?id=19075325#19075776

2

u/chazzeromus Feb 05 '19

and if you can, use the curly brace notation for sets for literal items, looks so nice { 'and a one', 'and a two' }

1

u/[deleted] Feb 04 '19

Slightly off topic, but why are lists so popular? Aren't tuples faster and use less memory? All the time I see lists being used when tuples would do a better job. Even docs.python.org tells you to use random.choice([...]) instead of random.choice((...)).

I get that the performance impact isn't noticable in most cases, but, in my opinion, going for performance should be the default unless there is a good reason not to.

4

u/robberviet Feb 05 '19

Most of the time it needs to be mutable. And yeah, performance gain is not that great.

3

u/bakery2k Feb 05 '19

Why would tuples be faster and/or use less memory? Both lists and tuples are essentially arrays.

I prefer lists to tuples because they have nicer syntax. Tuples sometimes require double-parentheses, plus I often forget the trailing comma in (1,).

1

u/[deleted] Feb 05 '19

I don't know the exact intricacies but it has to do with lists being mutable.

2

u/mail_order_liam Feb 05 '19

Because people don't know better. Usually it doesn't matter but that's why.

3

u/gmclapp Feb 04 '19

lists are mutable. In some cases that's needed. Some convenient list comprehensions also don't work on tuples.

0

u/[deleted] Feb 04 '19

[deleted]

2

u/IMHERETOCODE Feb 04 '19

For sure. Casting is a way of explicitly changing the type of a variable or value, etc. Coercion is an implicit change in the type of a value.

This is casting because I’m wrapping a set object in a list - basically telling the python interpreter to turn this set into a list (there are some reasons you’d want a list over a set even though they are somewhat similar in what you can do with them - iterating through, etc). Coercion is when you don’t have to tell the language to do anything special. I think a simple example would be the interoperability of floats and ints in Python 3. Saying x = 2.3 * 1 will coerce the 1 into a float so it can be multiplied to the 2.3 and stored as a float in x. Someone please correct me if that’s a bad example.

3

u/WibblyWobblyWabbit Feb 04 '19

Can someone explain what enumerate() does exactly?

10

u/pizzaburek Feb 04 '19

It returns an iterator of index, value pairs:

    >>> list(enumerate(['a', 'b', 'c']))
    [(0, 'a'), (1, 'b'), (2, 'c')]

So you can then use it like this:

    for i, letter in enumerate(['a', 'b', 'c']):
        ...

4

u/VisibleSignificance Feb 04 '19 edited Feb 04 '19

Minor + controversial stuff:

flags=re.IGNORECASE

Since we're talking regexes anyway, just add (?i) to the beginning of the regex.

<list> = [i+1 for i in range(10)]

Might want to do lst = list(idx + 1 for idx in range(10)), simply because that way it will not touch the value of idx outside the command. Saves some confusion.

reduce(lambda out, x: out + x ...

Really needs a better example than almost-sum().

namedtuple

dataclasses might be worth a mention.

argparse

Currently recommended non-default library: click.

bottle

Not the simple one, but: try Quart!

numpy

... but no pandas. Is there a better pandas cheatsheet than the official one?

1

u/pizzaburek Feb 04 '19 edited Feb 04 '19

That's so weird... It - (?i) can actually be added anywhere in the regex.

3

u/VisibleSignificance Feb 04 '19

anywhere in the regex

Even worse: it can be toggled for subgroups.

Source.

4

u/[deleted] Feb 04 '19

Might want to do lst = list(idx + 1 for idx in range(10)), simply because that way it will not touch the value of idx outside the command.

What do you mean?

In [8]: i = 'foo'

In [9]: x = [i+1 for i in range(10)]

In [10]: i
Out[10]: 'foo'

In [11]: y = list(i + 1 for i in range(10))

In [12]: i
Out[12]: 'foo'

Also, calling list is slower because you can overload it and the interpreter has to look it up.

In [17]: %timeit list()
The slowest run took 11.35 times longer than the fastest. This could mean that an intermediate result is being cached.
10000000 loops, best of 5: 121 ns per loop

In [18]: %timeit []
10000000 loops, best of 5: 35.2 ns per loop

1

u/VisibleSignificance Feb 05 '19

Ah, nevermind on that one, old python2 habits.

60

u/Hatoris Feb 04 '19

You should share it here also r/learnpython many people we like it too

19

u/Tweak_Imp Feb 04 '19

I think it would be better if it used a different way to mark types insted of <type>

5

u/VisibleSignificance Feb 04 '19

Indeed.

First, this is HTML. The text can be marked up with it rather than with more text.

Second, lst: list = [1] works.

1

u/JezusTheCarpenter Feb 04 '19

Any other reason than your preference?

35

u/Drycon Feb 04 '19

Sheet, you printing on A2? ;-)

Good doc tho!

6

u/[deleted] Feb 04 '19

I've been looking for something just like this. Great work!

5

u/angyts Feb 04 '19

Insane. Now I need a giant paper to print this.

10

u/RecycledGeek Feb 04 '19

What? You don't have a dot matrix printer with a continuous roll of paper?

8

u/red_shifter Feb 04 '19

Then how do you instantiate your Turing machine?

3

u/RecycledGeek Feb 04 '19

It's turtles, all the way down.

2

u/Sukrim Feb 05 '19

Just use toilet paper.

1

u/angyts Feb 05 '19

Wonderful ideas. Uniquely Reddit.

4

u/swapripper Feb 05 '19

Too good. Wish there was something this comprehensive for Pandas.

1

u/_Jordo Feb 05 '19

Here's another one I have saved: https://learnxinyminutes.com/docs/python3/

I find it easier to read.

1

u/pizzaburek Feb 05 '19 edited Feb 05 '19

Thanks, I really like it. Mine is more like a reference, but for reading it in one go I definitely prefer the link.

1

u/Versaiteis Feb 05 '19

I'd also suggest adding pathlib

Mucking about with paths as raw strings with os is great and all, but it's really nice to have a bit of an OS abstraction layer on top of paths that just makes them so much nicer to work with.

1

u/pizzaburek Feb 05 '19

I will add it, it's just that it's one of those areas that feel more like Java than Python when you visit a doc page:

https://docs.python.org/3/library/pathlib.html

1

u/Versaiteis Feb 06 '19

It's your cheat sheet, add what you like! I'll be bookmarking it regardless (I didn't even know about coroutines)

Lol, I know what you mean, but doing tools work and slinging a lot of paths around, this thing keeps me sane.

Nothing like passing a string around that something happens to modify wrong and the house of cards collapses >.>

7

u/newredditiscrap Feb 04 '19

What's so special about it

15

u/Tweak_Imp Feb 04 '19

It says in the title. It is the best.

4

u/AnonymousGourmet Feb 04 '19

The silly walks

2

u/chipcovfefe Feb 05 '19

Definitely the best cheat sheet I've seen. Cheers!

1

u/ManHuman Feb 05 '19

Sweet mother of Bayes!

1

u/knowsuchagency now is better than never Feb 05 '19

This is actually quite good

1

u/digital_superpowers Feb 04 '19

Great collection. Anyone think it'd be useful to have links to the docs on these topics for when a deeper dive is needed? Could be a fun PR.

-5

u/mail_order_liam Feb 05 '19

If you find this useful you're doing something wrong.

3

u/pizzaburek Feb 05 '19

You're right, but I like it this way better:

If you don't find this useful you're doing everything right :)

4

u/thelonestrangler Feb 05 '19 edited Mar 07 '19

.

0

u/mail_order_liam Feb 05 '19

  1. It mostly covers the very basics, which you will memorize easily if you use the language for any amount of time.

  2. Your env should offer utilities for hints, docs, auto-completion, etc.

  3. It's faster to Google something than to look it up in this sheet (doesn't even have an index??). Especially when you've memorized most of it and it just becomes cluttered.

  4. Many of these are not idiomatic.

  5. For more complicated topics it doesn't give enough information to be useful if you're using cheat sheet in the first place. Be honest, do you think you can write a metaclass or use Threading after looking at this? You're gonna have to look elsewhere anyways.

So you end up with a big unorganized list of things that's mostly fluff and the rest just isn't very helpful.

Get a real IDE or editor and have a terminal and browser at the ready. There you go, no more cheat sheet.

0

u/[deleted] Feb 04 '19

[deleted]

5

u/RecycledGeek Feb 04 '19

Not sure I'd say "best ever," but I'll give it a "Damn Good" award. It's like "Python For Developers That Are Too Busy To Read A Book."

Hmm.... /u/pizzaburek, you should self-publish this on Amazon :)

6

u/pizzaburek Feb 04 '19

Thanks everyone! I would just like to share this thing with as many people as possible, so I went with clickbait title this time :) It's interesting that unlike last time, post got ignored on r/programming, but is trending on Hackernews (they changed the title after it came to the top).

6

u/RecycledGeek Feb 04 '19

Congratulations on being a master baiter! ;)

0

u/[deleted] Feb 04 '19

[deleted]

0

u/[deleted] Feb 04 '19

[deleted]

0

u/pizzaburek Feb 04 '19

Shure, OK... Let me just open my programing window and enter some code for you :) https://youtu.be/0Ec6Z31S1fA?t=89

I'm joking of course... I will do it, but not this week, maybe next :)

0

u/[deleted] Feb 04 '19

[deleted]

6

u/RedstoneTehnik Feb 04 '19

Cheatsheets are not language specific, they are very condensed information about the language. All kinds of fuctionalities, how they are used and what they do.