How to indent Python list-comprehensions?

List comprehensions can be useful in certain situations, but they can also be rather horrible to read.. As a slightly exaggerated example, how would you indent the following?

allUuids = [x.id for x in self.db.query(schema.allPostsUuid).execute(timeout = 20) if x.type == "post" and x.deleted is not False]

From stackoverflow

• How about:

  allUuids = [x.id for x in self.db.query(schema.allPostsUuid).execute(timeout = 20) 
                     if (x.type == "post" and x.deleted is not False)]
  

  Generally, long lines can be avoided by pre-computing subexpressions into variables, which might add a minuscule performance cost:

  query_ids = self.db.query(schema.allPostsUuid).execute(timeout = 20)
  allUuids = [x.id for x in query_ids
                     if (x.type == "post" and x.deleted is not False)]
  

  By the way, isn't 'is not False' kind-of superfluous ? Are you worried about differentiating between None and False ? Because otherwise, it suffices to leave the condition as only: if (x.type == "post" and x.deleted)

  Piotr Lesnicki : I second that. Generally lines of more than 80 characters should be avoided in all languages.

  Peter Rowell : Agreed, but Python's syntax rules often encourage/force longer lines. Its whitespace handling in general is about my only complaint about the language.

  Piotr Lesnicki : With carefully chosen variables, and spliting lines with parenthesis or brackets, i never had to resort to too long lines (the problem is more often to avoid self.long_foo.very_long_bar.baz(....) using temporaries)

  Ali A : About the "By the way", it is not totally superfluous, take the case x.deleted = None, for example.

• For me that's too much. Maybe it's just a terrible example, since "type" and "deleted" would clearly be part of the db query.

  I tend to think that if a list comprehension spans multiple lines it probably shouldn't be a list comprehension. Having said that, I usually just split the thing at "if" like other people have and will answer here.

• It depends on how long they are. I tend to structure them like so:

  [x.id for x
   in self.db.query(schema.allPostsUuid).execute(timeout=20)
   if x.type == 'post' 
      and x.deleted is not False
      and ...
      and ...]
  

  That way every expression has its own line.

  If any line becomes too big I like to extract it out in a lambda or expression:

  transform = lambda x: x.id
  results = self.db.query(schema.allPostsUuid).execute(timeout=20)
  condition = lambda x: x.deleted is not False and ... and ...
  [transform(x) for x in results if condition(x)]
  

  And then if a lambda becomes too long it gets promoted to a function.

  Claudiu : these are NOT equivalent - the list comprehension is many times faster, as it doesn't have to do any function lookups. the appropriate translation would use for loops.

  Geo : Except for the performance hit , this is a very readable example !

  orestis : How would for loops prevent the function lookup? Also note that the loop in list comprehensions is implemented in C, and is therefore faster than the plain one.

  cdleary : Just a note -- you can use multiple if statements instead of using "and" multiple times. Kind of a funny little syntax truth.

  orestis : cdleary: Thanks! Never knew that. Seems a bit dirty, though :)

• Where I work, our coding guidelines would have us do something like this:

  all_posts_uuid_query = self.db.query(schema.allPostsUuid)
  all_posts_uuid_list = all_posts_uuid_query.execute(timeout=20)
  all_uuid_list = [
      x.id 
      for x in all_posts_uuid_list 
      if (
          x.type == "post" 
          and 
          not x.deleted  # <-- if you don't care about NULLs / None
      )
  ]
  

  Ali A : That's quite pretty.

  orip : That's how I do it

• allUuids = [x.id 
              for x in self.db.query(schema.allPostsUuid).execute(timeout = 20) 
              if x.type == "post" and x.deleted is not False]
  

• You should not use a list comprehension for that.

  List comprehensions are an awesome feature, but they are meant to be shortcuts, not regular code.

  For such a long snippet, you should use ordinary blocs :

  allUuids = []
  for x in self.db.query(schema.allPostsUuid).execute(timeout = 20) :
      if x.type == "post" and x.deleted is not False :
          allUuids.append(x.id)
  

  Exactly the same behavior, much more readable. Guido would be proud of you :-)

  Claudiu : this is a teeny bit slower, as you have to build the array step by step

  e-satis : I think the list comprehension does exactly the same under the hood. It's not because it's a one line expression that this is a one time operation...

  orip : e-statis: the function is the same, but list comprehensions can be significantly faster

  e-satis : Hum, I don't have time to benchmark it so I'll have to take your word :-) But anyway, readability matters more that speed in Python...

  e-satis : In the Python culture, you usually choose readability over speed. Python isn't a fast language anyway...