There are many things that we could consider under this subject. My motto is to make you aware with some of the important features, methods and pythonic stuff that you could refer from here and then do more Google for advanced learning. I will just give you a start.

NOTE: I consider you to be a Python developer. If you are just starting with Python, then I guess this post will become confusing for you. You could go through for quick reference on what to read next when you are done with your first Python tutorial.

Python is cool. Its trending and its obviously self-intuitive. Learning curve with python is awesome.

First some basic conceptual points before starting up:

1. Python is compiled and then interpreted, which means the source code gets compiled to byte code (Which is python specific representation and not the binary code, making startup speed optimized) and then subjected to Python Virtual Machine (PVM) where one by one instructions are read and executed, i.e. Interpreted.
2. Python is neither call-by-value or call-by-reference. It is call-by-object. As everything in python is object, functions, classes, properties and other methods.
3. Python not exactly utilizes the standard way of function overloading as it has the capability to handle multiple function arguments in a different way (We will understand this later in this post).
4. Python have two different data types distinguished based on whether objects of the type are mutable or immutable. The immutable types cannot be changed after they are created.
   • Immutable Types: int, float, long, complex, str, bytes, tuple,  frozen set
   • Mutable Types: byte array, list, set, dict

Pythonic Stuff:

Partial Function:

We have partial functions from functools library. With partial functions we can derive function with x parameters to a function with fewer parameters and fixed values set for the more limited function. Any callable object can be treated as a function.

from functools import partial
"""
Lets say we have a function for multiplying
two numbers.
"""
def multiply(x,y):
return x*y

# Implementing partial function
pf = partial(multiply, 2)
print pf(4)

"""
The result would be 8
"""

What exactly we are doing here is that we have a function multiply and an object “pf” which is deriving itself from multiply function with predefined value for “y”. So the same multiply function runs with single argument with another one as pre-defined.

 List Comprehensions

This is something that every python programmer should know. Its pythonic way. It can be used to construct lists in a very natural, easy way, like a mathematician is used to do. Its a substitute for the lambda function as well as the functions map(), filter() and reduce(). For most people the syntax of list comprehension is easier to be grasped.

<code>S = {x² : x in {0 ... 9}}</code>
<code>V = (1, 2, 4, 8, ..., 2¹²)</code>
<code>M = {x | x in S and x even}</code>

Yes, the above one is the same thing that we use to do in our schools. And you could do this exactly the mathematician way with python too. Refer below:

>>> S = [x**2 for x in range(10)]
>>> V = [2**i for i in range(13)]
>>> M = [x for x in S if x % 2 == 0]
>>>
>>> print S; print V; print M
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096]
[0, 4, 16, 36, 64]

Lets take another example to understand how we could avoid the usual things that we practice and use list comprehensions to speed up our programming.

For example, let’s say we need to create a list of integers which specify the length of each word in a certain sentence, but only if the word is not the word “the”.

To do it in traditional way we would write something like this:

line = "the indian ocean is an ocean with the word ocean"
words = line.split()
word_lengths = []
for word in words:
if word != "the":
word_lengths.append(len(word))

Lets do the above in pythonic way 😉 list comprehension way:

line = "the quick brown fox jumps over the lazy dog"
words = line.split()
word_lengths = [len(word) for word in words if word != "the"]

Both the code will do exactly the same thing and you could see the difference. Its too powerful when extensively used and really solves many problems very easily.

Generators

Ever thought of a function that could act like an iterator? That’s what generators do. Generators return an iterable set of items, one at a time, in a special way.

Lets see an example:

import random

def GetRand():
# returns 5 numbers between 1 and 100
for i in xrange(5):
yield random.randint(1, 100)

# returns a 6th number between 10 and 25
yield random.randint(10,25)

for random_number in GetRand():
print "Random number is: %d!" % random_number

A generator is simply a function which returns an object on which you can call next, such that for every call it returns some value, until it raises a StopIteration exception, signaling that all values have been generated. Such an object is called an iterator.

So what’s happening above; normally functions return a single value using return, just like in Java, C#, PHP or any other language. In Python, however, there is an alternative, called yeild. Using yeild anywhere in a function makes it a generator. So GetRand() is a function which yields number between “1 to 100” and “10 to 25”. Every call to next yields a single value, until all values have been yielded. for loops call next in the background.

Note: the above code is perfectly acceptable for expository purposes, but remember that in Python 2 firstn() is equivalent to the built-in xrange() function, and in Python 3 range() is a generator. The built-ins will always be much faster.

The performance improvement are awesome. Generators loads lazily (i.e. on demand generation of values), which translates to lower memory usage. Furthermore, we do not need to wait until all the elements have been generated before we start to use them. This is similar to the benefits provided by iterators, but the generator makes building iterators easy.

Decorators

To be very simplified, decorators are used for anything that you want to transparently “wrap” with additional functionality to a function. If you have ever come across to Django Framework it also utilizes this functionality as “login required” functionality on view functions.

Also if you are familiar with language like C# then you would probably have seen this mechanism to provide interoperability with C, COM, and web services.

Similarly if you are coming from JAVA backgrund, you would have practiced Aspect-Oriented Programming (AOP).

Lets quickly see an example on how exactly we could advantage with generators.

# Simple function
def func(arg1, arg2, ...):
pass
func = dec2(dec1(func))

# Lets say if we have to do this with decorators
@dec2
@dec1
def func(arg1, arg2, ...):
pass

Don’t get confused with what’s happening and how this will work. Its very simple. In mathematics, composition of functions (g o f)(x) translates to g(f(x)). In Python, @g @f def foo() translates to foo=g(f(foo) and from here only this concept has came in.

Note: Decorators do not accept arbitrary expressions.

Now, let’s have a comprehensive example for this:

def Again(old_function):
def new_function(*args, **kwds):
# We are doing old_function twice here
old_function(*args, **kwds)
old_function(*args, **kwds)
return new_function

# Now we want to repeat multiply function twice
>>> @Again
def multiply(num1, num2):
print num1*num2

>>> Multiply(2, 3)
6
6

What just happening above is once the method “multiply” is being called, it is passed to “Again” decorator to execute and the “Again” decorator runs it twice.

We could use Decorators in number of tasks like checking if user is logged in or not, to check if the argument passed in the function are in correct type or not, to manipulate existing function with some custom defined values and so on. It becomes very handy.

 Meta Classes

As the name says, Meta classes are nothing but the class of a class. Like a class defines how an instance of the class behaves, a Meta class defines how a class behaves. A class is an instance of a Meta class.

Lets take a small example:

class CustomMetaclass(type):
def __init__(cls, name, bases, dct):
print "Creating class %s using CustomMetaclass" % name
super(CustomMetaclass, cls).__init__(name, bases, dct)

class BaseClass(object):
__metaclass__ = CustomMetaclass

class Subclass1(BaseClass):
pass

"""
So the OUTPUT will be:
Creating class BaseClass using CustomMetaclass
Creating class Subclass1 using CustomMetaclass
"""

By creating a custom Meta class in this way, it is possible to change how the class is constructed. This allows you to add or remove attributes and methods, register creation of classes and subclasses creation and various other manipulations when the class is created.

I remember a question asked from one of my friend that if we want to interchange the functionality of form method GET and POST, how we could do that. So here it is, you could achieve it via Meta Classes. Try it of your own, I am not going to give whole lot code here 😉

Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don’t (the people who actually need them know with certainty that they need them, and don’t need an explanation about why).
– Tim Peters

Oh yes, I forgot to mention the beautiful Objective-C developers have definitely come across metaclasses, its the same thing like runtime has functions objc_allocateClassPair() and objc_registerClassPair() to create and register class-metaclass pairs. (We have lots of Objective-C developers having hands on to Python these days 🙂 )

Code Introspection

As what it says, it keeps an eye on your python code and reveals useful information about your program’s objects. Python is great with introspection.

Basically with code introspection, you could examine something to determine what it is, what it knows, and what it is capable of doing.

The most common method of introspection in Python is using the dir function to detail the attributes of an object. For example:

class foo(object):
def __init__(self, val):
self.x = val
def bar(self):
return self.x

>>> dir(foo(5))
['__class__', '__delattr__', '__dict__', '__doc__', '__getattribute__', '__hash__', '__init__', '__module__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__', '__weakref__', 'bar', 'x']

What actually happened here is the dir command gave all methods that were involved in running the above piece of code. And that is what it does.

There are several functions and utilities for code introspection that you can use:

help()
dir()
hasattr()
id()
type()
repr()
callable()
issubclass()
isinstance()
__doc__
__name__

NOTE: Introspection should not be confused with reflection, which goes a step further and is the ability for a program to manipulate the values, meta-data, properties and/or functions of an object at runtime.

So finally Introspection is nothing but a code looking at other modules and functions in memory as objects and gives information about them.

Reflection

Reflection is the ability a program to examine (type introspection) and modify the structure and behavior (specifically the values, meta-data, properties and functions) of the program at runtime.

Lets understand few elements of Reflection:

• Type: The type method enables to find out about the type of an object. The following tests return True:

type(3) is int
type('Hello') is str
type([1, 2]) is list
type([1, [2, 'Hello']]) is list
type({'city': 'Paris'}) is dict

• Duck typing: It provides an indirect means of reflection. It is a technique consisting in using an object as if it was of the requested type, while catching exceptions resulting from the object not supporting some of the features of the class or type.
• Getattr: Returns the value of an attribute of an object, given the attribute name passed as a string.

Also we have Isinstance, Callable and Dir. Google it and you will get to know each of them in more detail.

Now, lets just do a small example to sum up.

# I want to be able to call different methods on a Python class with dynamic function name, e.g.

class Obj(object):
def A(self, x):
print "A %s" % x

def B(self, x):
print "B %s" % x

o = Obj()

# normal route
o.A(1) # A 1
o.B(1) # B 1

# dynamically
foo(o, "A", 1) # A 1; equiv. to o.A(1)
foo(o, "B", 1) # B 1

"""
OK Lets make it work in Other way.
For this we could use getattr
"""
getattr(o, "A")(1)
"""
is equivalent to
"""
o.A(1)

Multiple Function Arguments

So you want to do function overloading? Same function name different ways to handle the different number of arguments, right? Okay lets straight way move to point; Use kwargs. You can use kwargs to let your functions take an arbitrary number of keyword arguments.

So lets dive into very direct code example. A nice example is also intuitive to refer from Python Docs:

"""
The *args will give you all function
parameters as a list:
"""
def foo(*args):
for a in args:
print a

>>>foo(1)
1

>>>foo(1,2,3)
1
2
3

"""
The **kwargs will give you all keyword arguments
except for those corresponding to a formal parameter
as a dictionary.
"""
def bar(**kwargs):
for a in kwargs:
print a, kwargs[a]

>>> bar(name="one", age=27)
age 27
name one

"""
Both idioms can be mixed with normal arguments to
allow a set of fixed and some variable arguments:
"""
def foo(kind, *args, **kwargs):
pass

It’s also worth noting that you can use * and ** when calling functions as well. This is a shortcut that allows you to pass multiple arguments to a function directly using either a list/tuple or a dictionary.

For example, if you have the following function:

def foo(x,y,z):
print "x=" + str(x)
print "y=" + str(y)
print "z=" + str(z)

# You can do things like:
>>> mylist = [1,2,3]
>>> foo(*mylist)
x=1
y=2
z=3

>>> mydict = {'x':4,'y':5,'z':6}
>>> foo(**mydict)
x=4
y=5
z=6

And probably this is the case why they acted bit smart and said no overloading 😛

So Importantly, we have many other stuffs to cover but it would become never ending and I am planning to add a part two for this post maybe within next 1 month. I will discuss about some more Python stuffs like Serialization, Built in exceptions, Unittest, Zip, Filter, cProfile, Overriding, and many other things.

Just make sure to consider this blog post as a todo list to excel your knowledge on the subject and you have to go onto more Google to grab more explanation, practices and experiences.

Let me know if anything specific you guys want me to cover. Connect with me via twitter or contact me from here. Cheers!