Short Notes on Python

From PaskvilWiki
Revision as of 17:18, 18 November 2019 by Admin (Talk | contribs)

Jump to: navigation, search

Timing, and memory, on Linux


On Linux, it's safer to use time.time()

import time
t = time.time()
# do some stuff
print "stuff took %1.3f", time.time() - t, "seconds"

On Windows, AFAIK, it's safer to use time.clock()


For me, the following does a good job getting memory usage (in kB) on Linux:

import resource
print resource.getrusage(resource.RUSAGE_SELF).ru_maxrss

Since resource is standard package, it should work on Windows too, but I don't know if it does, or what units are used if it works.

Importing Files

If you need to import a file '../mylib/', you can use the following snippet:

import sys, os
fld = os.path.realpath(os.path.abspath(os.path.join('..', 'mylib')))
if fld not in sys.path:
    sys.path.insert(0, fld)
import commons

# use your module now...

uWSGI, nginx, Flask

  • install uwsgi (incl. uwsgi python plugin), python flask, and nginx,

Setting Up uWSGI

  • create file that will hold the server logic, for instance:
from flask import Flask

app = Flask(__name__)

def hello():
    return "hello there!"
  • create uwsgi config file, wsgi.ini (minimal version here; read uwsgi docs for head-spinning array of configurables):
module          = main:app
master          = true
processes       = 5
socket          =
protocol        = http
plugin          = python
  • run uwsgi
uwsgi --ini wsgi.ini

Adding nginx Layer

  • remove the "protocol" directive from wsgi.ini, and add "die-on-term":
module          = main:app
master          = true
processes       = 5
socket          =
plugin          = python
die-on-term     = true
  • add a new vhost to nginx - /etc/nginx/sites-available/app.nginx:
server {
    listen 80;
    server_name my.awesome.domain;
    location / {
        include uwsgi_params;
    • communication through socket is also possible (see socket, chmod-socket, vacuum and other directives for uWSGI)
    • of course, create link in /etc/nginx/sites-enabled/, and restart nginx,

Run uWSGI daemon on boot - supervisor

  • install supervisor
apt-get install supervisor
  • add/edit /etc/supervisord.conf with content like this:

loglevel=info ; (others: warn,debug,trace)


[program:your app]
command=/usr/bin/uwsgi --ini wsgi.ini

Run uWSGI daemon on boot - systemd

  • create systemd file for uWSGI, /etc/systemd/system/uwsgi-app.service:
Description=Job that runs the uWSGI app

ExecStart=/usr/bin/uwsgi --ini wsgi.ini


Then you can start and stop the uwsgi service using:

# systemctl start uwsgi-app.service
# systemctl stop uwsgi-app.service

Once you're happy with the settings, enable the daemon to be run on boot:

# systemctl enable uwsgi-app.service


needs a bit of refreshment and updating...

based on.

Decorators are simple and expressive way to modify function without editing the source of the function itself. Or, the other way around, to modify multiple functions in the same way, without code duplication (DRY).

Decorators can be spotted in the code by starting with @ character. Decorator is a function (or class) that can do some additional work before or after the call to the decorated function. It can even call the decorated function multiple times, or not at all.

Decorators can be implemented as closures (my fave), or as classes; the following approaches are equivalent:

# using a function (closure) as a decorator
def beforeAndAfter(f):
    def decorated_fn():
        print("Before", f.__name__)
        print("After", f.__name__)
    return decorated_fn

def func():
    print "func() is in da' house!"

# using a class as a decorator
class beforeAndAfter(object):

    def __init__(self, f):
        self.f = f

    def __call__(self):
        print("Before", self.f.__name__)
        print("After", self.f.__name__)

def func():
    print "func() is in da' house!"


Note that in functional decorator, anything outside the body of decorated_fn() is equivalent to content of the __init__() constructor of the class-based decorator. This code is run during the initialization, only once for each decorated function, regardless of whether the decorated function is ever called in the code - you should avoid any heavy lifting there.

Decorators can also accept arguments, and (obviously should) forward arguments to the decorated function:

def beforeAndAfter(p1, p2):
    def wrap(f):
        def wrapped_f(*args, **kwargs):
            print "Decorator arguments:", p1, p2
            print("Before", f.__name__)
            f(*args, **kwargs)
            print("After", f.__name__)
        return wrapped_f
    return wrap

@beforeAndAfter("hello", "world")
def func(a, b=2):
    print "func() is in da' house,", a, b



I prefer installing virtualenv tool through pip, to make sure those are in sync version-wise.

# create a new venv
# it is better to keep venv's in some separate folder, not to pollute your project folder
$ virtualenv ~/.virtualenvs/my-new-env

# "log into" your venv; success can be seen by your command line being prefixed by venv name
$ source ~/.virtualenvs/my-new-env/bin/activate

# now you can install anything you need, tucked away in your venv:
(my-new-env) $ pip install -r requirements.txt

# when done, just deactivate
(my-new-env) $ deactivate


Make SQLAlchemy Read-Only

All write operations in SQLAlchemy pass through flush() method of your session. Just monkey-path it to do nothing!

engine = create_engine("connection string")
Session = sessionmaker(bind=engine, autoflush=False, autocommit=False)
session = Session()
session.flush = lambda *args,**kwargs: None