Using nested generators

Starting with Python 3.3, you can use delegated generators

def foo(x):
    yield from range(x, 0, -1)
    yield from range(x)

list(foo(5))

Viewing the source for classes/modules

Works great with syntax highlighting.

import inspect

inspect.getsource(azureml.automl.runtime.featurization.data_transformer)

Checking if an object has a certain property/method

if hasattr(myobject, 'myprop'):
    dostuff()

if hasattr(myobject, 'myfunction') and callable(myobject.myfunction):
    domorestuff()

– via StackOverflow

Connecting to MS SQL Server with SQLAlchemy

Using pymssql

pip install pymssql
pip install sqlalchemy

# Macs need FreeTDS, too
brew install unixodbc freetds

connection_str = 'mssql+pymssql://{user}:{pass}@{host}/{db}'

– via PyODBC docs and SQLAlchemy docs. Microsoft Docs is a good resource, too

Using (shudder) pyodbc

pip install pymssql
pip install sqlalchemy

# Macs need FreeTDS, too
brew install unixodbc freetds

But wait, there’s more! Run odbcinst -j to get the location of the odbcinst.ini (probably in the directory /usr/local/etc/). Edit odbcinst.ini to include the following:

[FreeTDS]
Description=FreeTDS Driver for Linux & MSSQL
Driver=/usr/local/lib/libtdsodbc.so
Setup=/usr/local/lib/libtdsodbc.so
UsageCount=1

# Option A - needs the bit above added in odbcinst.ini
connection_str = 'Driver={FreeTDS};Server={server};Database={database};Uid={user};Pwd={pass};Port={most_likely_1433}'

# Option B - doesn't need **any** to edit any files, but you end up inserting paths in connection strings
connection_str = 'Driver=/usr/local/lib/libtdsodbc.so;Server={server};Database={database};Uid={user};Pwd={pass};Port={most_likely_1433}'

pyodbc.connect(connection_str)

You can spend time doing much, much more configuration work as seen here, but this is the least amount of configuration I was able to get away with.

Note that if you neglect to include Port, you might encounter the dreaded '[08S01] [FreeTDS][SQL Server]Unable to connect: Adaptive Server is unavailable or does not exist (20009) (SQLDriverConnect)' error. You have been warned 🤷🏻‍♂️.

Adding time periods to dates

from datetime import datetime, timedelta

date = datetime(2021, 3, 5) + timedelta(days=6)

– via Stack Overflow

Adding other time periods such as years to dates

Use dateutil.

from datetime import datetime
from dateutil.relativedelta import relativedelta

date = datetime(2021, 3, 5) + relativedelta(years=6)

– via Stack Overflow

Format exception as string, including its stack trace

It’s all done using the traceback module.

import traceback

try:
    raise Exception
except Exception as e:
    print(str(e))                        # prints just the exception's message
    print(traceback.format_exc())   # prints the entire stack trace, too

– via Stack Overflow

How to reload a module used in a Jupyter Notebook (or elsewhere)

import data_source          # the module you're trying to reload

import importlib            # only works in Python 3.4+
importlib.reload(data_source)

– via Stack Overflow and GeeksForGeeks

Format numbers in f-strings

m=9
d=1

f'm={m:02d}/d={d:.2f}'
> 'm=09/d=1.00'

Format numbers to fixed decimals with Mako

#instead of
${your_number_var}

#use
${'{0:.2f}'.format(your_number_var)}

– via Stack Overflow

Open a local html file in the browser (useful for testing Mako templates 🙂)

import os
import webbrowser

webbrowser.open('file://' + os.path.realpath('out.html'))

– via Stack Overflow

Timezone conversion using pandas and pytz

tz_localize is ❤️

import pytz
import pandas as pd

date = '2021-01-04' 
pd.to_datetime(date)                                                                        # Timestamp('2021-01-04 00:00:00')
pd.to_datetime(date, utc=True)                                                              # Timestamp('2021-01-04 00:00:00+0000', tz='UTC')
pd.to_datetime(date).tz_localize(pytz.timezone('Europe/Bucharest'))                         # Timestamp('2021-01-04 00:00:00+0200', tz='Europe/Bucharest')
pd.to_datetime(date).tz_localize(pytz.timezone('Europe/Bucharest')).astimezone(pytz.utc)    # Timestamp('2021-01-03 22:00:00+0000', tz='UTC')

– some info via Stack Overflow

*ImportError: Start directory is not importable: * when running unit tests from PyCharm

I’ve had this happen when running python -m unittest discover -s ./tests -t <path_to_your_project_dir> from PyCharm/console, while at the same time python -m unittest discover -s ./tests succeeded.

The solution was to create an empty __init__.py file in the ./tests directory.

– via Stack Overflow, and PyCharm Docs

Case insensitive string comparison

Use casefold().

print('hello'.casefold() == 'world'.casefold())

print('i'.casefold() in 'team'.casefold())

Sorting a list of dictionaries by a value in the dictionary

newlist = sorted(list_to_be_sorted, key=lambda d: d['name'])

– via Stack Overflow

Uploading a file to Sharepoint

% pip install office365-rest-client

from office365.runtime.auth.authentication_context import AuthenticationContext
from office365.sharepoint.client_context import ClientContext

baseurl = 'https://your_company.sharepoint.com'
basesite = '/path/to/site' # every share point has a home.
siteurl = baseurl + basesite 

localpath = ./file.txt
remotepath = Shared Documents/file.txt # existing folder path under sharepoint site.

ctx_auth = AuthenticationContext(url)
ctx_auth.acquire_token_for_user(username, password)
ctx = ClientContext(siteurl, ctx_auth) # make sure you auth to the siteurl.

with open(localpath, 'rb') as content_file:
    file_content = content_file.read()

dir, name = os.path.split(remotepath)
file = ctx.web.get_folder_by_server_relative_url(dir).upload_file(name, file_content).execute_query()

– via this very useful Stack Overflow answer

Matplotlib - render text with transparent background

t = plt.text(0.5, 0.5, 'text', transform=ax.transAxes, fontsize=30)
t.set_bbox(dict(facecolor='red', alpha=0.5, edgecolor='red'))

For datetime indexes, just use a datetime instead of the first 0.5.

– via Stack Overflow

Concatenate two iterables

Use the “iterable unpacking” operator *, works with Python >= 3.5:

a = [1, 2, 3, 4]
b = range(5, 9)

c = [*a, *b]

– via Stack Overflow

Find all modules loaded in an application

import pkgutil

print(list(pkgutil.iter_modules()))

def find_module_package(module_name):
    for _, name, _ in pkgutil.iter_modules():
        if name == module_name:
            return pkgutil.get_loader(name).fullname
    return None

gunicorn.errors.HaltServer: <HaltServer ‘Worker failed to boot.’

Add --preload to the command, to get access to the error log

gunicorn -b :5000 --reload wsgi:APP --preload

– via Stack Overflow

Modify Starlette request headers in middleware

mutable_headers = request.headers.mutablecopy()
mutable_headers["whatevs"] = "whatevs"
request._headers = mutable_headers
request.scope.update(headers=request.headers.raw)

– via Stack Overflow

ModuleNotFoundError when trying to run a Python script directly from outside its path

PYTHONPATH=$(pwd) python app/core/my_file.py