===== dummy =====
==== useful links ====
* [[https://tutswiki.com/python/exception-handling/ | exception handling in python]] (tutswiki.com)
* very easy to follow; high information density; well thought out examples
* [[https://www.programiz.com/python-programming/online-compiler/ | Online python compiler]] (programiz.com)
* can be useful while doing live coding interviews.
* [[https://nedbatchelder.com/text/names.html | Facts and myths about Python names and values]] (nedbatchelder.com) - well explained.
==== links to documentation ====
* os.path.basename - https://docs.python.org/3/library/os.path.html#os.path.basename
* continue statement - https://docs.python.org/3/reference/simple_stmts.html#the-continue-statement
==== validate inputs ====
Sample snippet:
day = 'foo'
valid_days = ['sun', 'mon', 'tue']
assert day in valid_days, '{} is not a valid day. '\
'It should be one of {}'.format(day, valid_days)
Sample output:
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
in
----> 1 assert day in valid_days, '{} is not a valid day. '\
2 'It should be one of {}'.format(day, valid_days)
AssertionError: foo is not a valid day. It should be one of ['sun', 'mon', 'tue']
==== mod operator ====
In Python, the mod operator returns a number with the same sign as the second argument:
>>> 10%3
1
>>> (-10)%3
2
>>> 10%(-3)
-2
>>> (-10)%(-3)
-1
>>> import sys; print(sys.version)
3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0]
==== where is a module imported from ====
tags | module path
one way:
import importlib.util
importlib.util.find_spec("foo")
Alternative:
import imp
imp.find_module("foo")
Note:
* The imp module is deprecated in favour of importlib. So use the first approach whenever possible.
==== get the function name ====
my_function.__name__
Ref:- https://stackoverflow.com/questions/251464/how-to-get-a-function-name-as-a-string
==== empty file ====
Task: If input file is empty, write an empty output file
import os
if os.stat(input_file).st_size == 0:
logger.info('Input file is empty.')
logger.info('Writing an empty output file.')
with open(output_file, 'w') as fp:
pass
==== config files ====
* https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.defaults - start from here to see the list of functions available
* use extension .cfg for config files.
* based on: https://docs.python.org/3/library/configparser.html -> search for .cfg
* another choice is .ini . But I prefer .cfg over .ini
* more info on INI: https://en.wikipedia.org/wiki/INI_file
==== python mailing list ====
* archives -https://mail.python.org/pipermail/python-list/
* list info - https://mail.python.org/mailman/listinfo/python-list
==== python reference ====
* operator precedence - https://docs.python.org/3/reference/expressions.html#operator-precedence
==== python tutorials ====
* Classes - https://docs.python.org/3/tutorial/classes.html
* https://docs.python.org/3/tutorial/index.html
==== python release cycle ====
* https://python-release-cycle.glitch.me/ - useful to check when a Python version was released and when it would be EOL.
==== isinstance experiments ====
In [1]:
isinstance(True, int)
Out[1]:
True
In [2]:
type(True) is int
Out[2]:
False
Using Python 3.9.5
==== write an extra line to file ====
with open('filename.txt', 'w') as fh:
fh.write('\n')
Do not use os.linesep as a line terminator when writing files opened in text mode (the default); use a single '\n' instead, on all platforms.
Ref:- https://docs.python.org/3/library/os.html#os.linesep
See also:- https://stackoverflow.com/questions/4025760/python-file-write-creating-extra-carriage-return
==== find unused variables ====
pip install pylint
pylint module_name.py
find . -name "*.py" | xargs pylint
==== check if something is float ====
% python
Python 3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> isinstance(12.0, float)
True
>>> isinstance(12.0, int)
False
>>> isinstance(12, float)
False
==== check if something is int ====
% python
Python 3.11.4 (main, Jul 5 2023, 13:45:01) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> isinstance(12, int)
True
>>> isinstance(12, float)
False
>>> isinstance(12.0, int)
False
==== Python comparisons ====
$ python
Python 3.9.17 (main, Jul 5 2023, 20:47:11) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 4 == True
False
>>> 1 == True
True
==== Chained comparisons ====
Q: In Python ''3 < 4 == True'' evaluates to False.
$ python
Python 3.9.17 (main, Jul 5 2023, 20:47:11) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 3 < 4 == True
False
Why?
A: The ''<'' and ''=='' are comparison operators. If a, b, c, …, y, z are expressions and op1, op2, …, opN are comparison operators, then in python, a op1 b op2 c ... y opN z is equivalent to a op1 b and b op2 c and ... y opN z, except that each expression is evaluated at most once.
So ''3 < 4 == True'' is equivalent to ''3 < 4 and 4 == True'' which evaluates to ''True and False'' which turns out to be ''False''.
Ref:-
* https://docs.python.org/3/reference/expressions.html#comparisons - describes the above rule and also gives a list of comparison operators
==== List of comparison operators ====
"<" | ">" | "==" | ">=" | "<=" | "!="
| "is" ["not"] | ["not"] "in"
Ref:- https://docs.python.org/3/reference/expressions.html#comparisons
==== n choose k ====
% python
>>> from math import comb
>>> comb(10, 3)
120
>>> import sys
>>> sys.version
'3.9.16 (main, May 15 2023, 23:46:34) \n[GCC 11.2.0]'
See also:
* https://stackoverflow.com/questions/4941753/is-there-a-math-ncr-function-in-python - contains recipes for older versions of python
===== tasks =====
* [[get urls in a url]]
* [[find depth]]
===== difference between =====
==== os.getenv vs. os.environ.get ====
What is the difference between os.getenv vs os.environ.get?
None. I prefer to use os.getenv as it is less number of characters to type.
To experiment, use [[https://github.com/KamarajuKusumanchi/notebooks/blob/master/python/difference%20between%20os.getenv%20vs.%20os.environ.get.ipynb | difference between os.getenv vs. os.environ.get.ipynb]] (github.com/KamarajuKusumanchi).
Sample usage from https://github.com/dpguthrie/yahooquery/blob/master/yahooquery/base.py#LL926C1-L936C1
def __init__(self, **kwargs):
...
username = os.getenv("YF_USERNAME") or kwargs.get("username")
password = os.getenv("YF_PASSWORD") or kwargs.get("password")
if username and password:
self.login(username, password)
===== Snippets =====
=== import from another project ====
Say we have two projects cloned into two directories - github/project1 and github/project2. To import stuff from project1 into a file in project2
import os
import sys
# Note: os.path.dirname(__file__) can give an empty string,
# use os.path.dirname(os.path.abspath(__file__)) instead.
# Ref:- See Dmitry Trofimov's comment in https://stackoverflow.com/a/918178/6305733
dirname = os.path.dirname(os.path.abspath(__file__))
# Change the relative path as per your needs
newdir = os.path.abspath(os.path.join(dirname, '../project1'))
if newdir not in sys.path:
sys.path.insert(0, newdir)
tags | import from many levels up, import from another directory, relative directory name, path relative to current file
==== check for minimum python version ====
import sys
if sys.version_info[:2] < (3, 7):
raise RuntimeError('Requires Python 3.7 or greater')
==== print exception and continue ====
from traceback import format_exc
try:
# my code
except Exception as exc:
print 'caught exception while doing foo'
print format_exc()
print exc
==== print exception to log file ====
from traceback import format_exc
try:
# my code
except Exception as exc:
logger.error('Caught exception while doing stuff.')
logger.error(format_exc())
logger.error(exc)
When the code is executed in pycharm and the exception is encountered, the traceback is printed in both pycharm window and in the file.
To test the above code, you can raise a simple exception using
raise ValueError("test exception handling.")
==== sleep for 10 seconds ====
tags | sleep for N seconds
from time import sleep
sleep(10)
Ref:- https://realpython.com/python-sleep/ - well written article that talks about how you might use the sleep function in real life. Worth reading end-to-end once.
===== Reading files =====
==== read file into a list ====
Given
$ cat -A great.txt
kamaraju$
kamaraj $
kamara $
kamar $
kama $
kam $
ka $
k $
where the $ denotes the end of line.
$ du -b great.txt
72 great.txt
$ wc great.txt
8 8 72 great.txt
To read it
In [3]:
file = 'great.txt'
with open(file) as fh:
contents = [line.rstrip('\n') for line in fh]
print(contents)
['kamaraju', 'kamaraj ', 'kamara ', 'kamar ', 'kama ', 'kam ', 'ka ', 'k ']
In [4]:
len(contents)
Out[4]:
8
Ref:- https://stackoverflow.com/questions/15233340/getting-rid-of-n-when-using-readlines
You can also do it using readlines.
In [5]:
file = 'great.txt'
with open(file) as fh:
lines = fh.readlines()
lines = [line.rstrip('\n') for line in lines]
print(lines)
['kamaraju', 'kamaraj ', 'kamara ', 'kamar ', 'kama ', 'kam ', 'ka ', 'k ']
tags | file readlines strip newline
Another approach is to use read() with splitlines().
In [6]:
file = 'great.txt'
with open(file) as fh:
lines = fh.read().splitlines()
print(lines)
['kamaraju', 'kamaraj ', 'kamara ', 'kamar ', 'kama ', 'kam ', 'ka ', 'k ']
===== programming notes =====
==== which python executable am I running? ====
import sys
print(sys.executable)
For example
$ python
Python 3.11.5 | packaged by Anaconda, Inc. | (main, Sep 11 2023, 13:26:23) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.executable)
C:\Users\kkusuman\AppData\Local\conda\conda\envs\py311\python.exe
===== programming related =====
==== get the home directory ====
import os
home = os.path.expanduser("~")
Ref:- https://stackoverflow.com/questions/4028904/what-is-a-cross-platform-way-to-get-the-home-directory
==== write file relative to home directory ====
import os
home = os.path.expanduser("~")
path = os.path.join(home, "x", "data.txt")
==== round function ====
The expression ''round(n, r)'' rounds floating-point expression $n$ to the $10^{-r}$ decimal digit; for example, ''round(n, -2)'' rounds floating-point value $n$ to the hundreds place $(10^2)$. Similarly, ''round(n, 3)'' rounds floating-point value n to the thousandths place $(10^{-3})$.
===== format related =====
==== assert variable in a list ====
$ cat junk51.py
letters = ["a", "b", "c"]
x = "1"
assert x in letters, "{} is not a valid letter. It should be one of {}".format(
x, letters
)
$ python junk51.py
Traceback (most recent call last):
File "C:\Users\kkusuman\x\junk51.py", line 3, in
assert x in letters, "{} is not a valid letter. It should be one of {}".format(
AssertionError: 1 is not a valid letter. It should be one of ['a', 'b', 'c']
===== stack overflow links I came across =====
==== memory consumed by a function ====
* https://stackoverflow.com/a/49271549
* refers to | http://web.archive.org/web/20180416235121/http://ihorbobak.com/index.php/2018/02/22/python-process-memory-profiling/
==== call different functions based on variable ====
tags | passing function, function as value in dictionary, dispatch
* https://softwareengineering.stackexchange.com/questions/182093/why-store-a-function-inside-a-python-dictionary/182095
==== __all__ ====
* https://stackoverflow.com/questions/44834/what-does-all-mean-in-python - good explanation by Alec Thomas
* https://docs.python.org/3/tutorial/modules.html#importing-from-a-package - link to documentation; referenced in the above stackoverflow link.
==== dummy ====
* https://superuser.com/questions/1633073/why-are-tar-xz-files-15x-smaller-when-using-pythons-tar-library-compared-to-mac - Why are tar.xz files 15x smaller when using Python's tar library compared to macOS tar?
===== came across =====
==== dummy ====
* os.path.splitext(path) - Get the extension of a path
* https://docs.python.org/3/library/os.path.html#os.path.splitext