Regular Expression Matching Mechanisms

The re module provides two primary methods for pattern matching: match and search. The match function checks for a pattern strictly at the beginning of a string, returning a match object if successful or None otherwise. Conversely, search scans the entire string for a match. For patterns used repeatedly, compiling the pattern with re.compile is recommended to improve performance.

Lambda Functions and Variable Binding

Consider the following code:

def build_operations():
    return [lambda val: val * factor for factor in range(4)]

print([op(10) for op in build_operations()])

The output is [30, 30, 30, 30] rather than [0, 10, 20, 30]. This occurs due to late binding in closures; the lambda functions capture the variable factor by reference. By the time the lambdas are invoked, factor has reached its final value of 3. To achieve early binding and output [0, 10, 20, 30], use a default argument:

def build_operations():
    return [lambda val, f=factor: val * f for factor in range(4)]

Function Overloading in Dynamic Languages

Python does not support traditional function overloading found in statically typed languages like C++ or Java. Since Python uses dynamic typing, function parameters are not bound to specific types, making type-based overloading impossible. Furthermore, Python handles variable arguments (*args) and keyword arguments (**kwargs), as well as default values, allowing a single function to exhibit different behaviors based on input arguments without needing multiple definitions.

Custom Implementation of Max Function

Implementing the built-in max function requires handling both iterables and multiple arguments, along with the key and default parameters:

def custom_max(*args, key_func=None, fallback=None):
    if len(args) == 1:
        iterable = args[0]
        if not iterable:
            if fallback is not None:
                return fallback
            raise ValueError('Custom max() arg is an empty sequence')
        target_items = iterable
    else:
        target_items = args

    peak_item = next(iter(target_items)) if not isinstance(target_items, (list, tuple)) else target_items[0]
    peak_val = key_func(peak_item) if key_func else peak_item

    for item in target_items:
        current_val = key_func(item) if key_func else item
        if current_val > peak_val:
            peak_item, peak_val = item, current_val
    return peak_item

Counting Numeric Occurrences

To count occurrences of numeric types in a sequence, a dictionary can be used:

def count_numeric_frequency(data):
    freq_map = {}
    for element in data:
        if isinstance(element, (int, float)):
            freq_map[element] = freq_map.get(element, 0) + 1
    return freq_map

Alternatively, the Counter class from the collections module provides a concise solution:

from collections import Counter

def count_numeric_frequency(data):
    freq = Counter(data)
    return {k: v for k, v in freq.items() if isinstance(k, (int, float))}

Directory Traversal Techniques

The os.walk function generates file names in a directory tree by walking the tree either top-down or bottom-up:

import os

def traverse_directory(target_path):
    for root, dirs, files in os.walk(target_path):
        for d in dirs:
            print(os.path.join(root, d))
        for f in files:
            print(os.path.join(root, f))

Coin Change Combinations

Given denominations of 2, 3, and 5, calculating the number of ways to make change for 99 can be solved with recursion and memoization:

from functools import lru_cache

@lru_cache(maxsize=None)
def count_combinations(remaining):
    if remaining == 0:
        return 1
    if remaining < 0:
        return 0
    return (count_combinations(remaining - 2) + 
            count_combinations(remaining - 3) + 
            count_combinations(remaining - 5))

Constructing a Spiral Matrix

Generating an n x n matrix filled with numbers in a spiral order involves tracking direction and boundaries:

def generate_spiral(dimension):
    grid = [[0] * dimension for _ in range(dimension)]
    r, c = 0, 0
    val, direction = 1, 0
    
    while val <= dimension ** 2:
        if grid[r][c] == 0:
            grid[r][c] = val
            val += 1
            
        if direction == 0:
            if c < dimension - 1 and grid[r][c+1] == 0: c += 1
            else: direction += 1
        elif direction == 1:
            if r < dimension - 1 and grid[r+1][c] == 0: r += 1
            else: direction += 1
        elif direction == 2:
            if c > 0 and grid[r][c-1] == 0: c -= 1
            else: direction += 1
        else:
            if r > 0 and grid[r-1][c] == 0: r -= 1
            else: direction += 1
                
        direction %= 4
    return grid

Comprehension Expressions

Comprehensions allow concise creation of lists, sets, and dictionaries:

values = [1, 2, 3, 4]
print([v for v in values if v > 2])
print([v for v in values if v % 2 != 0])
print([(char, num) for char, num in zip('abcd', (1, 2, 3, 4, 5))])
print({x: f'item{x ** 2}' for x in (2, 4, 6)})
print(len({ch for ch in 'hello world' if ch not in 'abcdefg'}))

Class Attribute Inheritance

When subclasses inherit attributes without overriding them, they reference the parent class's value. Overriding the attribute in a subclass creates a new local attribute:

class Base:
    x = 1

class DerivedA(Base):
    pass

class DerivedB(Base):
    pass

print(Base.x, DerivedA.x, DerivedB.x)  # 1 1 1
DerivedA.x = 2
print(Base.x, DerivedA.x, DerivedB.x)  # 1 2 1
Base.x = 3
print(Base.x, DerivedA.x, DerivedB.x)  # 3 2 3

Object Creation vs Initialization

The __new__ method is responsible for creating the object instance, allocating memory, and returning the new object. It is a class method. The __init__ method initializes the newly created object's attributes and does not return anything. The two-phase process ensures the object exists before it is configured.

Calculating Day of the Year

Without using standard libraries, the day of the year can be calculated using a lookup table for month lengths:

def is_leap(year):
    return year % 4 == 0 and (year % 100 != 0 or year % 400 == 0)

def day_of_year(y, m, d):
    month_days = [
        [31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31],
        [31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]
    ]
    return sum(month_days[is_leap(y)][ :m-1]) + d

Using the datetime module simplifies this:

import datetime

def day_of_year(y, m, d):
    target = datetime.date(y, m, d)
    start = datetime.date(y, 1, 1)
    return (target - start).days + 1

Static Code Analysis Tools

Static analysis extracts attributes like complexity and adherence to PEP-8 without executing the code. Primary tools include Pylint, which checks for errors, code smells, and formatting issues, and Flake8, which combines Pyflakes, pycodestyle, and McCabe to detect logical errors, style violations, and complexity.

Magic Methods Overview

Python uses magic methods to implement operator overloading and protocols:

• Construction/Destruction: __new__, __init__, __del__
• Arithmetic: __add__, __sub__, __mul__
• Comparison: __eq__, __lt__, __gt__
• Context Managers: __enter__, __exit__
• Iterators: __iter__, __next__
• Callable Objects: __call__

Variable Arguments in Functions

The *args syntax collects arbitrary positional arguments into a tuple, while **kwargs collects arbitrary keyword arguments into a dictionary. Using both allows a function to accept any combination of arguments.

Execution Time Decorators

Decorators measure execution time by wrapping a function. A functional approach uses closures:

import time
from functools import wraps

def measure_duration(output_func):
    def decorator(target_func):
        @wraps(target_func)
        def wrapper(*args, **kwargs):
            start_moment = time.time()
            execution_result = target_func(*args, **kwargs)
            elapsed = time.time() - start_moment
            output_func(target_func.__name__, elapsed)
            return execution_result
        return wrapper
    return decorator

A class-based decorator leverages the __call__ method:

class MeasureDuration:
    def __init__(self, output_func):
        self.output_func = output_func

    def __call__(self, target_func):
        @wraps(target_func)
        def wrapper(*args, **kwargs):
            start_moment = time.time()
            execution_result = target_func(*args, **kwargs)
            elapsed = time.time() - start_moment
            self.output_func(target_func.__name__, elapsed)
            return execution_result
        return wrapper

Understanding Duck Typing

Duck typing determines object suitability based on the presence of methods or properties, rather than the object's type itself. If an object behaves like a file (having read and write methods), it can be treated as a file-like object. This simplifies design patterns in dynamic languages.

Variable Scope Resolution

Python resolves names using the LEGB rule, checking scopes in the following order: Local, Enclosing, Global, and Built-in. If a name is not found in any of these scopes, a NameError is raised.

Closure Concepts

A closure occurs when a nested function remembers the variables from its enclosing scope even after the outer function has finished executing. While powerful for maintaining state, closures can lead to memory overhead because the captured variables are not garbage collected as long as the closure exists.

Concurrency: Threading vs Multiprocessing

Threads share memory, making communication efficient, but the Global Interpreter Lock (GIL) in CPython restricts true parallelism for CPU-bound tasks. Processes bypass the GIL and can utilize multiple cores, but they require Inter-Process Communication (IPC), which adds overhead. I/O-bound tasks benefit from multithreading or asynchronous programming, while CPU-bound tasks benefit from multiprocessing.

Key Differences Between Python 2 and 3

Major changes include print becoming a function, strings defaulting to Unicode, integer division returning a float (using // for floor division), and the removal of long in favor of a unified int. The range function now returns an iterator instead of a list.

Monkey Patching Techniques

Monkey patching involves dynamically modifying code at runtime without altering the source file. It is useful for hot-fixing or replacing modules (e.g., using u instead of ), but it can lead to maintenance issues if overused.

import 
import u

.dumps = u.dumps
.loads = u.loads

Method Resolution Order

Python uses the C3 linearization algorithm to determine the method resolution order (MRO) in multiple inheritance scenarios. The MRO ensures that a method is called consistently, favoring a breadth-first-like search in diamond inheritance structures.

class BaseX:
    def identify(self):
        print('X', end='')

class BranchY(BaseX):
    def identify(self):
        super().identify()
        print('Y', end='')

class BranchZ(BaseX):
    def identify(self):
        super().identify()
        print('Z', end='')

class DerivedW(BranchY, BranchZ):
    def identify(self):
        super().identify()
        print('W', end='')

obj = DerivedW()
obj.identify()  # Outputs: XZYW

Evaluating Postfix Expressions

Postfix notation (Reverse Polish Notation) can be evaluated using a stack. Operands are pushed onto the stack; when an operator is encountered, the required operands are popped, the operation is performed, and the result is pushed back.

import operator

def compute_postfix(expression):
    operations = {
        '+': operator.add,
        '-': operator.sub,
        '*': operator.mul,
        '/': operator.truediv
    }
    stack = []
    for token in expression.split():
        if token.isdigit():
            stack.append(float(token))
        else:
            b = stack.pop()
            a = stack.pop()
            stack.append(operations[token](a, b))
    return stack.pop()

String Replacement Strategies

Basic replacements can use the string replace method, while complex patterns require the re.sub function.

import re

text = 'hello, world!'
print(text.replace('o', 'O'))

pattern = re.compile('[aeiou]')
print(pattern.sub('#', text))

Performance Profiling

The cProfile module measures where execution time is spent. For line-by-line analysis, the third-party line_profiler tool provides detailed insights when the @profile decorator is applied to the target function.

Random Number Generation

The random module offers various functions: random() for floats in [0.0, 1.0), uniform(a, b) for range floats, randint(a, b) for range integers, shuffle(x) for in-place list shuffling, choice(seq) for picking a single element, and sample(pop, k) for picking unique elements.

Thread Pool Architecture

Thread pools reduce the overhead of thread creation by maintaining a set of reusable threads. A manager assigns incoming tasks to idle threads, marking them as busy. When a task completes, the thread returns to the pool. This space-time tradeoff optimizes resource usage for applications handling numerous concurrent I/O operations.

Common Exception Types

KeyError occurs when a dictionary key is missing. TypeError arises when an operation is applied to an inappropriate type. ValueError is triggered when an operation receives an argument with the correct type but an invalid value (e.g., int('abc')).

Mutable Default Argument Pitfalls

Using a mutable object like a list as a default argument shares the same instance across function calls. Modifications persist, leading to unexpected behavior.

def append_element(new_item, collection=[]):
    collection.append(new_item)
    return collection

list1 = append_element(10)
list2 = append_element(123, [])
list3 = append_element('a')

print(list1)  # [10, 'a']
print(list2)  # [123]
print(list3)  # [10, 'a']

Handling Large Files

To read files larger than available memory, process them in chunks using an iterator or by specifying a read size, avoiding loading the entire file into RAM.

def process_large_file(filepath):
    with open(filepath, 'rb') as f:
        for chunk in iter(lambda: f.read(4096), b''):
            pass  # Process chunk

Modules and Packages

A module is a single Python file, while a package is a directory containing modules and an __init__.py file. They prevent naming collisions by providing distinct namespaces.

Coding Standards and PEP-8

Adhering to PEP-8 ensures code readability. Key rules include using 4 spaces for indentation, limiting lines to 79 characters, using snake_case for variables/functions, and PascalCase for classes. Avoid mutable default arguments and always import modules at the top of the file.

Property Decorators and Name Mangling

Properties (@property) control attribute access. Attributes prefixed with double underscores (__) undergo name mangling, making them harder to access directly from outside the class. However, assigning to a mangled name outside the class creates a new instance attribute rather than modifying the original.

class Shielded:
    def __init__(self, val):
        self.__data = val

    @property
    def data(self):
        return self.__data

obj = Shielded(1)
obj.__data = 2
print(obj.data)   # 1
print(obj.__data) # 2

Sorting Dictionaries by Values

Sorting a dictionary by its values is achieved using the sorted function with a lambda as the key argument:

prices = {'AAPL': 191.88, 'GOOG': 1186.96, 'IBM': 149.24}
sorted_tickers = sorted(prices, key=lambda t: prices[t], reverse=True)

Using Named Tuples

The namedtuple from the collections module creates tuple-like classes with named fields. They are memory-efficient, immutable, and enhance code readability by allowing access via names instead of indices.

from collections import namedtuple

Card = namedtuple('Card', ['suit', 'rank'])
first_card = Card('Hearts', 13)
print(first_card.suit, first_card.rank)